cpet/openbsd-ports
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
70,883 Commits 1 Branch 0 Tags 554 MiB
e88239a213
Commit Graph

118 Commits

Author SHA1 Message Date
espie
72c6360806 optimize heuristics for fetchqueue, as they take too much time 2011-06-01 12:34:29 +00:00
espie
647950c27f typo, duplicate work 2011-06-01 12:34:09 +00:00
espie
1e22893e8e if we -f -a, then scan SUPBDISTFILES too, and use the Distfile cache 
to write a list of all known distfiles, that can be used to remove extra
stuff from /usr/ports/distfiles
 2011-05-29 11:06:23 +00:00
espie
84266ee8ac if we're running -f, do the checksum part internally, since we already 
checksummed stuff we fetched (will allow auto-refetch eventually).
 2011-05-29 09:30:13 +00:00
espie
aeae163983 oops, the Builder is one indirection down now 2011-05-28 19:32:59 +00:00
espie
3f2603c2f3 fix unlocking in case of errors 2011-05-28 19:24:15 +00:00
espie
9f989ffafe fix 2011-05-28 08:21:39 +00:00
espie
f4d6937978 nicer lockname 2011-05-27 13:13:43 +00:00
espie
2fd23899dd explain the dump_dependencies/find_best optimization for quick-restarting 
dpb.
 2011-05-27 11:22:13 +00:00
espie
7ef05495f5 start moving the fetch/build code into separate subengine classes, to make 
it more obvious how much is shared (goal is to unify a large part of these).

In particular, the locking code/stuff already active is glaringly the same...

Polish the distfiles measurements. For now, 3 heuristics that kick in
at various stages of the build (a bit of black magic).

Fix pkgpath->{has} to record both "no dependencies" and "several distfiles
missing" to avoid grabbing small patchfiles which don't lead to valid
builds while heuristic#1 is active.
 2011-05-27 10:27:50 +00:00
espie
a819e7f3e8 improved fetch: do checksum as a separate process, to avoid freezing 
dpb while checksumming huge tarballs.
Use ftp -C in a systematic way, more complicated logic as to when to
remove temp files, when to keep them: if core exited okay, wrong size
is very bad. If checksum failed on a partial fetch, retry same site with
an empty file...
 2011-05-23 09:44:05 +00:00
espie
0c1194b39b only check checksum if we just fetched it, for now. 2011-05-22 09:19:08 +00:00
espie
d2a3a0fa83 dyslexia 2011-05-22 09:06:49 +00:00
espie
fbfc4fba47 fix checksum logging, avoid checksumming twice 2011-05-22 09:01:27 +00:00
espie
8cef021866 initial support for fetching distfiles, somewhat experimental yet. 
- clean up clock handling, jobs with timer.
- pass more parameters thru state
- tweak the engine to handle distfiles
- lots of new code in Fetch for new jobs.
 2011-05-22 08:21:38 +00:00
espie
afdb81839a finish refactoring options. 
Deprecate: -t and -T
use -DCONNECTION_TIMEOUT=... -DDISPLAY_TIMEOUT=... instead
New feature: -DSTUCK_TIMEOUT=
kill tasks when they don't show any progress for that long.
 2011-04-25 11:58:46 +00:00
espie
e7a4ca8c8a ignore whitespace 2011-04-18 11:40:23 +00:00
espie
953361b9c4 in case we log errors, go to the next iteration correctly. 
log the fullpkgpath explicitly when we can.
 2011-04-05 08:24:34 +00:00
espie
d76bdca371 tweak to allow subclasses of cores that get their own separate lists 
(such as local fetcher jobs)
 2011-03-22 19:49:56 +00:00
espie
cabe19ad60 tidy, minor 2011-03-22 19:48:53 +00:00
espie
fc9fbe61ee add the glue that parses fetch info and build distinfo for future fetching. 
also let Vars report stuff it doesn't know about to the logs file.
 2011-03-22 19:48:01 +00:00
espie
c6ed94f022 not sure why it happens, but log nopkgname... 2010-12-12 21:21:03 +00:00
espie
97286a5a18 better info rebuild: make Vars responsible for creating info "just in time" 
by keeping local state.
That way, when we rescan, we don't have to remove any info, and all affected
infos are replaced at the same time.
 2010-12-07 10:56:26 +00:00
espie
9c270c05b6 fix fix 
$build_info //= 1  indeed... ;(
 2010-12-06 19:35:06 +00:00
espie
ab01800701 tag dependencies discovered as pure dependencies with their parents, allow 
finding out non-sensical stuff in an easier way
 2010-12-06 13:20:45 +00:00
espie
d317d925fe minor warning fix 2010-12-06 13:19:44 +00:00
espie
9ed0d9cd66 move most of libtool code around. 2010-12-05 16:37:50 +00:00
espie
4d8b6b0032 allow 'dpb archivers' without extra artifacts, e.g., recognize and 
handle categories without classifying them as pkgpaths with errors.
 2010-12-05 10:06:12 +00:00
espie
1d73dd9276 also add older LIB_DEPENDS to WANTLIB 2010-11-22 21:16:42 +00:00
espie
1c0a277e2c prepare to deal with new depends 2010-11-14 07:44:53 +00:00
espie
e2d1250c4c show everything as WANTLIB for now, this needs more love obviously 2010-11-11 12:43:03 +00:00
espie
47bc97ea5c fix stupid warnings 2010-11-11 12:17:20 +00:00
espie
82ae79eeca in case the signature doesn't match, clean ALL packages that will be 
rebuilt. Otherwise, you may suffer through 3 or 4 rebuilds of qt3...

ouchie.
 2010-11-02 20:32:59 +00:00
espie
fb3a3c80f7 much better implementation of -R. In case of rebuilds, check does not "see" 
pkgfiles, unless they've been registered.
the engine queues everything to build, but normal build is only an
existence/signature check. The actual build tasks only get queued when
the package does not exist, or if its signature does NOT match the ports
tree.

Should make matthieu@ happy.
 2010-11-01 10:55:26 +00:00
espie
8a26786deb new file/option: save dependencies and be able to reuse them. 
makes for a "quick listing" of the most important ports first (e.g., groff).
document -R, even though it's not perfect yet.
flesh out all the recent parts and update the BUGS AND LIMITATIONS section.
 2010-10-31 11:07:20 +00:00
espie
f7f23f3bdd don't cheat, we have to make clean=package to avoid side-effects with 
arch-independent packages.
 2010-10-30 11:36:07 +00:00
espie
f072ab3a17 proof-of-concept implementation of -R (rebuild when signature changes). 
Somewhat inefficient (deep check of package vs port signature).
More proper approach would be to tell the engine to always build, have first
step be a check for signature if the package exist, and abort if it matches:
- here we double check everything
- propagating rebuilds means non-uptodate packages give rise to non uptodate.
 2010-10-30 11:19:38 +00:00
espie
86be07b384 simplify parameter passing slightly 2010-10-30 10:35:09 +00:00
espie
03b44c7dac refactor to make most of it reusable to e.g., make print-package-signature 2010-10-29 12:35:00 +00:00
espie
10e6cc104f engine improvements: 
- if several cores are available, recheck errors, so that we're not stuck
waiting for a core to finish to start new shit.
- $logdir/stop-<machine_name> allows you to stop a given machine while
you clean it up and (possibly) restart it (e.g., won't start new jobs)
 2010-10-29 11:51:42 +00:00
landry
66327fcf42 Readd missing space before "on $hostname" 2010-10-28 22:14:54 +00:00
espie
1f255bf2b5 not needed any longer 2010-10-28 16:40:48 +00:00
espie
1c010712c9 keep the output from dump-vars around, and save it in the logfile is 
something bad happens.

get rid of special reporter for Vars, E= is enough, people should look
in the paths/logfile now.
 2010-10-28 14:54:38 +00:00
espie
39fe683365 of course, confusing prepare and depends is bad 2010-10-28 14:21:18 +00:00
espie
e283301fe2 zap localhost from those lines too. 
will make display snappier on one core machine, and still be readable
otherwise...
 2010-10-28 12:51:37 +00:00
espie
7cf1b65950 do the next step even if prepare fails, since prepare is fairly rigid 
and depends will allow more flavors.
 2010-10-28 11:56:48 +00:00
espie
4a572ee2bd make it possible (simple) to rescan broken directories 
(we should possibly log the error messages).
Forgot to reset broken, so rescanning didn't help...
 2010-10-28 10:33:20 +00:00
espie
634fb00640 oops, give back that core. 2010-10-28 08:54:22 +00:00
espie
d1fc80b3fe use rescan after errors, so that bumps get picked up 2010-10-27 22:58:02 +00:00
espie
309af926e4 kill more code related to P= (will simplify next parts) 2010-10-27 22:53:32 +00:00
espie
d4ee90931f tie engine with var grabber, get ready to re-do port meta on error. 2010-10-27 12:58:26 +00:00
espie
8557f6f587 distinguishes between errors we create (E=) and locks we encounter (L=) 2010-10-26 16:57:10 +00:00
espie
1fecde2518 rework PkgPath to distinguish between empty and default flavors... 
turns out we probably don't need FULLSUBDIR after all.
compute SUBDIR better, possibly with quotes.
Remove trailing / on subdirs, in case people use completion.
 2010-10-26 15:45:09 +00:00
espie
cce4985d69 refactor. fix yet another bug (always return something) 2010-10-25 17:31:25 +00:00
espie
7a9e9b48bd use same pattern for oldlines. 
(actually wrote this and the next few commits at the spa, in my head.
Now I just need to type them in and commit).
 2010-10-25 17:23:13 +00:00
espie
e25225b1b1 display fix. I'm a moron 2010-10-25 10:46:11 +00:00
espie
663157b0c7 add an option for dpb to keep going if there are errors left. 
there's a bug in the display (to be fixed) as errors won't show properly
and you have to look manually though...
 2010-10-24 10:01:57 +00:00
espie
5d8d9e2600 if we didn't parse the path, then we don't have the pkgname, duh. 2010-10-23 21:46:03 +00:00
espie
721c339299 zap P/I distinction, it was a lousy idea anyways 2010-10-23 18:14:19 +00:00
espie
e694d95fa7 fix loop: we know we tried it, it didn't work, so don't try it AGAIN. 2010-10-23 18:02:59 +00:00
espie
7439821562 if we don't ssh to it, don't print the name either in errors, as requested by 
landry@
 2010-10-23 17:58:55 +00:00
espie
9aa92a99d4 saner: always go thru a DumpRecorder, and perform full save/retrieve from 
a dump recorder.
 2010-09-13 14:38:05 +00:00
jasper
04abd53ab1 - revert previous, it breaks builds for at least two builders. 
discussed with landry@ and naddy@
 2010-09-02 11:16:09 +00:00
naddy
807d4ad3da fix overquoting: we want to exit with 1, not a "exit 1" command not found error 2010-08-24 15:07:54 +00:00
espie
04cb4e82a5 move a large part of check-lib-depends into separate modules 2010-08-20 15:29:41 +00:00
espie
9b38b584e9 better namespace for make-plist internals, prepare to cut up check-lib-depends 2010-08-20 15:22:21 +00:00
espie
8409f0fef7 copy some stuff to the new organization, rename find-all-conflicts to 
check-conflicts for consistency
 2010-08-20 13:50:11 +00:00
espie
4c6d1b431f move stuff here from infra/build 2010-08-20 13:40:13 +00:00