cpet/openbsd-ports
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
80,642 Commits 1 Branch 0 Tags 554 MiB
4bca9f9bf5
Commit Graph

305 Commits

Author SHA1 Message Date
espie
681c718ee7 kill checksum-and-list 
optimize away some depend stages: since the main dpb process maintains
a synchronous list of live_depends, and only releases host locks synchronously,
it can check depends lists against already known live_depends (and avoid
some depend stages entirely).

naddy@ says it gains 40 mn on his box...
 2013-01-18 21:11:55 +00:00
espie
06112d2478 log the time spent waiting for locks. 
remove most of the "list distfiles" ad-hoc code, this stuff belongs in
bsd.port.mk.
 2013-01-16 10:38:56 +00:00
espie
3db511776e ChecksumAndList: unused yet. make it possible to save archive-file-lists 
for later perusal (e.g., locatedb for distfiles)
 2013-01-13 14:04:00 +00:00
espie
14e4155cd7 zap SUPDISTFILES earlier if not needed. 
framework to save listings of archives
 2013-01-13 14:03:20 +00:00
espie
e6106723a7 work-around for special need users 2013-01-12 14:44:41 +00:00
espie
dca439d6cc fix problem noticed by nigel@ 2013-01-12 13:51:43 +00:00
espie
e6c4598eeb inheritance works better when you don't forget to call superclass 2013-01-11 21:16:08 +00:00
espie
5b842818db belt and suspenders... don't know why, but sometimes, the stuff is not 
where I think it should be
 2013-01-11 20:11:54 +00:00
espie
a7fad66add show how many locks we waited for, that's easy to do and always fun 2013-01-11 17:39:03 +00:00
espie
0cd153f7f8 ... and tweak to waiting-for-nfs so that things are clearer 2013-01-11 17:35:43 +00:00
espie
a6c88d2fad explicitly wait for junk_lock... doesn't happen that often, a lot 
of try_lock will succeed, so the extra processes are cheap
 2013-01-11 17:23:51 +00:00
espie
d8b015e058 oopsie, broke junk 2013-01-11 16:59:07 +00:00
espie
4bf0c613db task->name misused, go grab directly phase/name so that it works like 
I want it to
 2013-01-11 16:35:47 +00:00
espie
d5af18631f logic goof 2013-01-11 16:24:26 +00:00
espie
36852b448d ouch 2013-01-11 16:11:21 +00:00
espie
2269ffdd0e run handle_job earlier, so we can see which junk_locks occurred in setup. 2013-01-11 15:48:51 +00:00
espie
ee29f76d8a revamp tasks a bit more: have Job::Port let each task do its own setup. 
as a result, try locking directly before the fork, so we can distinguish
tasks that did enter the lock directly from those that have to wait.

(stuff like depend! means it has to wait/had to wait).

Also, allows tasks to just "vanish" after setup: used for checksum (the
whole code is now in there), and for junk (if several ports try to junk
simultaneously).

keep better track of the junk lock thru a job, so that it's released when
we get out of a serialized sequence of tasks (thus, no longer
release/reobtain at end of show-prepare-results/junk).

checksum prints its own distsize now, something that was awkward before.
 2013-01-11 15:35:53 +00:00
espie
49361d348f force when redraw 2013-01-11 13:22:25 +00:00
espie
3b07fda58c make this a little bit shorter and faster. 
(considering how often it's called, yes, it's worth it)
 2013-01-10 22:42:21 +00:00
espie
2ca4447e66 get rid of show-prepare-results temp file, add glue to synchronize to 
normal log.
 2013-01-10 21:41:55 +00:00
espie
643f85b498 create pkgpaths objects is rather costly, so first do a string comparison 
instead of creating the same object again and again.

removes *most* of the calls to PkgPath->create/compose !
 2013-01-10 16:20:10 +00:00
espie
647327b6ba don't go thru patch/configure/build for very small ports. 2013-01-10 12:27:21 +00:00
espie
aa851e66b1 keep a trace of "small" ports: we won't log every detail concerning 
these
 2013-01-10 12:05:55 +00:00
espie
1d0a9bd7bb say goodbye to prepare/show-prepare-results, 
there's no need for a separate step: show-prepare-results should be
practically instantaneous, and if there's noise, prepare will exit(1)
anyways.
just need to explicitly log the output of prepare.
 2013-01-10 12:04:23 +00:00
espie
2abd089617 there's no need to fork a process if the checksums are all cached 2013-01-10 12:00:38 +00:00
espie
d4a9b9d633 use new functions to simplify 2013-01-10 11:56:53 +00:00
espie
dd11c9037a deleting DIST from child is pointless, do a proper finalize 2013-01-10 10:35:36 +00:00
espie
48c91e1d7f 'memoryhog' property: avoid starting two such jobs on the same host, if 
possible (mostly to be used for moz stuff which gobbles huge amounts of
memory while linking)
 2013-01-10 10:30:13 +00:00
espie
e2925ad3b7 zap info DIST if we don't actually need it. 2013-01-10 10:28:29 +00:00
espie
af0beabf88 a bit of refactoring: provide simple ways to check DPB_PROPERTIES without 
having a hash element spring into existence, and to list jobs running on
the same host.
 2013-01-10 10:26:34 +00:00
espie
ba805a9329 small optimization, fullpkgpath is expensive, and ordering by pkgpath is 
vastly enough in the current case
 2013-01-10 10:25:25 +00:00
espie
d0f75703ed pass -X paths thru: 
- the grabber passes them to PortBuilder
... which builds a hash of pkgnames
... and the port uninstall job excludes these from the list of ports to
junk.

Note this only works with -current pkg_delete, as I had to tell it to
ignore non-existent pkgnames in that context
 2013-01-07 10:59:41 +00:00
espie
7906ac8797 if I run into lockfiles that don't contain the requisite dpb property, 
just hold on until they're there.

Should happen right away, so easy to spot !
 2013-01-06 14:38:14 +00:00
espie
8a23d3ca7e ask the ports tree to clean up stale locks, finally... 2013-01-06 11:59:40 +00:00
espie
a8dab3a559 perl's gc doesn't play well with filehandles. 
So explicitly close what it will have problems tracking...

problem noticed by sthen@, obviously linked to my previous change to
avoid opening the same log file again and again...
 2013-01-05 23:38:08 +00:00
espie
a323bb36ea better 2013-01-05 21:48:50 +00:00
espie
c1c2864b82 give every task a "name". 2013-01-05 21:48:18 +00:00
espie
fe8936deab even if we've got pre-empted, we still have to lock. 
otherwise, we're going to unlock something we don't own, which might
be a race.
 2013-01-05 20:08:39 +00:00
espie
a5eeeec19c store information about live dependencies in dpb proper, to avoid 
synchronization issues with external fs...

the external locks are still used to synchronize with other dpbs running
 2013-01-05 20:06:29 +00:00
espie
7184d32670 retrieve the lists of paths locked per-host from the old lock scanner, 
to eventually be able to clean them in init.
 2013-01-05 19:59:43 +00:00
espie
ba6937b81c and put serialize next to the code it protects 
(again, no actual code change)
 2013-01-05 18:09:30 +00:00
espie
a18e4e7af2 move install out of the way (no code change) 2013-01-05 18:08:06 +00:00
espie
b159503365 log junk lock handling 2013-01-05 17:29:14 +00:00
espie
82fbc0e7fd avoid opening filehandle again and again. 
also, close STDERR after the STDOUT stuff was done, so that we get
error messages otherwise!
 2013-01-05 17:22:04 +00:00
espie
ed58c56937 move the "running prepare" message up to handle_output, so that *every* 
phase gets tagged.
 2013-01-05 16:57:31 +00:00
espie
97912f877b thought I had done this already: 
don't blindly append to history, but recreate it from scratch.
We need to remove lines corresponding to stuff that vanished at some
point but came back later!
 2013-01-05 16:10:18 +00:00
espie
96eed0203f crank factors up to 150, so that dpb consumes a bit less cpu yet. 2013-01-05 13:39:36 +00:00
espie
df5a288315 make sure we zap any lingering socket before we restart our master. 2013-01-04 21:19:18 +00:00
espie
a9b8de6833 switch to IO::File and explicit flush for the engine log. 
should be slightly faster
 2013-01-04 19:34:10 +00:00
espie
d606cc382f another half-baked change 2013-01-04 18:49:39 +00:00