3706 Commits

Author SHA1 Message Date
espie
6525c91219 sigh... if we force junking, we untaint.
but that's because we're tainted! so taint correctly
before we release the lock.
2013-10-13 19:57:07 +00:00
espie
b404b0376e slightly more intricate hack, better colors, that span full lines when they're
supposed to...
2013-10-13 19:19:53 +00:00
espie
a4f0496bb8 add optional colors with -DCOLOR. 2013-10-13 18:32:58 +00:00
espie
8877767529 - use separated Heuristics::Size object
- choose size/nosize in Config.
- use affinity or affinitystub
- move locker creation to State, so we can init cores right away.
- as core init has run, we can call DPB::Core->reap (event loop) while
processing size and build stats, so that the startup script may start
slightly earlier.
- setup more options in Config for Reporter to use.
- scaffolding in SubEngine/Build to be able to use affinity info right away
2013-10-13 18:31:50 +00:00
espie
4bd97c750a add (for now) stub code to provide a specific queue order when dealing
with affinity.
2013-10-13 18:24:24 +00:00
espie
e0c02a1816 the "build-in-memory" stuff is independent from the rest of heuristics:
split it off, and provide a "stubbed" version of Heuristics::Size and
of Affinity  that do nothing, to be used on one host configuration without
in-memory builds.
2013-10-13 18:23:35 +00:00
zhuk
a3b03a2014 Disable/improve some Python-related checks, after discussion with fgsch@
a long time ago. Namely, do not warn about .pyc/.pyo without .py, it's
perfectly legal.
2013-10-13 16:54:24 +00:00
zhuk
a87fa44b81 Add checks for some files having newline and EOF in MESSAGE, DESCR and such. 2013-10-13 16:47:18 +00:00
espie
769e3b4c10 more sensible defaults: if at least one host uses mem, turn on WANTSIZE,
unless we explicitly turn that off.
2013-10-13 10:34:55 +00:00
espie
80a00c8b0a the speedfactor case needs access to the weights. 2013-10-12 14:11:23 +00:00
espie
500c0733fc move the speedfactor special code out of the way and load it only
on demand.

Mostly for clarity, the speed/size difference isn't expected to be much.
2013-10-12 13:53:35 +00:00
espie
20bd660519 tweak order: if we force junk, this *must* run before depend, as
we're depending on junk to remove tagged stuff (and otherwise depend
won't run successfully).

This is different from "normal" junk, which does run after depend as
an optimization (makes little sense to remove packages we might be
reinstalling right away)
2013-10-12 13:52:03 +00:00
espie
0d96f0a882 document -e option 2013-10-12 10:19:20 +00:00
espie
6012cf8ec5 option -e for extra conflicts 2013-10-12 10:17:13 +00:00
espie
754ad67afc fix restart issues...
first tag we see will trigger a junk, unless we already
junked on that host during this run.

that way, we can forget whether we were building kde3 or kde4,
as we start with a "clean slate".

I could have junked at start, but this is much better, as it junks
"just in time" for the tag.
2013-10-11 11:51:04 +00:00
espie
f4bfcdb01d trying to figure out the spinning... 2013-10-11 10:26:44 +00:00
espie
26f524a63b well, setup doesn't redirect, so print in the right location instead
of on console...
2013-10-10 20:52:04 +00:00
espie
d93a5e545e oops, reversed value while copying test.
do the nojunk check in setup (so that we don't untaint if we're
not gonna run), and since we're testing in setup, use that to bypass
running altogether in such a case.

sorry landry, previous one was pretty bad... the reversed test result
is a stinker...
2013-10-10 15:48:16 +00:00
espie
c28f1287bc explain a bit more about serialized locking.
keep track of where we last ran junk on a host, so it's faster to backtrack
thru logfiles...
2013-10-10 10:48:35 +00:00
ajacoutot
a70ff0dc7b Add _gnome-initial-setup. 2013-10-10 08:40:34 +00:00
espie
d65a77a71b extra test for non_empty ? grasping at straws 2013-10-10 07:26:40 +00:00
espie
a1fa92f3db setup of serialized tasks *is* deterministic.
The moment when we're sure we're locked in uninstall
is the right moment to revisit tainted status...
not later, because if we do that in finalize, some tainted ports
may have finished building, and we may think we're untainted now.
as seen by landry@

Note also that setup runs in the main dpb process, so not concurrently with
the engine, and so the engine can safely check for tags right after setup,
even if pkg_delete has not finished, because by the time the port can lock
and reach depends, pkg_delete must have finished.
2013-10-10 07:15:49 +00:00
espie
722d74cc49 this should work around the spinning loop in fetch.
I still don't understand how it happens, since the queue
ought to be empty ?
2013-10-09 06:21:38 +00:00
espie
14c5104e65 have start return useful information, like whether it started something
or not, and propagate it thru the engine
2013-10-09 06:20:56 +00:00
espie
c17299861e on restart, if we find old locks with tags, those hosts start tainted 2013-10-08 17:40:41 +00:00
espie
5cdb3d9619 possibly allow us to taint cores near startup
stringize pids differently, so that we don't append an extra " " to localhost
2013-10-08 07:35:10 +00:00
espie
6486cfbb4d we no longer create specific local cores. Let install happen again by fixing
the test.
2013-10-07 20:36:27 +00:00
espie
0b4eda1a8c remove some scaffolding. Still exit violently if the former issue happens
again...
2013-10-07 20:23:39 +00:00
espie
7e98cfa567 don't mark_ready a core that runs something (seen by nigel and rpe).
replace the dirty trick I did to reuse start with an intermediate
method that's more "reentrant" (use_core) by not fiddling with core's
status.
(bug appeared thanks to over-eager refactoring and shoddy code...)
2013-10-07 20:23:13 +00:00
espie
37f400f8fb revisit, fetch cores MUST be pre-emptable for rebuilding-info,
otherwise pure -F fetch engine loses !
2013-10-07 20:01:55 +00:00
espie
0fd016ecd3 fix typo 2013-10-07 19:49:56 +00:00
espie
ec6775313c no need to give back the extra job in fetch_only now,
as it's not going to get used by the builder.
2013-10-07 19:27:25 +00:00
espie
9ab9a985bb debug scaffolding, abort early... I expect mark_ready gets called at the
wrong location or something :(
2013-10-07 18:02:07 +00:00
espie
a0a1e6b657 okay, fixed version where NoBuild subengine gets only what it actually
needs
2013-10-07 18:01:33 +00:00
espie
3e70a9c3c7 move decision to run show-size to PortBuilder, as we really need to know
whether we have a filehandle.
2013-10-07 17:50:29 +00:00
espie
959d2592a0 also protect there... 2013-10-07 06:40:21 +00:00
espie
ce99ea0d94 funny 2013-10-06 17:21:53 +00:00
espie
28006064d8 fix -F mode... subengine not yet ready to be split,
and we DON'T want an extra build job.
2013-10-06 15:49:22 +00:00
espie
0f33e01c84 special-case single core/single job: create a clone so that we start
building stuff in parallel with LISTING, and fold back LISTING at end.
2013-10-06 14:03:07 +00:00
espie
094b349a67 name tricky construct 2013-10-06 14:01:52 +00:00
espie
d9dda2ec11 provide hostcount, to be used to make some decisions later, like
whether we want to trigger affinity, or if we want listing job to be extra.
2013-10-06 13:53:12 +00:00
espie
40185ee800 NoBuild actually requires one piece out of Build... 2013-10-06 13:48:27 +00:00
espie
b46cee4c56 update my copyright to 2013 2013-10-06 13:33:24 +00:00
espie
885e9b335f and portbuilder too 2013-10-06 13:25:15 +00:00
espie
27212e543e split some code into separate files, so that it doesn't get
loaded if it's not needed.
2013-10-06 13:22:18 +00:00
espie
5d6d15a31f create a special 'in-between' task that runs (setup only actually)
between checksum and extract: that's when we're truely committed to
building the port there.

create the affinity markers at that point, and not sooner.
since we know whether we're building in memory, we can record that.

if we're building in-memory, always ask for size, since it's very very
fast.

special treatment for tagged ports on error: mark them as "cleaned" so
the host gets untainted eventually.
2013-10-06 12:40:43 +00:00
espie
7ef4b92ee4 rename "special" parameter to what it really is, building in memory.
return actual size for affinity files.

assume affinity files record previous choice and reuse it
2013-10-06 12:38:23 +00:00
espie
61f45b05c9 extend affinity info to also include memory consumption for in-memory ports 2013-10-06 12:37:05 +00:00
espie
22bb96f2a1 Nigel saw dpb burp out because of missing job name...
allow it to continue, no real idea what's wrong yet. :(
2013-10-06 12:36:12 +00:00
espie
71134ea839 well, turns out awaiting-locks can be fairly useful 2013-10-06 10:58:20 +00:00