cpet/openbsd-ports
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
100,581 Commits 1 Branch 0 Tags
Commit Graph

581 Commits

Author SHA1 Message Date
espie
5a357726a5 rearrange order of functions 2013-10-03 13:12:28 +00:00
espie
f2a006e3f4 code is now specialized enough 2013-10-03 12:55:44 +00:00
espie
5624cc905d disentangle some of the specific "build" code (installation and rebuild info) 
from the fetch subengine (in order to make it simpler to track the Fetch issue)
 2013-10-03 12:46:51 +00:00
espie
9dc08b3323 The "Engine" code is quite daunting, but in reality there are 3 things in 
there.

Finally split them for less confusing files.
 2013-10-03 08:03:27 +00:00
espie
279b28193c maybe want less long command lines eventually, push that out to be able 
to specialize on each task
 2013-10-03 08:02:55 +00:00
espie
43ca285162 gc dead code 2013-10-02 09:16:01 +00:00
espie
c55344661b exec even with !env 2013-10-02 09:14:29 +00:00
espie
ecdd3d280d turn on recorder by default 2013-10-02 09:13:27 +00:00
espie
bce6d9a4b1 split off most of the term display code to be able to reuse it 2013-10-02 09:02:40 +00:00
espie
4a5c4d7329 clean-up code a bit, move state into its own file 2013-10-02 09:00:34 +00:00
espie
ca08cdd7cf oops, fix -x case. 
reported by rd@thrush.com and seen on mirror-maker's emails...
 2013-09-30 19:08:35 +00:00
espie
53756512bd a few more options to limit forwarding 2013-09-25 08:49:49 +00:00
espie
399887a1bc tweak the squiggles to prefer ports with depends, up to a point 2013-09-25 08:49:07 +00:00
espie
1c7ed2817a let RECORD be subject to expand_path and normal parameters handling 2013-09-25 07:01:35 +00:00
espie
c845b65856 zap an extra shell for most remote/chrooted runs 2013-09-24 10:07:27 +00:00
espie
839dbe9437 sigh... if I want probabilistic results, cores MUST unsquiggle on a regular 
basis.
 2013-09-23 14:50:54 +00:00
espie
72779a9859 option: record terminal display, so that I can watch it later at hispeed... 2013-09-23 14:25:12 +00:00
espie
88aad66bcf very stupid logic error 2013-09-23 14:21:47 +00:00
espie
3fc3e4b5a2 allow non-integral values for wantsquiggles 
uses random to devote "part of a core" to running squiggles instead
of normal jobs.

(might be useful for finetuning clusters with lots of dual-core boxen)
 2013-09-23 13:00:02 +00:00
espie
44cd165892 make WRAPPER_OUTPUT disappear unless we are using it 2013-09-23 12:32:59 +00:00
espie
2151c24660 simplify a bit: slaves don't need the socket timeout/keepalive options. 
It only pertains to the master anyways
 2013-09-23 12:26:06 +00:00
espie
f0642c900e trim command-line: TRUST_PACKAGES is on by default these days. 
more accurate name: has_depends -> need_depends
 2013-09-23 12:25:17 +00:00
espie
1b515059dd show a '+' when we're building in-memory 2013-09-22 06:29:38 +00:00
espie
024e2893d8 the aliasing issue was hiding an other bug. 
clean up code a bit, letting cores tell us whether they can be swallowed
or become squiggles, so that fetch cores can choose not to take part at all.
 2013-09-21 11:31:19 +00:00
espie
449966f4cc wrong exec, need some work. 2013-09-21 09:20:17 +00:00
espie
c9b2bb1baa new properties: repair and nochecksum (on by default) 
control aspects of pkg_delete/pkg_add runs
 2013-09-21 08:56:43 +00:00
espie
253a1be562 activate squiggles(~): 
based on a suggestion by naddy@, reserve some cores on big hosts (>=4cpu)
to build the queue in reverse.

Those cores show up as ~ in the core list.
This gets rid of a large part of the exponential tail, where we spend
a lot of time waiting for locks...
 2013-09-21 08:46:06 +00:00
espie
c22694ed9b introduce squiggles: special cores used for building small ports. 
each host takes care of keeping its squiggles "alive". This may
include moving the squiggle property when the core gets swallowed,
and making sure listing/fetch/update distinfo cores are not squiggles.
 2013-09-21 08:44:32 +00:00
espie
5378aa5ff0 nail down distant hosts/chroot to go faster: use explicit /bin/sh, and 
sprinkle an exec in front of the last command of the sh -c, so that we
don't use up that many processes
 2013-09-21 08:41:55 +00:00
espie
36514ef156 avoid creating a core with fullhostname for localhost, preventing some 
unpleasant aliasing effects
 2013-09-21 08:40:09 +00:00
espie
4fec07f96a fix typo 2013-09-18 21:50:12 +00:00
espie
e804dca486 use a "shell" to run getvars, so that we can run it again LATER. 2013-09-18 15:30:40 +00:00
espie
ebc476364a leaner messages: don't show percent on tasks where it doesn't make sense. 
replace "unchanged for 28 seconds" by "frozen for 28s"

remove frozen message for "waiting-for-lock" entirely... there's already
a depend/junk task that shows this clearly, and the message tends to be
WAY too long anyways.
 2013-09-18 13:26:39 +00:00
espie
51cad93976 chroot= 
or
chroot=/
mean no chroot.
 2013-09-18 13:07:32 +00:00
espie
8ee7ce33ed fix unlock early condition for multi-package 2013-09-16 21:29:13 +00:00
espie
82235bd10f move the shell running code up to host. Makes no sense to have the same 
data structure for each Core, and a lot of this is actually tied to the
host proper.

(note that Core::Distant no longer makes a lot of sense)
 2013-09-16 11:23:50 +00:00
espie
f2c8a35b64 tweak time handling a bit, introduce a "real" vs "perceived" time in stats 
for people like me who do ^Z a lot.
 2013-09-14 14:58:12 +00:00
espie
12be45691d remove remains of nfs wait_timeout. 2013-09-14 09:42:11 +00:00
espie
59801ba696 next step in chrooting, mostly refactor 
- provide a chroot_user definition during prop finalize
- let shellclass be depend on the object built
- have all shells store prop and have access to it
- move the chrooted code into default core shell
- tweak running so that quoting/building commands happen everywhere
- change Config to recognize -B as a chroot override.

Tested thru distant chroot, local and distant !chroot
local chroot still not working, probably missing something stupid
 2013-09-11 10:44:13 +00:00
espie
66d4d051d1 make it possible to run distant hosts in a chroot 2013-09-09 14:59:53 +00:00
espie
b09a05b63a common code 2013-09-08 19:30:12 +00:00
espie
3ee5353b60 explain about sudo to the Core runner. 2013-09-08 11:10:58 +00:00
espie
e4abcadef0 even with Time::Hires, always log ts in the engine without decimals, 
makes for crappy logs otherwise...
 2013-09-06 17:11:01 +00:00
espie
df79447c1c quick fix for dpb -x... dirty, refactor needed 2013-09-06 15:58:09 +00:00
espie
63daf24a0e reorganize code, so the periodic sleeper task no longer knows 
its sleeping interval, but rather asks it from the reporter.

Give limiter customer access to timing information. Eventually,
the reporter could possibly shrink/enlarge the display timeout
automatically.
 2013-09-03 09:44:05 +00:00
espie
fc9e210a0f pretty unlikely, but the Limiter measurements may straddle a ^Z/fg, so 
it's better to adjust the timings there as well.
 2013-09-03 09:34:23 +00:00
espie
9eae21b57a error out if we can't open log files (got a weird permission error linked 
to a half-baked file-system).
 2013-09-03 08:53:49 +00:00
espie
1ecc61b2c8 so new_path relies on the pkgname being there, but the engine proper doesn't. 
confuses the hell out of some people (hi naddy!) as dpb restarts will notice
built packages regardless of the path.

make things more symetric by doing the check each time we go through tobuild.

I have some misgivings about this, since this is potentially expensive...
 2013-08-28 12:00:39 +00:00
espie
b3962116eb some features are not tested often enough. 
fix "stop-host".
problem noticed by Florian Obser
 2013-08-28 11:55:06 +00:00
espie
ad7e6062f1 make sure the new nfs watch stuff works (unconfuse engine/subengine 
relationship) now that landry@ can trigger this reproducibly.
 2013-08-07 06:56:42 +00:00