Commit Graph

126 Commits

Author SHA1 Message Date
Satoshi Asami
31fb70c70d Exit with error status in ${portbuild}/status when an error occurs.
Make INDEX build failure a fatal error.

Copy INDEX to INDEX-${branch} for later perusal.

There is no -a flag in OpenSSH's scp.

Use sed to remove /usr/ports and stuff from restricted.sh and
cdrom.sh, so they can be run without further modification using
relative directories.

Take more options, namely -norestr (don't build RESTRICTED ports),
-noplistcheck (turn off "extra files check"), and -nodummy (use
open-motif instead of Motif-dummy and XFree86-4-* instead of
XFree86-3).  -nodummy will become the default soon.  Also a new flag
-nocvs will prevent cvs update on ${branch}/src only (while -nocvsup
will prevent cvsup and cvs update on usr/ports).  -nocvs is to be used
when builds for more than one branch shares a same cvsup session.

Write out time cvsup is done so it can be used by log processing
scripts.

Handle dummy ports by using a more generic approach -- list the
directories they reside in (e.g., x11-toolkits/Motif-dummy) and
generate the package name from there, instead of hardcoding them
package names everywhere.  Hopefully this will go away when we get rid
of the special case handling for Motif and XFree86.

Add something called "quickports" -- these are packages with very long
dependency lists that always seem to throttle the parallel build near
the end.  By spelling them out explicitly, try to make make build
dependencies for them earlier while there are a lot of other stuff to
do.  (Our make doesn't do anything with the target list, but maybe I
can fix that someday.)

Copy bindist.tar to clients before the second run of package build
too.  It can be useful to "sneak" in fixes if something minorly wrong
is discovered in the bindist during the first run.  Since there is a
md5 checking, this copying will not be done unless the files are
actually different.

When -norestr is set, delete restricted packages and distfiles using
restricted.sh immediately after completion of package build.

Generate CHECKSUM.MD5 after packages are built.

Copy distfiles over to the ftp site after everything is done.  This is
run in the background on bak/distfiles so the next package build can
immediately start.
2000-08-29 08:51:57 +00:00
Satoshi Asami
03914a93bd Split build into two phases, checksum and package, and copy the
distfile to the server right after checksum.  This will make it
possible to "harvest" distfiles even for ports that don't currently
build.  (This was a reason for a lot of distfiles being lost in the
past -- someone commits a broken port, distfile doesn't get fetched to
ftp.FreeBSD.org since package doesn't build, and distfile is gone from
the master site before the port is fixed.)  This also enables more
efficient sharing of distfiles since we can set MASTER_SITE_OVERRIDE
to the port build server and have a better chance of finding the
distfiles there.

Instead of using a simple make package | tee, redirect the output of
make package into a separate log file and concatenate it later.  For
some reason, the gconf build always hangs after the build succeeds
(tee doesn't exit).

Handle the new arguments (-norestr for NO_RESTRICTED, -noplistcheck
for NOPLISTCHECK and -nodummy for NODUMMY) by converting them to
environment variables.

Add the branch=5 case.  Run ldconfig on /usr/lib/compat too.
2000-08-29 08:33:05 +00:00
Satoshi Asami
fcb5d2b7fe Instead of doing a pwd | grep (duh!) to find the branch we are
building ports for, pass it as an argument.

Pass NO_RESTRICTED, NOPLISTCHECK and NODUMMY to ssh as flags when said
environment variables are set.
2000-08-29 08:25:14 +00:00
Satoshi Asami
5fc1b924f9 The errors and logs are now in the archive directory named
[ae].${branch}.${date}, and "logs" and "errors" are just symlinks to
them.
2000-08-29 08:23:42 +00:00
Satoshi Asami
c867f8186d Change find command line to find potential security hazards (ports
that install setupd binaries etc.)

Submitted by:	kris
2000-08-29 08:22:39 +00:00
Satoshi Asami
04694e8990 Run build on three branches. Check the status file filled out by
dopackages and abort build if there were any errors.  It intentionally
leaves lock files behind in that case because it usually requires my
manual intervention to fix whatever's wrong (usually a botched INDEX
build due to someone forgetting to update dependency lines) and we
don't want subsequent failed tries to wipe out all the good packages
and distfiles we have accumulated.
2000-08-29 08:21:47 +00:00
Satoshi Asami
a88f6b297c Handle arbitrary number of log directories instead of just two. 2000-08-29 08:18:44 +00:00
Satoshi Asami
c0a421c34b Ignore empty loads files -- it seems scp in OpenSSH sometimes leaves
files empty even if the original target and the new source are both
non-empty.  (Old ssh1 never did that, but whether this is a bug is not
obvious.)
2000-08-29 08:17:08 +00:00
Satoshi Asami
abc8a78dc9 Handle three branches instead of two. Use a for loop instead of just
listing calls to comparelogs etc., so we can expand it into the future
(no, not 4 branches, but maybe another architecture).
2000-08-29 08:14:52 +00:00
Satoshi Asami
fd7f4efa48 Add -f to forcefully umount /usr/src and /usr/obj before reboot. 2000-08-29 08:13:18 +00:00
Satoshi Asami
ccaef40b74 Add branch=5 case and update some variables. Don't set NO_RESTRICTED,
this will be passed down by calling scripts if necessary.

Remove extra spaces in dependency list -- this sometimes confuses make
for some reason.
2000-08-29 08:12:16 +00:00
Satoshi Asami
199691e133 Add branch=5 case and update some variables. Don't set NO_RESTRICTED,
this will be passed down by calling scripts if necessary.
2000-08-29 08:04:13 +00:00
Satoshi Asami
e8c4933e13 Print uptime after the number of jobs running. This has no functional
effect since the job scheduler will only look at the first field, but
will give humans looking at the load reports some idea about the
machines (especially crashes).
2000-08-29 08:03:13 +00:00
Satoshi Asami
a3e9efec50 A little script that tries to "undo" a package build. Basically just renames
a bunch of directories back.  Use it when you want to void a botched build.
2000-08-29 08:00:51 +00:00
Satoshi Asami
f4528a378f A small script to copy packages over to the ftp site. It is not integrated
into the main build scripts yet so you need to call this one manually.
2000-08-29 07:58:44 +00:00
Satoshi Asami
0a59f74d84 A small script to copy distfiles from per-port directory to central
location.

This is done to avoid write-on-write race conditions, when two package
builds try to write the same distfile back into the server.  To avoid
this, the distfiles are first copied over to
distfiles/.pbtmp/${PKGNAME}, and then a .done file is created in that

directory.  This script runs on the server periodically to copy the
files to the central location.  Since this script copies one distfile
at a time, the race mentioned above doesn't exist.  Also, it will only
copy files from directories where the .done file exists to avoid
read-before-write race conditions.
2000-08-29 07:57:09 +00:00
Satoshi Asami
8dc4353d1b Remove bak/distfiles after it's done -- this script is now called in
the background with bak/distfiles to work with (so that the new
package build can run in parallel).
2000-08-29 07:52:55 +00:00
Satoshi Asami
7173c62988 (1) Use timestamp file to make log processing lazier -- only process
logs when there are logs newer than the .stamp file.

(2) Print time cvs update is done for reference.

(3) Print number of affected ports (the "aff." field).  Basically the
    number of times the port appears in INDEX minus one (itself), so
    it won't take care of all chained build dependencies, but it will
    give you some idea how critical a port is when looking at the long
    list.

(4) Catch a few more error cases.
2000-08-29 07:50:12 +00:00
Satoshi Asami
dd7b8f01ce Read package build log files and compile a list of "extra" files in an
HTML table format.
2000-08-29 07:45:02 +00:00
Mark Ovens
c8794db7c3 New script (and it's README) for searching the ports tree. Add this script
to the main README.

This was originally submitted as a port but was added to Tools/scripts
instead at alex's suggestion.

PR:		20863
Approved by:	alex
2000-08-28 11:37:23 +00:00
Will Andrews
ad8939a4f6 Make -n work again. Add new option -s to allow people to specify a
$DISTDIR to put distfiles in, if they can't touch $PORTSDIR/distfiles.
If this is freefall, default to $tmpdir, which is relatively secure since
most users on it are trusted.  Reset FETCH_BEFORE_ARGS to "-btA" instead
of "-btsA", so the distfile is actually fetched.  Add a method to allow
the name of the module to be changed if -i is used.  Update to use the
modulesupdate that's in /usr/local/bin.

Addport should now work properly on freefall, without -v option.

Submitted by:	roger (-s, $tmpdir by default on freefall)
Thanks to:	peter (copying modulesupdate to /usr/local/bin)
2000-08-19 08:08:47 +00:00
Will Andrews
727770ca84 Properly check -d argument, and return usage() if it's not there. Also use
Sys::Hostname with hostname() instead of `hostname`.

Hostname stolen from:	CVSROOT/log_accum.pl
2000-08-11 21:44:18 +00:00
Will Andrews
4367619221 Minor cleanups. Print usage() if no arguments are passed. Get rid of
unused $pwd variable.  Remove extra cruft.  Fix comments.

usage() problem submitted by:	knu
2000-08-11 14:31:28 +00:00
Satoshi Asami
ed58e5d6dd Add -nocopy option to prevent bindist.tar from being copied over. Could
be useful between the first and second phases of the build.

Do a "killall make fetch" to clear any remaining processes.
2000-08-10 20:01:42 +00:00
Will Andrews
b1a6849d31 Major update to addport. Many *MANY* bugfixes. Addport should now work
with a directory specified as ``.'' or containing a ``/''.  A new option -i
was added to allow people more control over the port's position in the CVS
repository.  A new feature was added to allow multiple ports to be added in
the same execution of addport.  Addport will now add a port to the category
Makefile properly if it is the first port in a category.

A major reorganization of addport was also performed - now the functions
are at the *bottom* instead of scattered around.  This should allow easier
understanding of the process addport goes through.  I've also added -w to
perl's exec args, and moved to a hash for getopts().

usage() was extended to help explain addport's functionality better.

Bugs submitted by:	nbm
Some help from:		sheldonh
2000-08-09 19:27:29 +00:00
Will Andrews
9f764ffc42 Fix special case where people use ``.'' for the argument to -d. I'm not
very good with perl yet, so anyone who can propose a better way to do
this (with s/// or m// or something using regex) that might also include
the case where the argument contains slashes (i.e. games/somegame).  But
anyways, this should catch folks who use ``.''.  :->

Submitted by:	obrien, sada
2000-08-06 02:51:30 +00:00
Satoshi Asami
92b5be3047 Make this run on ${portbuild}/bak/distfiles rather than
${portbuild}/distfiles so it can be interleaved with the next run of
build.
2000-08-04 07:20:52 +00:00
Maxim Sobolev
dc30f97f73 Add missed '>' to my e-mail. 2000-08-02 07:13:11 +00:00
Maxim Sobolev
c978a0f298 Add gnomedepends.py to our collection of helpfull scripts.
gnomedepends is a script, which analyses pkg/PLIST and gives an advice as to
which GNOME ports should be listes in {RUN,LIB}_DEPENDS for the port to ensure
correct removal of GNOME shared directories.
2000-08-02 07:10:24 +00:00
Maxim Sobolev
2e9b099ca0 Various fixes and improvements. 2000-07-19 15:41:51 +00:00
Alexander Langer
a53e3a9f5f Add hack for gzipped files that are not .tar.gz files and uncompress
them The-Right-Way(tm).

This whole uncompress-algorythm should be reworked, though.
2000-07-19 15:11:26 +00:00
Satoshi Asami
8d9ad9015d Fix benign bug which had the script claiming to delete any package that
includes a missing package's name as a substring (e.g., ja-mh-6.8.4 vs.
mh-6.8.4), when it wasn't actually deleting it anyway.
2000-07-12 23:29:30 +00:00
Maxim Sobolev
3af264c879 Add "-s" (silent) option useful for shell scripts. 2000-07-10 13:40:08 +00:00
Maxim Sobolev
42f3104099 Add checknewver.sh - a shell script which checks for availability of newest
version of distfiles on MASTER_SITES. This is still work in progress, so
please use with caution and don't forget report bugs and submit patches.
2000-07-07 15:00:38 +00:00
Maxim Sobolev
2ec55b0869 Add checknewvers.sh - a script which checks for availability of the newest
versions of distfiles on MASTER_SITES. This is still work in progress, so
please use with caution and don't forget report bugs and submit patches.
2000-07-07 14:57:41 +00:00
Satoshi Asami
d8176a286f Back out (1) in previous commit -- extra directories are fatal again.
Otherwise it is often too hard to find out which port left the directory
behind since packages propagates extra directories.  Also, many
of the extra directories contain files, so it makes no sense to ignore them.
2000-07-05 17:49:46 +00:00
Maxim Sobolev
54f7240973 Allow existing CVS_RSH environment var to take precedent. 2000-07-04 09:55:31 +00:00
Will Andrews
ed1447026c Allow existing CVS_RSH environment var to take precedent.
Submitted by:	sobomax
2000-07-03 19:58:41 +00:00
Satoshi Asami
f20bd10615 (1) Make extra files fatal errors only if they are files -- extra
directories are still included in the list of "extra stuff", but
    are not considered fatal (for now).

(2) Use ls's -a flag to print out list of all files in ${PREFIX}, for
    obvious reasons.
2000-07-02 09:32:30 +00:00
Will Andrews
bbdaf83188 Fix addport so that it will work on freefall. Remove old cruft left
around from older versions of addport.  Make addport a little more helpful
by actually displaying usage() if no args are passed.
2000-06-30 23:33:35 +00:00
Maxim Sobolev
9ef89b7e83 Add close-pr - a slightly hacked version of edit-pr(1) to quickly close PR. 2000-06-26 17:49:04 +00:00
Alexander Langer
b26dd59505 Add extract-lines for .Z archives.
Change MAINTAINER: alex@big.endian.de --> alex@FreeBSD.org

Reviewed by:	will
2000-06-10 11:12:10 +00:00
Maxim Sobolev
9a158320b9 Add a traling slash ("/") to the /usr/ports/ and /usr/ports/distfiles/ to make
the script working for the cases when either one directory is a symlink.

Submitted by:	Juriy Goloveshkin <juriy@avias.com>, Volodymyr Kostyrko <arcade@limbo.dn.ua>
2000-06-06 10:05:06 +00:00
Maxim Sobolev
68170052a9 Add tag and MAINTAINER. 2000-06-05 15:50:32 +00:00
Maxim Sobolev
6fed6d6bdd Add distclean.sh - a little script to check set of distfiles against currently
installed ports collection and prompt to remove unmatching entries (i.e.
distfiles that doesn't have corresponding md5 file).

Hardly requiested by:	will
2000-06-05 15:47:25 +00:00
Will Andrews
2247ec04c2 Update for multiple ports. Also add MAINTAINER/Created [on,by] lines.
Submitted by:	maintainer
2000-05-24 05:29:54 +00:00
Will Andrews
d48209bc9b Add checksum.sh script that checks a port or ports for broken distfiles and
gives a diff of any port with broken distfiles, if the distfile names are
the same.  This script will help keep more ports' up-to-date distfiles on
ftp.FreeBSD.org.

PR:		18723
Submitted by:	Alexander Langer <alex@cichlids.com>
2000-05-22 10:02:42 +00:00
Maxim Sobolev
9a170ebda5 Make getpr work for remote committers as well. 2000-05-19 08:09:53 +00:00
Satoshi Asami
d36051792d (1) Make this file callable in two phases (checksum, package) so distfiles
can be fetched even if the packaging fails.  This is to ensure distfiles
    with correct checksums can be fetched and put on ftp.FreeBSD.org
    before they disappear from the original master sites.

(2) Delete port and all dependencies after packaging, and print out
    list of

 (a) Security-related files (set[ug]id flag set or world-writable)

 (b) Extra files and directories

Requested by:	 kris (2a only)
2000-05-08 07:37:28 +00:00
Michael Haro
546a427a27 Readme describing all the scripts in this directory. Somebody good at
writing docs should probably fix this file.
2000-05-01 20:00:10 +00:00