jmq
/
pkg-get
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

pkg-repgen: do more tasks in parallel 

update the html index too, when individual packages are given

pkg-get docs: explain in more detail the effect of -r (alternate root)
This commit is contained in:
John McQuah 2023-06-18 18:25:55 -04:00
parent f769b5251e
commit 57f3b29e2e
8 changed files with 469 additions and 383 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
ChangeLog
View File

@ -1,9 +1,14 @@
ChangeLog for pkg-get ChangeLog for pkg-get
0.4.8 - Fix pre- and post-install scripts to accommodate rootfs other 0.4.8 - Fix doscript() to accommodate rootfs other than "/",
than "/", and ports with dashes in their names and ports with dashes in their names
- Use Perl modules where possible, to avoid spawning external processes
- Consolidate code - Consolidate code
- Do more tasks in parallel when generating the metadata
- Update the html index too, when individual packages are passed to
pkg-repgen
- Use native Perl routines to reduce the number of external processes
- Replace wget by curl, to avoid depending on packages outside 'core'
- Update man-pages and README
0.4.7 - Fix man-page location 0.4.7 - Fix man-page location

28
README
View File

@ -51,9 +51,17 @@ LIMITATIONS
The client and the server must be configured to use the same The client and the server must be configured to use the same
pkgmk compression mode, otherwise the client will try to download pkgmk compression mode, otherwise the client will try to download
a tarball with the wrong suffix. This is only a problem if you sometimes a tarball with the wrong suffix. This is only a problem if you sometimes
compile ports on the client device. By allowing you to maintain your compile ports on the client machine and prefer a different compression mode
client device solely with binary packages, pkg-get makes the contents of (better suited to its less-powerful hardware?). By allowing you to maintain
/etc/pkgmk.conf mostly irrelevant. your client machine solely with binary packages, pkg-get makes the contents
of /etc/pkgmk.conf mostly irrelevant, so you can simply put a verbatim copy
of the server's pkgmk.conf on the client machine.
The pkg-get configuration file does not offer as many settings as the one
for prt-get. In particular, you cannot change "addcommand", "rmcommand", or
"runscriptscommand"; these are hard-coded as /usr/bin/pkgadd, /usr/bin/pkgrm,
and /bin/bash, respectively. There is also currently no mechanism to respect
"preferhigher", when doing 'pkg-get diff' or 'pkg-get sysup'.
'pkg-get depends' and 'prt-get quickdep' do not handle more than one port, 'pkg-get depends' and 'prt-get quickdep' do not handle more than one port,
unlike the corresponding commands in prt-get. Therefore it is not as unlike the corresponding commands in prt-get. Therefore it is not as
@ -65,12 +73,14 @@ Alas, such a switch is also absent from the design of pkg-get. Use
the --test switch with prt-get itself, for the closest approximation the --test switch with prt-get itself, for the closest approximation
of previewing the outcome from a 'pkg-get depinst' operation. of previewing the outcome from a 'pkg-get depinst' operation.
Among the prt-get commands that have no counterpart in pkg-get 'pkg-get dependent' does not support the --recursive option. Other
(grpinst, fsearch, deptree, listorphans, ls, cat, edit, cache), useful prt-get commands (grpinst, fsearch, deptree, listorphans, ls,
only the 'grpinst' command is of possible interest; the remaining cat, edit, cache) have no counterpart in pkg-get. Of these omissions,
commands are just as easily delegated to prt-get itself. If you want only the 'grpinst' command is of possible interest for binary package
a Perl implementation that does provide these missing commands, consider management; the unimplemented commands and options are just as easily
the script written by user farkuhar [1]. delegated to prt-get itself. If you want a Perl implementation that does
provide these missing commands, consider the script written by user
farkuhar [1].
pkg-get only makes use of the hard dependencies listed by the port pkg-get only makes use of the hard dependencies listed by the port
maintainer, not any of the eager linking that might have occurred on the maintainer, not any of the eager linking that might have occurred on the

74
doc/pkg-get.8
View File

@ -1,4 +1,3 @@
." Text automatically generated by txt2man-1.4.7
.TH pkg-get 8 "July 13, 2006" "" "" .TH pkg-get 8 "July 13, 2006" "" ""
.SH NAME .SH NAME
\fBpkg-get \fP- a package management tool for CRUX Linux \fBpkg-get \fP- a package management tool for CRUX Linux
@ -20,30 +19,30 @@ can be generated by repository maintainers with the
.TP .TP
.B .B
install <package1> [<package2>..<packageN>] install <package1> [<package2>..<packageN>]
Install given packages, Install given packages, download if necessary.
download if necessary.
.TP .TP
.B .B
update <package1> [<package2>..<packageN>] update <package1> [<package2>..<packageN>]
Update given packages, Update given packages, download if necessary.
download if necessary.
.TP .TP
.B .B
diff [\fB--all\fP] diff [\fB--all\fP]
Show a list of outdated packages. The \fB--all\fP option also displays locked packages. Show a list of outdated packages.
The \fB--all\fP option also displays locked packages.
.TP .TP
.B .B
quickdiff quickdiff
Show a compact list of outdated packages. Show a compact list of outdated packages.
.TP .TP
.B .B
sysup sysup [\fB--all\fP]
Update all outdated packages. Download if necessary. Update all outdated packages, download if necessary.
Locked packages are excluded from the operation
unless \fB--all\fP is passed.
.TP .TP
.B .B
depinst <package1> [<package2>..<packageN>] depinst <package1> [<package2>..<packageN>]
Install given packages Install given packages and relative dependencies.
and relative dependencies.
.TP .TP
.B .B
depends <package> depends <package>
@ -51,13 +50,17 @@ Show a recursive list of dependencies for package
and their installation status. and their installation status.
.TP .TP
.B .B
quickdep <package>
Show a brief list of dependencies for package.
.TP
.B
dependent <package> [\fB--all\fP] dependent <package> [\fB--all\fP]
Show installed (or all with the \fB--all\fP option) packages Show installed (or all with the \fB--all\fP option) packages
that depend from package. that depend from package.
.TP .TP
.B .B
quickdep <package> path <package>
Show a brief list of dependencies for package. Show local path of package.
.TP .TP
.B .B
info <package> info <package>
@ -68,8 +71,8 @@ current <package>
Show currently installed version of package. Show currently installed version of package.
.TP .TP
.B .B
path <package> isinst <package>
Show local path of package. Display whether a package is installed.
.TP .TP
.B .B
readme <package> readme <package>
@ -84,10 +87,6 @@ listinst
List all installed packages. List all installed packages.
.TP .TP
.B .B
isinst <package>
Display whether a package is installed.
.TP
.B
dsearch <string> dsearch <string>
Search for packages which name or description contain <string>. Search for packages which name or description contain <string>.
.TP .TP
@ -101,9 +100,9 @@ List all duplicates ports (present in more than one repository).
.TP .TP
.B .B
printf <format string1> [\fB--filter\fP=<filter>] printf <format string1> [\fB--filter\fP=<filter>]
Description blatantly Description blatantly stolen from \fBprt\-get\fP(8).
stolen from prt-get man file. Print formatted port list. Format string can Print formatted port list. Format string can contain variables,
contain variables, which are replaced like this: which are replaced like this:
.RS .RS
.IP \(bu 3 .IP \(bu 3
%n -> name %n -> name
@ -128,7 +127,7 @@ contain variables, which are replaced like this:
.IP \(bu 3 .IP \(bu 3
%M -> "Nobody". for compatibility with prt-get %M -> "Nobody". for compatibility with prt-get
.IP \(bu 3 .IP \(bu 3
%P -> "Nobody". for compatibility with prt-get %P -> "None". for compatibility with prt-get
.IP \(bu 3 .IP \(bu 3
%l -> is locked ("yes"/"no") %l -> is locked ("yes"/"no")
.IP \(bu 3 .IP \(bu 3
@ -138,7 +137,7 @@ repository.
.RE .RE
.PP .PP
Use "\\n" and "\\t" to format your output (no additional format Use "\\n" and "\\t" to format your output (no additional format
specified suported). The optional format string2 can contain the codes suported). The optional format string2 can contain the
same variables as format string1 and is used to sort the output. same variables as format string1 and is used to sort the output.
You can specify a wildcard filter to filter by package name. You can specify a wildcard filter to filter by package name.
.TP .TP
@ -160,12 +159,8 @@ Syncronize local packages with the ones from the remote repository.
If the repository is local, this \fIcommand\fP does nothing. If the repository is local, this \fIcommand\fP does nothing.
.TP .TP
.B .B
sysup
Update all outdated packages.
.TP
.B
help help
Display brief help screen. Display brief help screen.
.TP .TP
.B .B
version version
@ -174,12 +169,15 @@ Show \fBpkg-get\fP version.
.TP .TP
.B .B
\fB-r\fP <root> \fB-r\fP <root>
Use <root> directory when wrapping pkgadd. Note that Use <root> directory for operations involving the package database
this only works with update / install ! (install, depinst, update, isinst, current, listinst, diff, quickdiff, sysup).
This option does not affect the directories from which \fBpkg-get\fP loads the
metadata or tarballs; those settings are still governed by the entries in
the configuration file.
.TP .TP
.B .B
\fB-do\fP \fB-do\fP
Download only. Applicable to: sysup, depinst, install, update. Download only. Applicable to: sysup, install, depinst, update.
.TP .TP
.B .B
\fB-f\fP \fB-f\fP
@ -199,15 +197,15 @@ Use the specified configuration file.
.TP .TP
.B .B
\fB--pre-install\fP \fB--pre-install\fP
executes pre-install script if available. Execute pre-install script if available.
.TP .TP
.B .B
\fB--post-install\fP \fB--post-install\fP
executes post-install script if available. Execute post-install script if available.
.TP .TP
.B .B
\fB--install-scripts\fP \fB--install-scripts\fP
executes pre-install and post-install Execute pre-install and post-install
scripts if available. scripts if available.
.RE .RE
.PP .PP
@ -222,13 +220,13 @@ Configuration is handled by the /etc/pkg-get.conf file,
Install sqlite and pysqlite. Install sqlite and pysqlite.
.TP .TP
.B .B
\fBpkg-get\fP depinst kdebase \fB-f\fP \fBpkg-get\fP depinst qt6-base \fB-f\fP
Install kdebase and all its dependencies, forcing upgrade. Install qt6-base and all its dependencies, forcing upgrade.
.TP .TP
.B .B
\fBpkg-get\fP sysup \fB-do\fP \fBpkg-get\fP sysup \fB-do\fP
Download new releases of all the outdated packages. Download new releases of all the outdated packages.
.SH AUTHORS .SH AUTHORS
Simone Rota <sip@varlock.com> Simone Rota <sip@varlock.com>, John McQuah <jmcquah@disroot.org>
.SH SEE ALSO .SH SEE ALSO
\fBpkgadd\fP(8), \fBprt-get\fP(8) \fBpkgadd\fP(8), \fBprt-get\fP(8), \fBpkg-repgen\fP(8)

245
doc/pkg-get.8.txt
View File

@ -1,101 +1,176 @@
pkg-get(8)
NAME NAME
pkg-get - a package management tool for CRUX Linux pkg-get - a package management tool for CRUX Linux
SYNOPSIS SYNOPSIS
pkg-get command <arguments> [options] pkg-get command <arguments> [options]
DESCRIPTION DESCRIPTION
pkg-get is a simple package management tool for CRUX Linux. pkg-get is a simple package management tool for CRUX Linux.
It tries to replicate some of the most useful features of the It tries to replicate some of the most useful features of the
port management tool prt-get(8) to be used with binary packages. port management tool prt-get(8) to be used with binary packages.
pkg-get requires a remote or local package repository that pkg-get requires a remote or local package repository that can be
can be generated by repository maintainers with the generated by repository maintainers with the pkg-repgen(8) script.
pkg-repgen(8) script.
COMMANDS COMMANDS
install <package1> [<package2>..<packageN>] Install given packages, install <package1> [<package2>..<packageN>]
download if necessary. Install given packages, download if necessary.
update <package1> [<package2>..<packageN>] Update given packages,
download if necessary. update <package1> [<package2>..<packageN>]
diff [--all] Show a list of outdated packages. The --all option also displays locked packages. Update given packages, download if necessary.
quickdiff Show a compact list of outdated packages.
sysup Update all outdated packages. Download if necessary. diff [--all]
depinst <package1> [<package2>..<packageN>] Install given packages Show a list of outdated packages. The --all option also
and relative dependencies. displays locked packages.
depends <package> Show a recursive list of dependencies for package
and their installation status. quickdiff
dependent <package> [--all] Show installed (or all with the --all option) packages Show a compact list of outdated packages.
that depend from package.
quickdep <package> Show a brief list of dependencies for package. sysup [--all]
info <package> Show information about package. Update all outdated packages, download if necessary.
current <package> Show currently installed version of package. Locked packages are excluded from the operation unless
path <package> Show local path of package. --all is passed.
readme <package> Print README information (if available) for the package.
list List all packages in the repository. depinst <package1> [<package2>..<packageN>]
listinst List all installed packages. Install given packages and relative dependencies.
isinst <package> Display whether a package is installed.
dsearch <string> Search for packages which name or description contain <string>. depends <package>
search <string> Search for packages which name contains <string>. Show a recursive list of dependencies for package and
dup List all duplicates ports (present in more than one repository). their installation status.
printf <format string1> [--filter=<filter>] Description blatantly
stolen from prt-get man file. Print formatted port list. Format string can quickdep <package>
contain variables, which are replaced like this: Show a brief list of dependencies for package.
- %n -> name
- %p -> path dependent <package> [--all]
- %v -> version Show installed (or all with the --all option) packages
- %r -> release that depend from package.
- %d -> description
- %e -> dependencies path <package>
- %u -> url Show local path of package.
- %R -> Readme ("yes"/"no")
- %E -> pre-install script ("yes"/"no") info <package>
- %O -> post-install script ("yes"/"no") Show information about package.
- %M -> "Nobody". for compatibility with prt-get
- %P -> "Nobody". for compatibility with prt-get current <package>
- %l -> is locked ("yes"/"no") Show currently installed version of package.
- %i -> "no" if not installed, "yes" if it's installed and up to
date and "diff" if it's installed and a new version is in the isinst <package>
repository. Display whether a package is installed.
Use "\\n" and "\\t" to format your output (no additional format readme <package>
specified suported). The optional format string2 can contain the Print README information (if available) for the package.
same variables as format string1 and is used to sort the output.
You can specify a wildcard filter to filter by package name. list List all packages in the repository.
listinst
List all installed packages.
dsearch <string>
Search for packages whose name or description contains
<string>.
search <string>
Search for packages whose name contains <string>.
dup List all duplicate ports (present in more than one
repository).
printf <format string1> [--filter=<filter>]
Description blatantly stolen from prt-get(8). Print
formatted port list. Format string can contain variables,
which are replaced like this:
• %n -> name
• %p -> path
• %v -> version
• %r -> release
• %d -> description
• %e -> dependencies
• %u -> url
• %R -> Readme ("yes"/"no")
• %E -> pre-install script ("yes"/"no")
• %O -> post-install script ("yes"/"no")
• %M -> "Nobody". for compatibility with prt-get
• %P -> "None". for compatibility with prt-get
• %l -> is locked ("yes"/"no")
• %i -> "no" if not installed, "yes" if it's installed
and up to date and "diff" if it's installed and
a new version is in the repository.
Use "\n" and "\t" to format your output (no additional format
codes supported). The optional format string2 can contain
the same variables as format string1 and is used to sort
the output. You can specify a wildcard filter to filter by
package name.
lock <package1> [<package2>..<packageN>]
Lock a package (ignore updates).
unlock <package1> [<package2>..<packageN>]
Unlock a package.
listlocked
Display a list of locked packages.
sync Syncronize local packages with the ones from the remote
repository. If the repository is local, this command does
nothing.
help Display brief help screen.
version
Show pkg-get version.
lock <package1> [<package2>..<packageN>] Lock a package (ignore updates).
unlock <package1> [<package2>..<packageN>] Unlock a package.
listlocked Display a list of locked packages.
sync Syncronize local packages with the ones from the remote repository.
If the repository is local, this command does nothing.
sysup Update all outdated packages.
help Display brief help screen.
version Show pkg-get version.
OPTIONS OPTIONS
-r <root> Use <root> directory when wrapping pkgadd. Note that -r <root>
this only works with update / install ! Use <root> directory for operations involving the package
-do Download only. Applicable to: sysup, depinst, install, update. database (install, depinst, update, isinst, current,
-f Force installing / upgrading. This is passed to pkgadd(8). listinst, diff, quick‐ diff, sysup). This option does
-im Ignore md5sum mismatches. not affect the directories from which pkg-get loads the
--aargs="arguments" pass the specified arguments to pkgadd(8). metadata or tarballs; those settings are still governed
--config=/path/to/file Use the specified configuration file. by the entries in the configuration file.
--pre-install executes pre-install script if available.
--post-install executes post-install script if available. -do Download only. Applicable to: sysup, install, depinst,
--install-scripts executes pre-install and post-install update.
scripts if available.
-f Force installing / upgrading. This is passed to pkgadd(8).
-im Ignore md5sum mismatches.
--aargs="arguments"
Pass the specified arguments to pkgadd(8).
--config=/path/to/file
Use the specified configuration file.
--pre-install
Execute pre-install script if available.
--post-install
Execute post-install script if available.
--install-scripts
Execute pre-install and post-install scripts if available.
CONFIGURATION CONFIGURATION
Configuration is handled by the /etc/pkg-get.conf file, Configuration is handled by the /etc/pkg-get.conf file, options
options are explained in the file itself. are explained in the file itself.
EXAMPLES EXAMPLES
pkg-get install sqlite pysqlite Install sqlite and pysqlite. pkg-get install sqlite pysqlite
pkg-get depinst kdebase -f Install kdebase and all its dependencies, forcing upgrade. Install sqlite and pysqlite.
pkg-get sysup -do Download new releases of all the outdated packages.
pkg-get depinst qt6-base -f
Install qt6-base and all its dependencies, forcing upgrade.
pkg-get sysup -do
Download new releases of all the outdated packages.
AUTHORS AUTHORS
Simone Rota <sip@varlock.com> Simone Rota <sip@varlock.com>, John McQuah <jmcquah@disroot.org>
SEE ALSO SEE ALSO
pkgadd(8), prt-get(8) pkgadd(8), prt-get(8), pkg-repgen(8)
July 13, 2006 pkg-get(8)

14
doc/pkg-repgen.8
View File

@ -1,4 +1,3 @@
." Text automatically generated by txt2man-1.4.7
.TH pkg-repgen 8 "July 13, 2006" "" "" .TH pkg-repgen 8 "July 13, 2006" "" ""
.SH NAME .SH NAME
\fBpkg-repgen \fP- generate a package repository for pkg-get \fBpkg-repgen \fP- generate a package repository for pkg-get
@ -13,12 +12,13 @@
\fBpkg-repgen\fP generates files needed by a \fBpkg-get\fP(8) package \fBpkg-repgen\fP generates files needed by a \fBpkg-get\fP(8) package
repository. It also generates a html index of the packages. repository. It also generates a html index of the packages.
.SH USAGE .SH USAGE
run \fBpkg-repgen\fP from the directory containing packages to Run \fBpkg-repgen\fP from the directory containing packages to
generate the whole repository. generate the whole repository.
Specify a list of package names to update only those packages Specify a list of package names to update only those packages
(this is expecially useful with large repositories). (this is especially useful with large repositories).
Note that the html index is not updated when single Even when individual packages are specified, the entire directory
packages are specified. is still processed to collect all the readmes and {pre,post}-install
scripts.
.SH OPTIONS .SH OPTIONS
.TP .TP
.B .B
@ -39,9 +39,9 @@ use alternative prtdir for prt-get commands
.SH EXAMPLES .SH EXAMPLES
\fBpkg-repgen\fP \fBpkg-repgen\fP
.PP .PP
\fBpkg-repgen\fP kdebase kdelibs \fBpkg-repgen\fP qt6-base qt6-tools
.SH AUTHORS .SH AUTHORS
Simone Rota <sip@varlock.com> Simone Rota <sip@varlock.com>, John McQuah <jmcquah@disroot.org>
index generation code adapted from Jukka Heino's portspage index generation code adapted from Jukka Heino's portspage
.SH SEE ALSO .SH SEE ALSO
\fBpkg-get\fP(8), \fBprt-get\fP(8) \fBpkg-get\fP(8), \fBprt-get\fP(8)

51
doc/pkg-repgen.8.txt
View File

@ -1,35 +1,46 @@
pkg-repgen(8)
NAME NAME
pkg-repgen - generate a package repository for pkg-get pkg-repgen - generate a package repository for pkg-get
SYNOPSIS SYNOPSIS
pkg-repgen [options][package1...packageN] pkg-repgen [options][package1...packageN]
DESCRIPTION DESCRIPTION
pkg-repgen generates files needed by a pkg-get(8) package pkg-repgen generates files needed by a pkg-get(8) package
repository. It also generates a html index of the packages. repository. It also generates a html index of the packages.
USAGE USAGE
run pkg-repgen from the directory containing packages to Run pkg-repgen from the directory containing packages to generate
generate the whole repository. the whole repository. Specify a list of package names to
Specify a list of package names to update only those packages update only those packages (this is especially useful with large
(this is expecially useful with large repositories). repositories). Even when individual packages are specified,
Note that the html index is not updated when single the entire directory is still processed to collect all the
packages are specified. readmes and {pre,post}-install scripts.
OPTIONS OPTIONS
--header=FILE insert FILE at the beginning of the html index --header=FILE
--header=FILE insert FILE at the beginning of the html index insert FILE at the beginning of the html index
--title=title use the specified title for the index page
--prtdir=DIR use alternative prtdir for prt-get commands --header=FILE
insert FILE at the beginning of the html index
--title=title
use the specified title for the index page
--prtdir=DIR
use alternative prtdir for prt-get commands
EXAMPLES EXAMPLES
pkg-repgen pkg-repgen
pkg-repgen kdebase kdelibs pkg-repgen qt6-base qt6-tools
AUTHORS AUTHORS
Simone Rota <sip@varlock.com> Simone Rota <sip@varlock.com>, John McQuah <jmcquah@disroot.org>
index generation code adapted from Jukka Heino's portspage index generation code adapted from Jukka Heino's portspage
SEE ALSO SEE ALSO
pkg-get(8), prt-get(8) pkg-get(8), prt-get(8)
July 13, 2006 pkg-repgen(8)

66
scripts/pkg-get.pl
View File

@ -2,7 +2,7 @@
# #
# pkg-get - A binary package management utility for CRUX Linux # pkg-get - A binary package management utility for CRUX Linux
# Copyright (C) 2004-2006 Simone Rota <sip@varlock.com> # Copyright (C) 2004-2006 Simone Rota <sip@varlock.com>
# Copyright (c) 2006-2013 by CRUX team (http://crux.nu) # Copyright (C) 2006-2023 by CRUX team (http://crux.nu)
# #
# This program is free software; you can redistribute it and/or modify # This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by # it under the terms of the GNU General Public License as published by
@ -79,7 +79,7 @@ SWITCH: {
if ($command eq "listlocked") { listlocked(); last SWITCH; } if ($command eq "listlocked") { listlocked(); last SWITCH; }
if ($command eq "printf") { doprintf(); last SWITCH; } if ($command eq "printf") { doprintf(); last SWITCH; }
if ($command =~ /^(isinst|current)$/) { current(); last SWITCH; } if ($command =~ /^(isinst|current)$/) { current(); last SWITCH; }
if ($command =~ /^(|quick)diff$/) { diff($1); last SWITCH; } if ($command =~ /^(diff|quickdiff|sysup)$/) { diff($1); last SWITCH; }
if ($command eq "dup") { dup(); last SWITCH; } if ($command eq "dup") { dup(); last SWITCH; }
if ($command =~ /^(depends|quickdep)$/) { depends(); last SWITCH; } if ($command =~ /^(depends|quickdep)$/) { depends(); last SWITCH; }
if ($command =~ /^(install|update)$/) { upinst(@ARGV); last SWITCH; } if ($command =~ /^(install|update)$/) { upinst(@ARGV); last SWITCH; }
@ -151,7 +151,6 @@ sub readconfig {
# Populate a hash of locked packages # Populate a hash of locked packages
sub get_locked { sub get_locked {
(-f $LOCKFILE) or system("touch $LOCKFILE");
open (my $fL, $LOCKFILE) or return; open (my $fL, $LOCKFILE) or return;
while (<$fL>) { $locked{$_} = 1; } while (<$fL>) { $locked{$_} = 1; }
close ($fL); close ($fL);
@ -439,7 +438,12 @@ sub installpkg {
$pptasks{$pkg{'name'}} = ""; $pptasks{$pkg{'name'}} = "";
if ($download_only) {return 1;} if ($download_only) {return 1;}
if ($force){$aa = $aa."-f ";} if ($force){$aa = $aa."-f ";}
if ($root) {$aa = $aa."-r ".$root." ";} if ($root) {
$aa = $aa."-r ".$root." ";
(-f "$root/$pkg{'path'}/PKGINST") or
system("install -D $pkg{'path'}/PKGINST $root/$pkg{'path'}/PKGINST")
or die "Failed to copy PKGINST to $root, aborting.\n";
}
if ($pkg{'pre_install'} eq "yes" and ($install_scripts or $pre_install)) {doscript("pre",%pkg);} if ($pkg{'pre_install'} eq "yes" and ($install_scripts or $pre_install)) {doscript("pre",%pkg);}
my $fullpath = $pkg{'path'}."/".$pkg{'name'}."#".$pkg{'version'}."-".$pkg{'release'}.".pkg.tar.$compress"; my $fullpath = $pkg{'path'}."/".$pkg{'name'}."#".$pkg{'version'}."-".$pkg{'release'}.".pkg.tar.$compress";
print "pkg-get: /usr/bin/pkgadd $upgrade $aa$fullpath\n"; print "pkg-get: /usr/bin/pkgadd $upgrade $aa$fullpath\n";
@ -452,7 +456,7 @@ sub installpkg {
sub doscript { sub doscript {
my ($when, %pkg) = @_; my ($when, %pkg) = @_;
my $cmd = ($root) ? "chroot $root " : ""; my $cmd = ($root) ? "chroot $root " : "";
$cmd .= "/bin/bash $pkg{'path'}/PKGINST $pkg{name} $when"; $cmd .= "/bin/bash $pkg{'path'}/PKGINST $pkg{'name'} $when";
if (system($cmd) == 0) { if (system($cmd) == 0) {
$pptasks{$pkg{'name'}} .= " [$when: ok]"; $pptasks{$pkg{'name'}} .= " [$when: ok]";
} else { } else {
@ -519,7 +523,7 @@ sub sync {
} }
} }
# Show info/path/readme for the package ##################################### # Show info/path/readme for the package ####################################
sub info { sub info {
my ($type, $arg) = @ARGV; my ($type, $arg) = @ARGV;
foreach my $repo(@repos) { foreach my $repo(@repos) {
@ -541,7 +545,7 @@ sub info {
print "Package '$arg' not found\n"; print "Package '$arg' not found\n";
} }
# List packages containing given string (name/description) ################### # List packages containing given string (name/description) #################
sub search { sub search {
my $arg = $ARGV[1]; my $arg = $ARGV[1];
my $type = ($ARGV[0] =~ /^d/) ? "desc" : "name"; my $type = ($ARGV[0] =~ /^d/) ? "desc" : "name";
@ -701,9 +705,9 @@ sub doprintf {
} }
} }
# Show differences between installed and available packages ################ # Show or resolve differences between installed and available packages #####
sub diff { sub diff {
my $format = shift; my $gotdiff = 0; my %found; my $format = shift; my %found; my @diff;
foreach my $repo(@repos) { foreach my $repo(@repos) {
my ($dir, $url) = split(/\|/, $repo); my ($dir, $url) = split(/\|/, $repo);
@ -715,20 +719,23 @@ sub diff {
$found{$pkg{'name'}} = 1; $found{$pkg{'name'}} = 1;
if ($pkg{'instversion'} ne $pkg{'version'}."-".$pkg{'release'}) { if ($pkg{'instversion'} ne $pkg{'version'}."-".$pkg{'release'}) {
next if (($locked{$pkg{'name'}}) and (! $all)); next if (($locked{$pkg{'name'}}) and (! $all));
if (($gotdiff == 0) and ($format ne "quick")) { if (($#diff < 0) and ($format !~ /^(quick|sysup)/)) {
print "Differences between installed packages and packages repo:\n\n"; print "Differences between installed packages and packages repo:\n\n";
printf("%-19s %-19s %-19s\n\n","Package","Installed","Available in the repositories"); printf("%-19s %-19s %-19s\n\n","Package","Installed","Available in the repositories");
} }
$gotdiff = 1;
my $lastcol = ($locked{$pkg{'name'}}) ? "locked" : $pkg{'version'}."-".$pkg{'release'}; my $lastcol = ($locked{$pkg{'name'}}) ? "locked" : $pkg{'version'}."-".$pkg{'release'};
print "$pkg{'name'} " if ($format eq "quick"); push @diff, $pkg{'name'};
printf("%-19s %-19s %-19s\n", $pkg{'name'}, $pkg{'instversion'}, $lastcol) if ($format ne "quick"); print "$pkg{'name'} " if ($format =~ /^quick/);
printf("%-19s %-19s %-19s\n", $pkg{'name'}, $pkg{'instversion'}, $lastcol) if ($format !~ /^(quick|sysup)/);
} }
} }
close(REPO); close(REPO);
} }
print "\n" if (($format eq "quick") and ($gotdiff == 1)); print "\n" if (($format =~ /^quick/) and ($#diff >= 0));
print "No differences found\n" if ($gotdiff == 0); print "No differences found\n" if ($#diff < 0);
# proceed with updates if sysup was requested
upinst("update",@diff) if (($#diff >= 0) and ($format eq "sysup"));
} }
# Display duplicate packages (found in more than one repo) ################# # Display duplicate packages (found in more than one repo) #################
@ -801,13 +808,8 @@ sub upinst {
my ($cmd, @args) = @_; my $aa; my ($cmd, @args) = @_; my $aa;
($curraction, $aa) = ($cmd =~ /^up/) ? ("updated","-u") : ("installed",""); ($curraction, $aa) = ($cmd =~ /^up/) ? ("updated","-u") : ("installed","");
getinstalled() if (! %installed); getinstalled() if (! %installed);
if ($root) {
foreach my $repo (@repos) {
my ($dir, $url) = split(/\|/, $repo);
(! -f "$dir/PKGINST") or system("install -D $dir/PKGINST $root$dir/PKGINST");
}
}
foreach my $pkgname(@args) { foreach my $pkgname(@args) {
my %pkg = getpackage($pkgname, 1); my %pkg = getpackage($pkgname, 1);
if (not %pkg) { if (not %pkg) {
@ -824,27 +826,7 @@ sub upinst {
printresults(); printresults();
} }
# Update out of date packages ############################################## # Install given package, along with dependencies ###########################
sub sysup {
my %found; my @diff;
foreach my $repo(@repos) {
my ($dir, $url) = split(/\|/, $repo);
open(REPO, "$dir/PKGREPO") or exiterr("could not open $dir/PKGREPO");
while (<REPO>) {
chomp;
my %pkg = parsepackage($_, $dir, $url, 1);
next if ($found{$pkg{'name'}}); $found{$pkg{'name'}} = 1;
if ( ($pkg{'instversion'}) and
($pkg{'instversion'} ne $pkg{'version'}."-".$pkg{'release'})
and (! $locked{$pkg{'name'}}) ) {
push(@diff, $pkg{'name'});
}
}
close(REPO);
}
upinst("update",@diff) if (@diff);
}
sub depinst { sub depinst {
my @toinst; my %seen; my @toinst; my %seen;
$curraction = "installed"; $curraction = "installed";

363
scripts/pkg-repgen.pl
View File

@ -22,7 +22,7 @@ GetOptions("prtdir=s"=>\$prtdir, "title=s"=>\$title, "header=s"=>\$header, "foot
our $compress = "gz"; our $compress = "gz";
open CONFIG, "/etc/pkgmk.conf" or die "Could not open /etc/pkgmk.conf"; open CONFIG, "/etc/pkgmk.conf" or die "Could not open /etc/pkgmk.conf";
while (<CONFIG>) { while (<CONFIG>) {
$compress = $1 if m/^PKGMK_COMPRESSION_MODE=(.*)(#|$)/; $compress = $1 if m/^PKGMK_COMPRESSION_MODE=(.*)(\#|$)/;
} }
close CONFIG; close CONFIG;
$compress =~ s/["' ]//g; $compress =~ s/["' ]//g;
@ -32,213 +32,215 @@ $prtget .= " --no-std-config --config-set=\"prtdir $prtdir\"" if ($prtdir);
my @dirlist = glob("*#*.pkg.tar.$compress"); my @packages; my @dirlist = glob("*#*.pkg.tar.$compress"); my @packages;
if ($#ARGV >= 0) { # single packages if ($#ARGV >= 0) { # single packages
foreach my $pkgname (@ARGV) { foreach my $pkgname (@ARGV) {
my @hits = grep /^$pkgname#/ @dirlist; my @hits = sort grep { /^$pkgname\#/ } @dirlist;
push(@packages,@hits) if (@hits); push(@packages,$hits[-1]) if (@hits);
} }
} else { } else {
@packages = @dirlist; @packages = @dirlist;
} }
# Populate some hashes with a single run of prt-get # Populate hashes using a single run of prt-get
our %path; our %depends; our %descrip; our %flags; our %path; our %depends; our %descrip; our %flags;
our %oldDeps; our %oldFlags; our %du; our %md5sums; my @validkeys = @dirlist;
fill_hashes_from_prtget(); map { s/\#.*// } @validkeys;
my %printme = map { $_ => 1 } @validkeys;
if ($#ARGV >= 0) { open (my $ppf, "$prtget printf '%n^%p^%e^%d^%E:%O:%R\n' |");
pkg_single("REPO"); pkg_single("DEPS"); while (<$ppf>) {
} else { chomp;
pkg_dir("REPO"); pkg_dir("DEPS"); my ($name,$repo,$deps,$desc,$prepostread) = split /\^/;
next if (! $printme{$name});
$path{$name} = $repo . "/" . $name;
$depends{$name} = $deps;
$desc =~ s/\:/ /g;
$descrip{$name} = $desc;
$flags{$name} = $prepostread;
} }
close ($ppf);
# Needed for alternating colors in the html index
my %parity = ( 0 => "even", 1 => "odd" );
# Generate the metadata files
($#ARGV >= 0) ? pkg_single() : pkg_dir();
# Generate README and PKGINST # Generate README and PKGINST
pkgread(); pkgreadscripts();
pkginst();
sub fill_hashes_from_prtget { ######################## individual packages ########################
my @validkeys = @packages;
map { s/#.*// } @validkeys;
open (my $ppf, "$prtget printf '%n^%p^%e^%d^%E^%O^%R\n' |");
while (<$ppf>) {
my ($name,$repo,$deps,$desc,$haspre,$haspost,$hasreadme) = split /\^/;
next if (! grep { ($_ eq $name) } @validkeys);
$path{$name} = $repo . "/" . $name;
$depends{$name} = $deps;
$desc =~ s/\:/ /g;
$descrip{$name} = $desc;
chomp($hasreadme);
$flags{$name} = join(":", $haspre, $haspost, $hasreadme);
}
close ($ppf);
}
sub fill_hashes_from_prevrun {
my $oldRepo = shift;
open (my $fh, $oldRepo) or return;
while (<$fh>) {
chomp;
if ($oldRepo eq "PKGDEPS") {
my ($iPkg, $iDep) = split /:/;
$iPkg =~ s/\s+//g;
$iDep =~ s/,/ /g;
$oldDeps{$iPkg} = $iDep;
} elsif ($oldRepo eq "PKGREPO") {
my ($iPkg, $iSize, $iMD5, $iDesc, $iPre, $iPost, $iReadme) = split /:/;
$iPkg =~ s/\s+//g;
$oldFlags{$iPkg} = join (":", $iPre, $iPost, $iReadme);
}
}
close ($fh);
}
# generate dependency map or repository for individual packages
sub pkg_single { sub pkg_single {
my $db = shift; my $name; my ($pname, $dbO, $oname, $pdeps, $desc, $du, $md5, $ppr);
my $status = "+ Generating "; my $count = 0; my ($pver, $url, $date); # needed for the html index
$status .= ($db eq "REPO") ? "repository\n" : "dependencies\n";
print $status;
fill_hashes_from_prevrun("PKG$db");
my $hasnew = 0;
foreach my $p (@ARGV) {
my @matches = grep /^$p#/, @packages;
if ($#matches != 0) {
print "Package '$p' not found or duplicate\n"; next;
}
my $match = $matches[0];
$name = $match;
$name =~ s/#.*//;
if ( ($db eq "DEPS") and
((! $oldDeps{$name}) or ($oldDeps{$name} ne $depends{$name})) ) {
$hasnew = 1;
} elsif ($db eq "REPO") {
$du{$match} = (-s $match);
$md5sums{$match} = digest_file_hex($match,"MD5");
if (! $descrip{$name}) {$descrip{$name} = "N.A."};
if (! $flags{$name}) {$flags{$name} = "no:no:no"};
if (! $oldFlags{$name}) { $hasnew = 1; }
}
}
return unless ($hasnew == 1);
my $dict = ($db eq "DEPS") ? "depends" : "flags"; foreach my $db ("PKGREPO", "PKGDEPS", "index.html") {
open (my $fh, ">PKG$db.new"); my $firstrun = 0; my $dbNew;
foreach my $mp (sort keys %$dict) { my $status = "+ Generating ";
$name = $mp; $name =~ s/#.*//; if ($db eq "PKGREPO") {
if ($db eq "REPO") { $status .= "repository\n";
printf $fh "%-s:%-s:%-s:%-s:%-s\n", $name, $du{$mp}, } elsif ($db eq "PKGDEPS") {
$md5sums{$mp}, $descrip{$name}, $flags{$name}; $status .= "dependencies\n";
} elsif ($db eq "DEPS") { } else {
printf $fh "%-30s:%s\n", $name, $depends{$name}; $status .= "html index\n";
}
print $status;
open (my $dbOld, "$db") or $firstrun=1;
if ( ($firstrun == 1) and ($db eq "index.html") ) {
printheader();
rename($db, "$db.new");
open ($dbNew, ">>$db.new");
} else {
open ($dbNew, ">$db.new");
}
PACKAGE: foreach my $p (sort @packages) {
$count++ if ($db eq "index.html");
$pname = $p; $pver = $p;
$pname =~ s/#.*//; $pver =~ s/.*#(.*)\.pkg\.tar\.*/$1/;
if ($db eq "PKGREPO") {
$du = (-s $p);
$md5 = digest_file_hex($p,"MD5");
$desc = (! $descrip{$pname}) ? "N.A." : $descrip{$pname};
$ppr = (! $flags{$pname}) ? "no:no:no" : $flags{$pname};
printf $dbNew "%-s:%-s:%-s:%-s:%-s\n",
$p, $du, $md5, $desc, $ppr if ($firstrun == 1);
} elsif ($db eq "PKGDEPS") {
$pdeps = (! $depends{$pname}) ? "" : $depends{$pname};
printf $dbNew "%-30s:%s\n", $pname, $pdeps if ($firstrun == 1);
} else {
$date = isotime( (stat($p))[9], 1);
$url = $p;
$url =~ s/\#/\%23/;
htmlrow($dbNew,$parity{($count % 2)},$pname,$url,$pver,
$descrip{$pname},$date) if ($firstrun == 1);
}
next PACKAGE if ($firstrun == 1);
while ($dbO = <$dbOld>) {
chomp($dbO);
if (($db eq "index.html") and ($dbO !~ m/^<tr class/)) {
# Try to ensure that header lines are copied verbatim,
# by exploiting the alphabetical sorting below.
# Not guaranteed to work with every locale!
$oname = " 0";
} else {
# should be able to extract the old pkg name from this line
$oname = $dbO;
$oname =~ s/\s*\:.*// if ($db eq "PKGDEPS");
$oname =~ s/(.*)\#.*pkg\.tar.*/$1/ if ($db eq "PKGREPO");
$oname =~ s/.*a href="(.*)\%23.*/$1/ if ($db eq "index.html");
$count++ if ($db eq "index.html");
}
if ($oname lt $pname) {
print $dbNew "$dbO\n";
} elsif (($oname ge $pname) and ($db eq "PKGREPO")) {
printf $dbNew "%-s:%-s:%-s:%-s:%-s\n", $p, $du, $md5, $desc, $ppr;
} elsif (($oname ge $pname) and ($db eq "PKGDEPS")) {
printf $dbNew "%-30s:%s\n", $pname, $pdeps
} else {
# either overwrite the old entry in the html index,
# or insert this entry before the first line that
# would come after $p when sorted alphabetically.
htmlrow($dbNew,$parity{($count % 2)},$pname,$url,$pver,
$descrip{$pname},$date);
}
print $dbNew "$dbO\n" if ($oname gt $pname);
last if ($oname ge $pname);
} }
} }
close ($fh); while ( ($firstrun != 1) and ($dbO = <$dbOld>) ) {
rename("PKG$db.new", "PKG$db"); if ($db eq "index.html") {
$count++;
$dbO =~ s/class="(even|odd)"/class="$parity{($count % 2)}"/;
}
print $dbNew $dbO;
}
close($dbNew);
($firstrun == 1) or close($dbOld);
rename("$db.new", "$db");
printfooter($count) if (($firstrun == 1) and ($db eq "index.html"));
}
} }
######################## full repository ######################## ######################## full repository ########################
# generate dependency map or the repository/index page
sub pkg_dir { sub pkg_dir {
my $db = shift; my %seen; print "+ Generating dependencies\n";
my $status = "+ Generating "; open (my $iD, ">PKGDEPS");
$status .= ($db eq "DEPS") ? "dependencies\n" : "repository\n"; print "+ Generating repository\n";
print $status; open (my $iR, ">PKGREPO");
printheader();
open (my $fh, ">PKG$db"); my $count = 0;
if ($db eq "DEPS") { open (my $ih, '>>index.html');
foreach my $name (@packages) { foreach my $p (@packages) {
$name =~ s/#.*//; next if ($seen{$name}); chomp($p);
if (($depends{$name}) and ($depends{$name} ne "")) { my $date = isotime( (stat($p))[9], 1);
printf $fh "%-30s : %-s\n", $name, $depends{$name}; $count++;
} my ($name, $version, $url) = ($p, $p, $p);
$seen{$name} = 1; $name =~ s/\#.*//;
$version =~ s/^.*\#//;
$version =~ s/\.pkg\.tar\.[gbx]z*//;
$url =~ s/\#/\%23/;
if (($depends{$name}) and ($depends{$name} ne "")) {
printf $iD "%-30s : %-s\n", $name, $depends{$name};
} }
} elsif ($db eq "REPO") { my $du = (-s $p);
our $parity = "odd"; my $md5 = digest_file_hex($p,"MD5");
my $count = 0; if (! $descrip{$name}) {$descrip{$name} = "N.A.";}
printheader(); if (! $flags{$name}) { $flags{$name} = "no:no:no"; }
open (my $ih, '>>index.html'); printf $iR "%-s:%-s:%-s:%-s:%-s\n", $p,$du,$md5,$descrip{$name},$flags{$name};
foreach my $p (@packages) { htmlrow($ih,$parity{($count % 2)},$name,$url, $version,$descrip{$name},$date);
chomp($p);
my $date = (stat($p))[9];
$count++;
my ($name, $version, $url) = ($p, $p, $p);
$name =~ s/#.*//;
$version =~ s/^.*\#//;
$version =~ s/\.pkg\.tar\.[gbx]z*//;
$url =~ s/\#/\%23/;
my $du = (-s $p);
my $md5 = digest_file_hex($p,"MD5");
$md5 =~ s/ .*$|\n//g;
if (! $descrip{$name}) {$descrip{$name} = "N.A.";}
if (! $flags{$name}) { $flags{$name} = "no:no:no"; }
printf $fh "%-s:%-s:%-s:%-s:%-s\n", $p,$du,$md5,$descrip{$name},$flags{$name};
print $ih "<tr class=\"$parity\">";
print $ih "<td>$name</td>";
print $ih "<td><a href=\"$url\">$version</a></td>";
print $ih "<td>$descrip{$name}</td>";
print $ih "<td>" . isotime($date, 1) . "</td>";
print $ih "</tr>\n";
if ($parity eq "odd") { $parity = "even"; }
else { $parity = "odd"; }
}
close $ih;
printfooter($count);
} }
close $fh; close($ih);
printfooter($count);
close($iR);
close($iD);
} }
# generate README file # consolidate all the README and install scripts for the available packages
sub pkgread { sub pkgreadscripts {
print "+ Generating README\n"; print "+ Generating README\n";
open (my $fh, '>PKGREAD'); open (my $fR, '>PKGREAD');
print $fh "# README files for repository. Do NOT remove this line.\n"; print $fR "# README files for repository. Do NOT remove this line.\n";
foreach my $name (@packages) {
$name =~ s/#.*//;
if (-f "$path{$name}/README"){
print $fh "##### PKGREADME: $name\n";
open(my $readme, "$path{$name}/README");
while (<$readme>){ print $fh $_; }
close($readme);
}
}
close $fh;
}
# generate pre-install scripts file
sub pkginst {
print "+ Generating scripts\n"; print "+ Generating scripts\n";
open (my $fh, '>PKGINST'); open (my $fS, '>PKGINST');
print $fh '#!/usr/bin/env bash print $fS '#!/usr/bin/env bash
# #
# PKGINST: pre- and post-install scripts for CRUX packages # PKGINST: pre- and post-install scripts for CRUX packages
# #
run_script() { run_script() {
case "$1" in case "$1" in
'; ';
foreach my $name (@packages) {
$name =~ s/#.*//; foreach my $name (sort @dirlist) {
$name =~ s/\#.*//;
if (-f "$path{$name}/README"){
print $fR "##### PKGREADME: $name\n";
open(my $readme, "$path{$name}/README");
while (<$readme>){ print $fR $_; }
close($readme);
}
foreach my $when ("pre", "post") { foreach my $when ("pre", "post") {
if (-f "$path{$name}/${when}-install"){ if (-f "$path{$name}/${when}-install"){
print $fh "$name.$when)\n"; print $fS " $name.$when)\n";
open(my $rs, "$path{$name}/${when}-install"); open(my $rs, "$path{$name}/${when}-install");
while (<$rs>){ while (<$rs>){
chomp; chomp;
if (! m/^#!.*sh/) { print $fh " $_\n"; } print $fS " $_\n"
} unless (m/^\#(!.*sh|\s*EOF|\s*End)/);
close($rs); }
print $fh " ;;\n"; close($rs);
} print $fS " ;;\n";
}
} }
} }
print $fh "esac\n}\n\n"; print $fS " esac\n}\n\n";
print $fh '[ "$1" ] && [[ "$2" == @(pre|post) ]] && run_script "$1.$2"'; print $fS '[ "$1" ] && [[ "$2" == @(pre|post) ]] && run_script "$1.$2"';
print $fh "\n"; print $fS "\n";
close $fh; close $fS;
close $fR;
} }
######################## html index subs ######################## ######################## html index subs ########################
sub printheader { sub printheader {
@ -294,9 +296,7 @@ EOH
print $ih " <h2>$title</h2>\n"; print $ih " <h2>$title</h2>\n";
if ($header) { if ($header) {
open(FILE, $header) or die "Couldn't open header file"; open(FILE, $header) or die "Couldn't open header file";
while (<FILE>) { while (<FILE>) { print $ih " " . $_; }
print $ih " " . $_;
}
close(FILE); close(FILE);
} }
@ -307,6 +307,13 @@ EOH
close($ih); close($ih);
} }
sub htmlrow {
my ($ih, $odd, $name, $url, $version, $desc, $date) = @_;
print $ih "<tr class=\"$odd\"><td>$name</td>";
print $ih "<td><a href=\"$url\">$version</a></td>";
print $ih "<td>$desc</td><td>$date</td></tr>\n";
}
sub printfooter { sub printfooter {
my $count = $_[0]; my $count = $_[0];
open (my $ih, '>>index.html'); open (my $ih, '>>index.html');
@ -314,9 +321,7 @@ sub printfooter {
print $ih " <p><b>$count packages</b></p>\n"; print $ih " <p><b>$count packages</b></p>\n";
if ($footer) { if ($footer) {
open(FILE, $footer) or die "Couldn't open footer file"; open(FILE, $footer) or die "Couldn't open footer file";
while (<FILE>) { while (<FILE>) { print $ih " " . $_; }
print $ih " " . $_;
}
close(FILE); close(FILE);
} }
print $ih " <p><i>Generated by <a href=\"http://www.varlock.com\">pkg-repgen</a> on " . isotime() . ".</i></p>\n"; print $ih " <p><i>Generated by <a href=\"http://www.varlock.com\">pkg-repgen</a> on " . isotime() . ".</i></p>\n";