fix PKGINST to accommodate ports with dashes in their names.

respect --install-root when configured with 'runscripts yes'.

streamline the pkg-repgen script.
This commit is contained in:
John McQuah 2023-06-17 11:24:34 -04:00
parent 98a2df234e
commit f769b5251e
6 changed files with 922 additions and 1201 deletions

View File

@ -1,5 +1,12 @@
ChangeLog for pkg-get
0.4.8 - Fix pre- and post-install scripts to accommodate rootfs other
than "/", and ports with dashes in their names
- Use Perl modules where possible, to avoid spawning external processes
- Consolidate code
0.4.7 - Fix man-page location
0.4.6 - Fixed warnings on output of diff command
- Use compression-mode defined in pkgmk.conf
- pkg-repgen.pl: Improved prt-get commands and added --prtdir switch

57
README
View File

@ -1,8 +1,8 @@
INTRODUCTION
----------------------------------------------------------------------------
pkg-get is a package / repository management tool for CRUX Linux.
Syntax and features are very close (often a carbon copy)
to the ones found in the port management tool 'prt-get'
Syntax and features are very close to (often a carbon copy of)
the ones found in the port management tool 'prt-get'
by Johannes Winkelmann.
In fact pkg-get was developed as a prt-get/ports drop-in replacement
for systems in which it is preferable to handle binary packages instead
@ -11,10 +11,10 @@ of compiling ports.
ARCHITECTURE
----------------------------------------------------------------------------
The local machines sync metadata files (available packages,
readme files, dependencies, etc) from a remote (http or ftp)
The client machines sync metadata files (available packages,
readme files, dependencies, etc) from a remote server (http or ftp)
OR a local path.
Once the metadata is present on the local machine, the usual
Once the metadata files are on the client machine, the usual
operations of installing, removing, getting info on packages
are available.
@ -24,15 +24,17 @@ QUICK START
Server:
A repository can be generated using 'pkg-repgen' in a
dir containing packages. It will take a while since md5sums
have to be calculated.
have to be calculated. Alternatively, you can pass one or
more arguments to 'pkg-repgen', indicating the individual
packages for which metadata will be created.
Client:
Adjust settings in /etc/pkg-get.conf, then use the 'pkg-get sync'
command to gather metadata from the server (if remote). You can now
use the commands as described in the manual, i.e.:
use the commands as described in the manual, e.g.:
pkg-get info apache
pkg-get depinst kdebase
pkg-get depinst qt6-base
pkg-get listinst
See the manual page for a detailed list of commands and options.
@ -40,5 +42,42 @@ See the manual page for a detailed list of commands and options.
REQUIREMENTS
----------------------------------------------------------------------------
For the client nothing outside the CRUX 'core' collection
For the client, nothing outside the CRUX 'core' collection
For the server, prt-get
LIMITATIONS
----------------------------------------------------------------------------
The client and the server must be configured to use the same
pkgmk compression mode, otherwise the client will try to download
a tarball with the wrong suffix. This is only a problem if you sometimes
compile ports on the client device. By allowing you to maintain your
client device solely with binary packages, pkg-get makes the contents of
/etc/pkgmk.conf mostly irrelevant.
'pkg-get depends' and 'prt-get quickdep' do not handle more than one port,
unlike the corresponding commands in prt-get. Therefore it is not as
straightforward to preview the list of packages that would be installed,
before running a 'depinst' operation with multiple targets.
The limitation above would have been mitigated by a --test switch.
Alas, such a switch is also absent from the design of pkg-get. Use
the --test switch with prt-get itself, for the closest approximation
of previewing the outcome from a 'pkg-get depinst' operation.
Among the prt-get commands that have no counterpart in pkg-get
(grpinst, fsearch, deptree, listorphans, ls, cat, edit, cache),
only the 'grpinst' command is of possible interest; the remaining
commands are just as easily delegated to prt-get itself. If you want
a Perl implementation that does provide these missing commands, consider
the script written by user farkuhar [1].
pkg-get only makes use of the hard dependencies listed by the port
maintainer, not any of the eager linking that might have occurred on the
build machine. As a result, 'pkg-get depinst foo' might omit some of the
packages needed by 'foo'. User ppetrov^ has contributed some helper scripts
to facilitate the fixing of these broken binaries; visit the site [2] to
download them.
[1] https://git.sdf.org/jmq/Documentation/src/branch/master/scripts/prt-auf
[2] https://github.com/slackalaxy/depsck

6
TODO
View File

@ -4,8 +4,10 @@ TODO file for pkg-get
- add more commands:
- deptree (?)
- lock/unlock/listlocked (?)
- grpinst (?)
- optimize the pkg-repgen script
- allow 'depends' and 'quickdep' to process multiple arguments
- add a --test switch (?)
- improve pkg-get help information

View File

@ -3,7 +3,7 @@
# pkg-get configuration file
# package repositories (remote)
# The first two are remote repoistories, the last is a local one
# The first two are remote repositories, the last is a local one
pkgdir /usr/packages/server|http://www.somesite.com/packages
pkgdir /usr/packages/java|http://www.foobar.com/java
pkgdir /usr/packages/games

File diff suppressed because it is too large Load Diff

View File

@ -12,6 +12,7 @@
use warnings;
use strict;
use Getopt::Long;
use Digest::file qw(digest_file_hex);
our $prtget = "/usr/bin/prt-get"; our $prtdir;
our $title = "CRUX Packages"; our $header; our $footer;
@ -21,219 +22,218 @@ GetOptions("prtdir=s"=>\$prtdir, "title=s"=>\$title, "header=s"=>\$header, "foot
our $compress = "gz";
open CONFIG, "/etc/pkgmk.conf" or die "Could not open /etc/pkgmk.conf";
while (<CONFIG>) {
$compress = $1 if m/^PKGMK_COMPRESSION_MODE="(.*)"\n/;
$compress = $1 if m/^PKGMK_COMPRESSION_MODE=(.*)(#|$)/;
}
close CONFIG;
$compress =~ s/["' ]//g;
$prtget .= " --no-std-config --config-set=\"prtdir $prtdir\"" if ($prtdir);
my @dirlist = glob("*#*.pkg.tar.$compress"); my @packages;
if ($#ARGV >= 0) { # single packages
pkgrepo_single();
pkgdeps_single();
pkgread();
pkginst();
foreach my $pkgname (@ARGV) {
my @hits = grep /^$pkgname#/ @dirlist;
push(@packages,@hits) if (@hits);
}
} else {
if ($prtdir) {
$prtget = "$prtget --no-std-config --config-set=\"prtdir $prtdir\"";
@packages = @dirlist;
}
# Populate some hashes with a single run of prt-get
our %path; our %depends; our %descrip; our %flags;
our %oldDeps; our %oldFlags; our %du; our %md5sums;
fill_hashes_from_prtget();
if ($#ARGV >= 0) {
pkg_single("REPO"); pkg_single("DEPS");
} else {
pkg_dir("REPO"); pkg_dir("DEPS");
}
# Generate README and PKGINST
pkgread();
pkginst();
sub fill_hashes_from_prtget {
my @validkeys = @packages;
map { s/#.*// } @validkeys;
open (my $ppf, "$prtget printf '%n^%p^%e^%d^%E^%O^%R\n' |");
while (<$ppf>) {
my ($name,$repo,$deps,$desc,$haspre,$haspost,$hasreadme) = split /\^/;
next if (! grep { ($_ eq $name) } @validkeys);
$path{$name} = $repo . "/" . $name;
$depends{$name} = $deps;
$desc =~ s/\:/ /g;
$descrip{$name} = $desc;
chomp($hasreadme);
$flags{$name} = join(":", $haspre, $haspost, $hasreadme);
}
pkgrepo();
pkgdeps();
pkgread();
pkginst();
close ($ppf);
}
######################## single packages ########################
# generate dependencies
sub pkgdeps_single {
print "+ Generating dependencies\n";
my $hasnew = 0;
foreach my $p (@ARGV) {
my @packages = glob("$p#*.pkg.tar.$compress");
if ($#packages == 0) {
my $found = 0;
my $package = $packages[0];
$package =~ s/#.*//;
my $deps = `$prtget printf "%e" --filter="$package"`;
if ($deps ne "") {
my $isnew = `grep "$p .*:" PKGDEPS`;
if ($isnew eq ""){ # package is new, put deps at the end.
open (my $fh, '>>PKGDEPS');
printf $fh "%-30s : %-s\n", $package, $deps;
close $fh;
$hasnew = 1;
} else {
system("sed -i \"/^$p /s/: .*\$/: $deps/\" PKGDEPS");
}
}
} else {
print "Package '$p' not found or duplicate\n"
}
}
if ($hasnew == 1){system("sort -o PKGDEPS PKGDEPS")};
}
# generate the main repository file
sub pkgrepo_single {
print "+ Generating repository\n";
my $hasnew = 0;
foreach my $p (@ARGV) {
my @packages = glob("$p#*.pkg.tar.$compress");
if ($#packages == 0) {
my $found = 0;
my $package = $packages[0];
my $name = $package;
$name =~ s/#.*//;
my $du = (-s $package);
my $md5 = `md5sum $package`;
$md5 =~ s/ .*$|\n//g;
my $des=`$prtget printf %d --filter="$name"`;
$des =~ s/:/ /g;
if ($des eq ""){$des = "N.A."};
my $flags=`$prtget printf %E:%O:%R --filter="$name"`;
if ($flags eq "") {$flags = "no:no:no"}
my $isnew = `grep "$p#" PKGREPO`;
if ($isnew eq ""){ # package is new, put it at the end
open (my $fh, '>>PKGREPO');
printf $fh "%-s:%-s:%-s:%-s:%-s\n", $package,$du,$md5,$des,$flags;
close $fh;
$hasnew = 1;
} else {
my $newp = "$package:$du:$md5:$des:$flags";
system("sed -i \"s/^$p#.*\$/$newp/\" PKGREPO");
}
#printf $fh "%-s:%-s:%-s:%-s\n", $du,$md5,$des,$flags;
} else {
print "Package '$p' not found or duplicate\n"
sub fill_hashes_from_prevrun {
my $oldRepo = shift;
open (my $fh, $oldRepo) or return;
while (<$fh>) {
chomp;
if ($oldRepo eq "PKGDEPS") {
my ($iPkg, $iDep) = split /:/;
$iPkg =~ s/\s+//g;
$iDep =~ s/,/ /g;
$oldDeps{$iPkg} = $iDep;
} elsif ($oldRepo eq "PKGREPO") {
my ($iPkg, $iSize, $iMD5, $iDesc, $iPre, $iPost, $iReadme) = split /:/;
$iPkg =~ s/\s+//g;
$oldFlags{$iPkg} = join (":", $iPre, $iPost, $iReadme);
}
}
if ($hasnew == 1){system("sort -o PKGREPO PKGREPO")};
close ($fh);
}
# generate dependency map or repository for individual packages
sub pkg_single {
my $db = shift; my $name;
my $status = "+ Generating ";
$status .= ($db eq "REPO") ? "repository\n" : "dependencies\n";
print $status;
fill_hashes_from_prevrun("PKG$db");
my $hasnew = 0;
foreach my $p (@ARGV) {
my @matches = grep /^$p#/, @packages;
if ($#matches != 0) {
print "Package '$p' not found or duplicate\n"; next;
}
my $match = $matches[0];
$name = $match;
$name =~ s/#.*//;
if ( ($db eq "DEPS") and
((! $oldDeps{$name}) or ($oldDeps{$name} ne $depends{$name})) ) {
$hasnew = 1;
} elsif ($db eq "REPO") {
$du{$match} = (-s $match);
$md5sums{$match} = digest_file_hex($match,"MD5");
if (! $descrip{$name}) {$descrip{$name} = "N.A."};
if (! $flags{$name}) {$flags{$name} = "no:no:no"};
if (! $oldFlags{$name}) { $hasnew = 1; }
}
}
return unless ($hasnew == 1);
my $dict = ($db eq "DEPS") ? "depends" : "flags";
open (my $fh, ">PKG$db.new");
foreach my $mp (sort keys %$dict) {
$name = $mp; $name =~ s/#.*//;
if ($db eq "REPO") {
printf $fh "%-s:%-s:%-s:%-s:%-s\n", $name, $du{$mp},
$md5sums{$mp}, $descrip{$name}, $flags{$name};
} elsif ($db eq "DEPS") {
printf $fh "%-30s:%s\n", $name, $depends{$name};
}
}
close ($fh);
rename("PKG$db.new", "PKG$db");
}
######################## full repository ########################
# generate dependencies
sub pkgdeps {
print "+ Generating dependencies\n";
my @packages = glob("*#*.pkg.tar.$compress");
open (my $fh, '>PKGDEPS');
foreach my $package (@packages) {
$package =~ s/#.*//;
my $deps = `$prtget printf "%e" --filter="$package"`;
if ($deps ne "") {
printf $fh "%-30s : %-s\n", $package, $deps;
# generate dependency map or the repository/index page
sub pkg_dir {
my $db = shift; my %seen;
my $status = "+ Generating ";
$status .= ($db eq "DEPS") ? "dependencies\n" : "repository\n";
print $status;
open (my $fh, ">PKG$db");
if ($db eq "DEPS") {
foreach my $name (@packages) {
$name =~ s/#.*//; next if ($seen{$name});
if (($depends{$name}) and ($depends{$name} ne "")) {
printf $fh "%-30s : %-s\n", $name, $depends{$name};
}
$seen{$name} = 1;
}
}
close $fh;
}
} elsif ($db eq "REPO") {
our $parity = "odd";
my $count = 0;
printheader();
open (my $ih, '>>index.html');
foreach my $p (@packages) {
chomp($p);
my $date = (stat($p))[9];
$count++;
my ($name, $version, $url) = ($p, $p, $p);
$name =~ s/#.*//;
$version =~ s/^.*\#//;
$version =~ s/\.pkg\.tar\.[gbx]z*//;
$url =~ s/\#/\%23/;
my $du = (-s $p);
my $md5 = digest_file_hex($p,"MD5");
$md5 =~ s/ .*$|\n//g;
if (! $descrip{$name}) {$descrip{$name} = "N.A.";}
if (! $flags{$name}) { $flags{$name} = "no:no:no"; }
printf $fh "%-s:%-s:%-s:%-s:%-s\n", $p,$du,$md5,$descrip{$name},$flags{$name};
print $ih "<tr class=\"$parity\">";
print $ih "<td>$name</td>";
print $ih "<td><a href=\"$url\">$version</a></td>";
print $ih "<td>$descrip{$name}</td>";
print $ih "<td>" . isotime($date, 1) . "</td>";
print $ih "</tr>\n";
# generate the main repository file and index page
sub pkgrepo {
print "+ Generating repository\n";
my @packages = glob("*#*.pkg.tar.$compress");
our $odd = "odd";
my $count = 0;
open (my $fh, '>PKGREPO');
printheader();
open (my $ih, '>>index.html');
foreach my $package (@packages) {
my $date = (stat($package))[9];
$count++;
$package =~ s/\n//g;
my $name = $package;
$name =~ s/#.*//;
my $du = (-s $package);
my $md5 = `md5sum $package`;
$md5 =~ s/ .*$|\n//g;
my $des=`$prtget printf %d --filter="$name"`;
$des =~ s/:/ /g;
if ($des eq ""){$des = "N.A."};
my $flags=`$prtget printf %E:%O:%R --filter="$name"`;
if ($flags eq "") {$flags = "no:no:no"}
printf $fh "%-s:%-s:%-s:%-s:%-s\n", $package,$du,$md5,$des,$flags;
my $version = $package;
$version =~ s/^.*\#//;
$version =~ s/\.pkg\.tar\.[gbx]z*//;
print $ih "<tr class=\"$odd\">";
print $ih "<td>$name</td>";
my $url = $package;
$url =~ s/\#/\%23/;
print $ih "<td><a href=\"$url\">$version</a></td>";
print $ih "<td>$des</td>";
print $ih "<td>" . isotime($date, 1) . "</td>";
print $ih "</tr>\n";
if ($odd eq "odd") { $odd = "even"; }
else { $odd = "odd"; }
if ($parity eq "odd") { $parity = "even"; }
else { $parity = "odd"; }
}
close $ih;
printfooter($count);
}
close $fh;
close $ih;
printfooter($count);
}
# generate README file
sub pkgread {
print "+ Generating README\n";
my @packages = glob("*#*.pkg.tar.$compress");
open (my $fh, '>PKGREAD');
print $fh "# README files for repository. Do NOT remove this line.\n";
foreach my $package (@packages) {
$package =~ s/#.*//;
my $path = `$prtget path $package`;
$path =~ s/\n//g;
if (-f "$path/README"){
print $fh "##### PKGREADME: $package\n";
open(my $readme, "$path/README");
while (<$readme>){
my $line = $_;
print $fh $line;
}
foreach my $name (@packages) {
$name =~ s/#.*//;
if (-f "$path{$name}/README"){
print $fh "##### PKGREADME: $name\n";
open(my $readme, "$path{$name}/README");
while (<$readme>){ print $fh $_; }
close($readme);
}
}
close $fh;
}
# generate pre-post install scripts file
# generate pre-install scripts file
sub pkginst {
print "+ Generating scripts\n";
open (my $fh, '>PKGINST');
print $fh
"
#!/bin/bash
print $fh '#!/usr/bin/env bash
#
# PKGINST: pre-post install scripts for CRUX packages
";
my @packages = glob("*#*.pkg.tar.$compress");
foreach my $package (@packages) {
$package =~ s/#.*//;
my $path = `$prtget path $package`;
$path =~ s/\n//g;
my $normal= $package;
$normal =~ s/[^[:alnum:]]/_/g;
if (-f "$path/pre-install"){
print $fh "${normal}_pre_install() {\n";
open(my $pre, "$path/pre-install");
while (<$pre>){
my $line = $_;
print $fh $line;
}
close($pre);
print $fh "}\n\n";
# PKGINST: pre- and post-install scripts for CRUX packages
#
run_script() {
case "$1" in
';
foreach my $name (@packages) {
$name =~ s/#.*//;
foreach my $when ("pre", "post") {
if (-f "$path{$name}/${when}-install"){
print $fh "$name.$when)\n";
open(my $rs, "$path{$name}/${when}-install");
while (<$rs>){
chomp;
if (! m/^#!.*sh/) { print $fh " $_\n"; }
}
close($rs);
print $fh " ;;\n";
}
if (-f "$path/post-install"){
print $fh "${normal}_post_install() {\n";
open(my $post, "$path/post-install");
while (<$post>){
my $line = $_;
print $fh $line;
}
close($post);
print $fh "}\n\n";
}
}
print $fh "\n\n";
print $fh 'if [ ! -z "$1" ]; then $1; fi';
print $fh "esac\n}\n\n";
print $fh '[ "$1" ] && [[ "$2" == @(pre|post) ]] && run_script "$1.$2"';
print $fh "\n";
close $fh;
}
@ -243,16 +243,16 @@ sub pkginst {
sub printheader {
open (my $ih, '>index.html');
print $ih <<EOH;
print $ih <<EOH;
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
EOH
print $ih " <title>$title</title>\n";
print $ih " <title>$title</title>\n";
print $ih <<EOH;
print $ih <<EOH;
<style type="text/css">
body
{
@ -291,55 +291,50 @@ EOH
<body>
EOH
print $ih " <h2>$title</h2>\n";
if ($header) {
open(FILE, $header) or die "Couldn't open header file";
while (<FILE>) {
print $ih " " . $_;
}
close(FILE);
}
print $ih " <h2>$title</h2>\n";
if ($header) {
open(FILE, $header) or die "Couldn't open header file";
while (<FILE>) {
print $ih " " . $_;
}
close(FILE);
}
print $ih " <table width=\"100%\" cellspacing=\"0\">\n";
print $ih " <tr class=\"header\"><td><b>Port</b></td><td><b>Version</b></td><td><b>Description</b></td>";
print $ih "<td><b>Last modified</b></td>";
print $ih "</tr>\n";
close($ih);
print $ih " <table width=\"100%\" cellspacing=\"0\">\n";
print $ih " <tr class=\"header\"><td><b>Port</b></td><td><b>Version</b></td><td><b>Description</b></td>";
print $ih "<td><b>Last modified</b></td>";
print $ih "</tr>\n";
close($ih);
}
sub printfooter {
my $count = $_[0];
my $count = $_[0];
open (my $ih, '>>index.html');
print $ih " </table>\n";
print $ih " <p><b>$count packages</b></p>\n";
if ($footer) {
open(FILE, $footer) or die "Couldn't open footer file";
while (<FILE>) {
print $ih " " . $_;
}
close(FILE);
}
print $ih " <p><i>Generated by <a href=\"http://www.varlock.com\">pkg-repgen</a> on " . isotime() . ".</i></p>\n";
print $ih <<EOH;
print $ih " </table>\n";
print $ih " <p><b>$count packages</b></p>\n";
if ($footer) {
open(FILE, $footer) or die "Couldn't open footer file";
while (<FILE>) {
print $ih " " . $_;
}
close(FILE);
}
print $ih " <p><i>Generated by <a href=\"http://www.varlock.com\">pkg-repgen</a> on " . isotime() . ".</i></p>\n";
print $ih <<EOH;
</body>
</html>
EOH
close($ih);
close($ih);
}
sub isotime {
my $time = (shift or time);
my $accuracy = (shift or 2);
my @t = gmtime ($time);
my $year = $t[5] + 1900;
my $month = sprintf("%02d", $t[4] + 1);
my $day = sprintf("%02d", $t[3]);
my $time = (shift or time);
my $accuracy = (shift or 2);
my @t = gmtime ($time);
my $year = $t[5] + 1900;
my $month = sprintf("%02d", $t[4] + 1);
my $day = sprintf("%02d", $t[3]);
if ($accuracy == 1) {
return "$year-$month-$day";
}
return "$year-$month-$day " . sprintf("%02d:%02d:%02d UTC", $t[2], $t[1], $t[0]);
return "$year-$month-$day" if ($accuracy == 1);
return "$year-$month-$day " . sprintf("%02d:%02d:%02d UTC", $t[2], $t[1], $t[0]);
}