Commit Graph

116 Commits (1d885159f2f7ff68ed75d92af584174a1208ca01)

Author SHA1 Message Date
David Phillips 69b9c2444b libutil/recurse: only opendir if recursing
Previous behaviour was to call opendir regardless of if we are actually going
to be recursing into the directory. Additionally, some utilities that use
DIRFIRST benefit from running the function pointed to by fn before the call
to opendir. One such example is `chmod [-R] 777 dir` on a directory with mode
000, where it will be expected for chmod to first give itself rwx before
optionally listing the directory to traverse it.
2017-10-01 09:34:47 -07:00
Michael Forney 51e432cc44 cp: Only call chmod with -p or -a
Previously, when the destination file was created with fopen, we needed
to use fchmod to set its permissions.

Now that we pass in the mode to creat, we already get the desired
behavior of creating the file with the same mode as the source file
modified by the user's file creation mask.

This fixes the issue where a directory or special file created with
mkdir/mknod does not end up with the appropriate mode with -p or -a
(since it may have been narrowed by the umask).

This also allows us to clear the SUID and SGID bits from the mode if the
chown fails, as specified by POSIX.
2017-07-14 07:50:50 +02:00
Michael Forney 3276fbea1c concat: Use plain read/write instead of buffered stdio
If we are just copying data from one file to another, we don't need to
fill a complete buffer, just read a chunk at a time, and write it to the
2017-07-14 07:50:47 +02:00
Michael Forney 9a3b12525b Don't use buffered IO (fread) when not appropriate
fread reads the entire requested size (BUFSIZ), which causes tools to
block if only small amounts of data are available at a time. At best,
this causes unnecessary copies and inefficiency, at worst, tools like
tee and cat are almost unusable in some cases since they only display
large chunks of data at a time.
2017-07-03 21:04:14 +02:00
Michael Forney 5cb3a1eba1 libutil: Add writeall utility function
writeall makes successive write calls to write an entire buffer to the
output file descriptor. It returns the number of bytes written, or -1 on
the first error.
2017-07-03 21:04:12 +02:00
Michael Forney 529e50a7ad mkdir: Fix created directory permissions
Previously, with -p, the specified directory and all of its parents
would be 0777&~filemask (regardless of the -m flag). POSIX says parent
directories must created as (0300|~filemask)&0777, and of course if -m
is set, the specified directory should be created with those

Additionally, POSIX says that for symbolic_mode strings, + and - should
be interpretted relative to a default mode of 0777 (not 0).

Without -p, previously the directory would be created first with
0777&~filemask (before a chmod), but POSIX says that the directory shall
at no point in time have permissions less restrictive than the -m mode

Rather than dealing with mkdir removing the filemask bits by calling
chmod afterward, just clear the umask and remove the bits manually.
2017-07-03 21:03:11 +02:00
Michael Forney 6ac5f01cc9 mkdir -p: Fail if argument exists, but is not a directory
If it is a directory, we can just return straightaway.
2017-07-03 21:03:09 +02:00
Michael Forney a5612b0d08 Remove st != NULL checks from recursor functions
In the description of 3111908b03, it says
that the functions must be able to handle st being NULL, but recurse
always passes a valid pointer. The only function that was ever passed
NULL was rm(), but this was changed to go through recurse in
2f4ab52739, so now the checks are
2017-07-03 21:03:02 +02:00
Hiltjo Posthuma af392d1a76 libutil: fix leaks 2017-05-07 13:50:26 +02:00
Michael Forney fa0e5d6378 libutil/unescape: NULL terminate unescaped string
In commit 30fd43d7f3, unescape was
simplified significantly, but the new version failed to NULL terminate
the resulting string.

This causes bad behavior in various utilities, for example tr:


  $ echo b2 | tr '\142' '\143'


  $ echo b2 | tr '\142' '\143'

This bug breaks libtool's usage of tr, causing gcc to fail to build with
2017-03-24 10:40:32 +01:00
Mattias Andrée 30fd43d7f3 libutil/unescape.c: simplify and add \E
Signed-off-by: Mattias Andrée <>
2017-02-06 15:47:01 -08:00
Mattias Andrée 9a903c63de libutil/unescape.c: only print argv0 once on error
Signed-off-by: Mattias Andrée <>
2017-02-06 14:13:40 -08:00
Michael Forney 2481042651 cp: Also preserve atime/mtime for symlinks
Laslo: Fixed style a bit and added comment
2016-12-27 14:50:58 +01:00
Michael Forney e03a57df92 cp: Check result of utimensat
POSIX says that if duplicating the modification/access times fails, then
an error should be written to stderr.
2016-12-27 14:46:11 +01:00
Michael Forney 52e49329e5 crypt: Add some missing error checks for cryptsum
Previously, if a file failed to read in a checksum list, it would be
reported as not matched rather than a read failure.

Also, if reading from stdin failed, previously a bogus checksum would be
printed anyway.
2016-12-27 14:02:32 +01:00
Michael Forney 87f40834a3 parsemode: No need to return after eprintf
Also, since parsemode exits on failure, don't bother checking return
value in xinstall (this would never trigger anyway because mode_t can be
2016-12-27 13:33:35 +01:00
Michael Forney 8ca79a2993 linecmp: Handle NUL bytes properly
Test case:

if [ "$(printf 'a\na\0b' | ./sort -u)" = "$(printf 'a\na\0b')" ] ; then
	echo pass
	echo fail
2016-07-09 10:09:50 +01:00
Mattias Andrée 44a6d65832 *sum: support - when using -c
Signed-off-by: Mattias Andrée <>
2016-03-26 08:18:47 +00:00
FRIGN 515525997c Fix linecmp() to return correct values 2016-03-11 15:38:36 +00:00
FRIGN 3debc5e064 Add linecmp() 2016-03-10 08:48:09 +00:00
FRIGN 19c0ca9830 Properly increment line lenght on edge-case in getlines() 2016-03-10 08:48:09 +00:00
FRIGN eb9bda8787 Support NUL-containing lines in sort(1)
For sort(1) we need memmem(), which I imported from OpenBSD.
Inside sort(1), the changes involved working with the explicit lengths
given by getlines() earlier and rewriting some of the functions.

Now we can handle NUL-characters in the input just fine.
2016-03-10 08:48:09 +00:00
FRIGN e4810f1cdb Support NUL-containing lines in cols(1)
This required an architectural change in getlines() by also storing
the line length.
2016-03-10 08:48:09 +00:00
FRIGN a88906b423 Rever the strmem() addition and add a TODO element
strmem() was not very well thought out. The thing is the following:
If the string contains a zero character, we want to match it, and not
stop right there in place.

The "real" solution is to use memmem() where needed and replace all
functions that assume zero-terminated-strings from standard input, which
could lead to early string-breakoffs.
This requires a strict tracking of string lengths.
2016-02-26 09:54:46 +00:00
FRIGN 3396088666 Implement strmem() and use it in join(1)
We want our delimiters to also contain 0 characters and have them
handled gracefully.
To accomplish this, I wrote a function strmem(), which looks for a
certain, arbitrarily long memory subset in a given string.
memmem() is a GNU extension and forces you to call strlen every time.
2016-02-26 09:54:46 +00:00
Mattias Andrée a392cd475e add sha512-224sum (SHA512/224) and sha512-256sum (SHA512/256)
Signed-off-by: Mattias Andrée <>
2016-02-24 10:40:57 +00:00
Mattias Andrée ae1da536bb add sha224sum and sha384sum
Signed-off-by: Mattias Andrée <>
2016-02-24 10:15:16 +00:00
FRIGN 6adb9b8ccd Fix compilation error 2016-02-21 08:52:48 +00:00
FRIGN 70adb1252d Do a range check on the resulting octal 2016-02-21 08:52:48 +00:00
FRIGN 41a600e1b8 Allow \0ooo octal escapes
Yeah well, the old topic. POSIX allows \0123 and \123 octals in
different tools, in printf, depending on %b or other things.
We'll just keep it simple and just allow 4 digits. the 0 does not make
a difference anyway.
2016-02-21 08:52:48 +00:00
FRIGN e40fc2b176 Check argv0 in xvprintf()
You never know, given printf'ing NULL-strings might crash the
program, we shouldn't just pass argv0 blindly to it.
2015-12-21 14:13:36 +00:00
sin 3b1b50cffa Do not indent label 2015-12-21 09:55:01 +00:00
FRIGN 94e92d9cc0 Refactor eprintf.c
When we move the exit() out of venprintf(), we can reuse it for
weprintf(), which basically had duplicate code.
I also renamed venprintf() to xvprintf (extended vprintf) so it's
more obvious what it actually does.
2015-12-21 09:34:13 +00:00
sin 67157b7c4e Use SSIZE_MAX for overflow check in parseoffset()
There's no such thing as OFF_MAX so we can't use that.  off_t is
signed, so use SSIZE_MAX which will typically match the range of
2015-11-26 10:35:46 +00:00
sin d24584466c Revert "enmasse: For the special case of 2 args, do not distinguish between dirs and files"
This reverts commit a564a67c4ea70e90a4dc543814458e4903869d3e.

Not as trivial as I thought.  This breaks cp when used as:
cp -r /foo/bar /baz

The old code expands this to:
cp -r /foo/bar /baz/bar
2015-11-13 14:31:34 +00:00
sin 4c1f9ecdd2 Align end of comment 2015-11-13 14:24:09 +00:00
sin 996f992ac6 enmasse: For the special case of 2 args, do not distinguish between dirs and files
This produces consistent error messages when moving or copying a file or dir to
2015-11-13 14:21:07 +00:00
FRIGN dae33f42d4 Fix typo in libutil/fshut.c 2015-10-26 16:53:28 +00:00
FRIGN 8be7c42863 Make strtol() parsing even stricter in parseoffset()
Be strict about what we pass to it and how we handle errors.
The base-check is done by strtol anyway.
Also improve error-reporting.
2015-09-30 19:44:11 +01:00
FRIGN 870a75076d Harden parseoffset() even more
1) Check for NULL.
2) Check for empty strings.
3) Clarify error-messages.
2015-09-30 19:44:10 +01:00
FRIGN 64929039e9 Don't forget to scale in parseoffset() 2015-09-30 19:44:10 +01:00
FRIGN 007df69fc5 Add parseoffset()
This is a utility function to allow easy parsing of file or other
offsets, automatically taking in regard suffixes, proper bases and
so on, for instance used in split(1) -b or od -j, -N(1).
Of course, POSIX is very arbitrary when it comes to defining the
parsing rules for different tools.
The main focus here lies on being as flexible and consistent as
possible. One central utility-function handling the parsing makes
this stuff a lot more trivial.
2015-09-30 19:44:10 +01:00
Hiltjo Posthuma 53be158979 code-style: whitespace fixes 2015-09-30 19:44:10 +01:00
Michael Forney 1d28fbd6cf mv, cp: Preserve nanosecond timestamps
Otherwise, we run into problems in a typical autoconf-based build

  - config.status is created at some point between two seconds.
  - config.status is run, generating Makefile by first writing to a file
    in /tmp, and then mv-ing it to Makefile.
  - If this mv happens before the beginning of the next second, Makefile
    will be created with the same tv_sec as config.status, but with
    tv_nsec = 0.
  - When make runs, it sees that Makefile is older than config.status,
    and re-runs config.status to generate Makefile.
2015-05-16 13:34:00 +01:00
FRIGN 0545d32ce9 Handle '-' consistently
In general, POSIX does not define /dev/std{in, out, err} because it
does not want to depend on the dev-filesystem.
For utilities, it thus introduced the '-'-keyword to denote standard
input (and output in some cases) and the programs have to deal with
it accordingly.

Sadly, the design of many tools doesn't allow strict shell-redirections
and many scripts don't even use this feature when possible.

Thus, we made the decision to implement it consistently across all
tools where it makes sense (namely those which read files).

Along the way, I spotted some behavioural bugs in libutil/crypt.c and
others where it was forgotten to fshut the files after use.
2015-05-16 13:34:00 +01:00
Hiltjo Posthuma 29649762b3 libutil/getlines: fix potential crash
linelen was uninitialized if for example:

$ > empty
$ sort ls.c empty
2015-05-08 16:38:06 +01:00
Hiltjo Posthuma 3f01706837 libutil/getlines: use known line length
also style: linelen = length of getline(), this was slightly confusing.
2015-05-07 18:18:36 +01:00
Hiltjo Posthuma adf9f47525 Revert "libutil/getlines: use known line length"
This reverts commit c69a70ddfd5c2b1514d9efd1c7a0fcbee5b0d2e7.
2015-05-07 18:18:36 +01:00
Hiltjo Posthuma bd67e7d92d libutil/getlines: use known line length 2015-05-07 18:18:35 +01:00
sin 2deb40290e Use off_t in humansize() as it is more descriptive and applicable 2015-04-29 16:42:49 +01:00