Commit Graph

97 Commits

Author SHA1 Message Date
FRIGN
3debc5e064 Add linecmp() 2016-03-10 08:48:09 +00:00
FRIGN
19c0ca9830 Properly increment line lenght on edge-case in getlines() 2016-03-10 08:48:09 +00:00
FRIGN
eb9bda8787 Support NUL-containing lines in sort(1)
For sort(1) we need memmem(), which I imported from OpenBSD.
Inside sort(1), the changes involved working with the explicit lengths
given by getlines() earlier and rewriting some of the functions.

Now we can handle NUL-characters in the input just fine.
2016-03-10 08:48:09 +00:00
FRIGN
e4810f1cdb Support NUL-containing lines in cols(1)
This required an architectural change in getlines() by also storing
the line length.
2016-03-10 08:48:09 +00:00
FRIGN
a88906b423 Rever the strmem() addition and add a TODO element
strmem() was not very well thought out. The thing is the following:
If the string contains a zero character, we want to match it, and not
stop right there in place.

The "real" solution is to use memmem() where needed and replace all
functions that assume zero-terminated-strings from standard input, which
could lead to early string-breakoffs.
This requires a strict tracking of string lengths.
2016-02-26 09:54:46 +00:00
FRIGN
3396088666 Implement strmem() and use it in join(1)
We want our delimiters to also contain 0 characters and have them
handled gracefully.
To accomplish this, I wrote a function strmem(), which looks for a
certain, arbitrarily long memory subset in a given string.
memmem() is a GNU extension and forces you to call strlen every time.
2016-02-26 09:54:46 +00:00
Mattias Andrée
a392cd475e add sha512-224sum (SHA512/224) and sha512-256sum (SHA512/256)
Signed-off-by: Mattias Andrée <maandree@kth.se>
2016-02-24 10:40:57 +00:00
Mattias Andrée
ae1da536bb add sha224sum and sha384sum
Signed-off-by: Mattias Andrée <maandree@kth.se>
2016-02-24 10:15:16 +00:00
FRIGN
6adb9b8ccd Fix compilation error 2016-02-21 08:52:48 +00:00
FRIGN
70adb1252d Do a range check on the resulting octal 2016-02-21 08:52:48 +00:00
FRIGN
41a600e1b8 Allow \0ooo octal escapes
Yeah well, the old topic. POSIX allows \0123 and \123 octals in
different tools, in printf, depending on %b or other things.
We'll just keep it simple and just allow 4 digits. the 0 does not make
a difference anyway.
2016-02-21 08:52:48 +00:00
FRIGN
e40fc2b176 Check argv0 in xvprintf()
You never know, given printf'ing NULL-strings might crash the
program, we shouldn't just pass argv0 blindly to it.
2015-12-21 14:13:36 +00:00
sin
3b1b50cffa Do not indent label 2015-12-21 09:55:01 +00:00
FRIGN
94e92d9cc0 Refactor eprintf.c
When we move the exit() out of venprintf(), we can reuse it for
weprintf(), which basically had duplicate code.
I also renamed venprintf() to xvprintf (extended vprintf) so it's
more obvious what it actually does.
2015-12-21 09:34:13 +00:00
sin
67157b7c4e Use SSIZE_MAX for overflow check in parseoffset()
There's no such thing as OFF_MAX so we can't use that.  off_t is
signed, so use SSIZE_MAX which will typically match the range of
off_t.
2015-11-26 10:35:46 +00:00
sin
d24584466c Revert "enmasse: For the special case of 2 args, do not distinguish between dirs and files"
This reverts commit a564a67c4ea70e90a4dc543814458e4903869d3e.

Not as trivial as I thought.  This breaks cp when used as:
cp -r /foo/bar /baz

The old code expands this to:
cp -r /foo/bar /baz/bar
2015-11-13 14:31:34 +00:00
sin
4c1f9ecdd2 Align end of comment 2015-11-13 14:24:09 +00:00
sin
996f992ac6 enmasse: For the special case of 2 args, do not distinguish between dirs and files
This produces consistent error messages when moving or copying a file or dir to
itself.
2015-11-13 14:21:07 +00:00
FRIGN
dae33f42d4 Fix typo in libutil/fshut.c 2015-10-26 16:53:28 +00:00
FRIGN
8be7c42863 Make strtol() parsing even stricter in parseoffset()
Be strict about what we pass to it and how we handle errors.
The base-check is done by strtol anyway.
Also improve error-reporting.
2015-09-30 19:44:11 +01:00
FRIGN
870a75076d Harden parseoffset() even more
1) Check for NULL.
2) Check for empty strings.
3) Clarify error-messages.
2015-09-30 19:44:10 +01:00
FRIGN
64929039e9 Don't forget to scale in parseoffset() 2015-09-30 19:44:10 +01:00
FRIGN
007df69fc5 Add parseoffset()
This is a utility function to allow easy parsing of file or other
offsets, automatically taking in regard suffixes, proper bases and
so on, for instance used in split(1) -b or od -j, -N(1).
Of course, POSIX is very arbitrary when it comes to defining the
parsing rules for different tools.
The main focus here lies on being as flexible and consistent as
possible. One central utility-function handling the parsing makes
this stuff a lot more trivial.
2015-09-30 19:44:10 +01:00
Hiltjo Posthuma
53be158979 code-style: whitespace fixes 2015-09-30 19:44:10 +01:00
Michael Forney
1d28fbd6cf mv, cp: Preserve nanosecond timestamps
Otherwise, we run into problems in a typical autoconf-based build
system:

  - config.status is created at some point between two seconds.
  - config.status is run, generating Makefile by first writing to a file
    in /tmp, and then mv-ing it to Makefile.
  - If this mv happens before the beginning of the next second, Makefile
    will be created with the same tv_sec as config.status, but with
    tv_nsec = 0.
  - When make runs, it sees that Makefile is older than config.status,
    and re-runs config.status to generate Makefile.
2015-05-16 13:34:00 +01:00
FRIGN
0545d32ce9 Handle '-' consistently
In general, POSIX does not define /dev/std{in, out, err} because it
does not want to depend on the dev-filesystem.
For utilities, it thus introduced the '-'-keyword to denote standard
input (and output in some cases) and the programs have to deal with
it accordingly.

Sadly, the design of many tools doesn't allow strict shell-redirections
and many scripts don't even use this feature when possible.

Thus, we made the decision to implement it consistently across all
tools where it makes sense (namely those which read files).

Along the way, I spotted some behavioural bugs in libutil/crypt.c and
others where it was forgotten to fshut the files after use.
2015-05-16 13:34:00 +01:00
Hiltjo Posthuma
29649762b3 libutil/getlines: fix potential crash
linelen was uninitialized if for example:

$ > empty
$ sort ls.c empty
2015-05-08 16:38:06 +01:00
Hiltjo Posthuma
3f01706837 libutil/getlines: use known line length
also style: linelen = length of getline(), this was slightly confusing.
2015-05-07 18:18:36 +01:00
Hiltjo Posthuma
adf9f47525 Revert "libutil/getlines: use known line length"
This reverts commit c69a70ddfd5c2b1514d9efd1c7a0fcbee5b0d2e7.
2015-05-07 18:18:36 +01:00
Hiltjo Posthuma
bd67e7d92d libutil/getlines: use known line length 2015-05-07 18:18:35 +01:00
sin
2deb40290e Use off_t in humansize() as it is more descriptive and applicable 2015-04-29 16:42:49 +01:00
Dionysis Grigoropoulos
2d6cde1862 humansize: Use uintmax_t for size
du(1) breaks on 32-bit size_t for files greater than 4G.
2015-04-28 11:36:58 +01:00
FRIGN
5595af5742 Convert humansize() to accept a size_t instead of a double
General convention is to use size_t to store sizes of all kinds.
Internally, the function uses double anyway, but at least this
doesn't clobber up the API any more and there's a chance in the
future to make this function a bit cleaner and not use this dirty
static buffer hack any more.
2015-04-25 11:43:14 +01:00
sin
10b57e8a3d Actually print <space> to stream in putword() too 2015-04-21 18:00:47 +01:00
sin
c914a2feca Update putword() to accept a FILE * 2015-04-21 18:00:47 +01:00
sin
b9d60bee87 Move mkdirp() to libutil 2015-04-20 18:04:08 +01:00
sin
31af8555a7 Add LICENSE header to fshut.c 2015-04-20 18:04:08 +01:00
FRIGN
f83d7bc647 Add SILENT flag to recurse()
recurse() is getting smarter every day. I expect it to pass the Turing
test in a few months.
Along the way, it was reported that "rm -f" on nonexistant files reports
their missing as an internal recurse()-error.
So recurse() knows when to shut up, I added the SILENT flag to fix all
these things.
2015-04-20 11:12:40 +01:00
FRIGN
7b2465c101 Add maxdepth to recurse()
This also makes more sense.
2015-04-20 11:12:40 +01:00
FRIGN
e14d9412f8 Properly handle recursion in recurse()
The restructuring of recurse() in the last few weeks actually broke
the recursion-flags in different tools.
As a long-term goal, the recursor should have a field "maxdepth"
which should be "1" for the non-Rflag-case. "0" stands for unlimited.
2015-04-20 11:12:40 +01:00
sin
bb2c0cff45 Fix function definition style for fshut.c 2015-04-05 09:16:50 +01:00
FRIGN
3eee8e1509 Remove DEBUG-define for eprintf.c
Prepend program name only when fmt doesn't begin with "usage".
2015-04-05 09:13:56 +01:00
FRIGN
0c470f5563 Remove fflush-check from fshut()
Basically, it's a conflict between POSIX and ISO C what do to when
input streams are passed to fflush().
POSIX mandates that the seeking-position should be synced, but ISO C
says it's undefined behaviour.
We love POSIX, but the standard-documents specify that in all conflict
cases, ISO C wins, so this breaks with EBADF on BSD's.

musl and glibc follow POSIX behaviour, which makes sense, but involves
numerous portability concerns.

To get around this, we just don't check fflush() and rely on the fact
that no implementation sets ferror on the file-stream in fflush if it
is an input stream, so every issue caught in fflush() is caught later
with ferror() and fclose().

Add a comment to fshut() because this stuff is so complicated, it
took us a day to figure out.
2015-04-05 09:13:56 +01:00
FRIGN
11e2d472bf Add *fshut() functions to properly flush file streams
This has been a known issue for a long time. Example:

printf "word" > /dev/full

wouldn't report there's not enough space on the device.
This is due to the fact that every libc has internal buffers
for stdout which store fragments of written data until they reach
a certain size or on some callback to flush them all at once to the
kernel.
You can force the libc to flush them with fflush(). In case flushing
fails, you can check the return value of fflush() and report an error.

However, previously, sbase didn't have such checks and without fflush(),
the libc silently flushes the buffers on exit without checking the errors.
No offense, but there's no way for the libc to report errors in the exit-
condition.

GNU coreutils solve this by having onexit-callbacks to handle the flushing
and report issues, but they have obvious deficiencies.
After long discussions on IRC, we came to the conclusion that checking the
return value of every io-function would be a bit too much, and having a
general-purpose fclose-wrapper would be the best way to go.

It turned out that fclose() alone is not enough to detect errors. The right
way to do it is to fflush() + check ferror on the fp and then to a fclose().
This is what fshut does and that's how it's done before each return.
The return value is obviously affected, reporting an error in case a flush
or close failed, but also when reading failed for some reason, the error-
state is caught.

the !!( ... + ...) construction is used to call all functions inside the
brackets and not "terminating" on the first.
We want errors to be reported, but there's no reason to stop flushing buffers
when one other file buffer has issues.
Obviously, functionales come before the flush and ret-logic comes after to
prevent early exits as well without reporting warnings if there are any.

One more advantage of fshut() is that it is even able to report errors
on obscure NFS-setups which the other coreutils are unable to detect,
because they only check the return-value of fflush() and fclose(),
not ferror() as well.
2015-04-05 09:13:56 +01:00
Hiltjo Posthuma
27f258dd34 libutil/getlines: style fix 2015-03-29 21:55:34 +02:00
Hiltjo Posthuma
9f97430143 libutil/getlines: fix crash with no lines
because b->lines and b->nlines would be 0 with no lines read.

reproduce: printf '' | sort or cols

bug was introduced by commit: 66a5ea722d
2015-03-29 21:48:49 +02:00
Hiltjo Posthuma
a9bedca038 fix some signed/unsigned warnings and style fixes 2015-03-27 22:48:05 +01:00
FRIGN
9144d51594 Check getline()-return-values properly
It's not useful when 0 is returned anyway, so be sure that we have a
string with length > 0, this also solves some indexing-gotchas like
"len - 1" and so on.
Also, add checked getline()'s whenever it has been forgotten and
clean up the error-messages.
2015-03-27 14:49:48 +01:00
FRIGN
b6b977f63d Audit tar(1), add DIRFIRST-flag to recurse()
I've been wanting to do this for a while now, as tar(1) used to
be one of messiest and cruftiest tools.
First off, before walking through the audit, I'll talk about
what the DIRFIRST-flag for recurse() does.
It basically calls fn() on the first-level-dir before calling
it's subentries. It's necessary here, because else the order
of the tar-files would've been wrong (it would try to create
dir/file before creating dir/).

Now, to the audit:
1)  Update manpage, fix mistake that compression is also available
    for compressing. It's only available for extracting.
2)  Define the major, minor and makedev macros from glibc by ourselves.
    No need to rely on them, as they are common sense.

decomp()
3)  Simple refactorization.

putoctal()
4)  Add a truncation check for snprintf().

archive()
5)  BUGFIX: Add checks to any checkable function, don't blindly call
    them, this is harmful and there are 100 ways to exploit that.
6)  Use estrlcpy() instead of snprintf() wherever possible, fix
    alignment.
7)  BUGFIX: Terminate the result-buffer of readlink(), check if
    it even succeeded.
8)  Fix sizeof()-formatting.

unarchive()
9)  BUGFIX: Add checks to any checkable function, don't blindly call
    them, this is harmful and there are 100 ways to exploit that.
10) BUGFIX: strtoul can happily return negative numbers. Add checks
    for that and also if the full string has been processed.
11) Remove calls to perror(). We have eprintf, use it.
12) BUGFIX: "minor = strtoul(h->mode, 0, 8);". We need h->minor of
    course.
13) Fix typo "usupported", remove fprintf-call.

print()
14) Check fread().

xt()
15) Get rid of snprintf-magic. Use estrlcat().
16) BUGFIX: check for ferror() on the tarfile.

usage()
17) Update it. The old usage() was like 1000 years old.

main()
18) Add DIRFIRST-flag to the recursor.
19) Don't print usage() when a mode is re-set. We allow this in
    general.
20) Add function checks and fix error messages.
21) Add tarfilename-global for proper error-messages.
2015-03-21 01:30:47 +01:00
FRIGN
58098575e7 Audit cp() in libutil
1) Rename cp_HLPflag -> cp_follow for consistency.
2) Use function-pointers for stat to clear up the code.
3) BUGFIX: TERMINATE THE RESULT BUFFER OF READLINK !!!
   It's something I noticed earlier and it actually lead to some
   pretty insane behaviour on our side using glibc (musl somehow
   magically solves this).
   Basically, symlinks used to contain the data of the file they
   pointed to. I wondered for weeks where this came from and now
   this has finally been solved.
4) BUGFIX: Do not unconditionally unlink target-files. Even GNU
   coreutils do it wrong.
   The basic idea is this:
   If fflag == 0 --> don't touch target files if they exist.
   If fflag == 1 --> unlink all and don't error out when we try
                     to unlink a file which doesn't exist.
5) Use estrlcpy and estrlcat instead of snprintf for path building.
6) Make it clearer what happens in preserve.
2015-03-19 17:57:12 +01:00