For loop detection, a history is mandatory. In the process of also
adding a flexible struct to recurse, the recurse-definition was moved
to fs.h.
The motivation behind the struct is to allow easy extensions to the
recurse-function without having to change the prototypes of all
functions in the process.
Adding flags is really simple as well now.
Using the recursor-struct, it's also easier to see which defaults
apply to a program (for instance, which type of follow, ...).
Another change was to add proper stat-lstat-usage in recurse. It
was wrong before.
While auditing du(1) I realized that there's no way the over 100 lines
of procedures in du() would pass the audit.
Instead, I decided to rewrite this section using recurse() from libutil.
However, the issue was that you'd need some kind of payload to count
the number of bytes in the subdirectories and use them in the higher
hierarchies.
The solution is to add a "void *data" data pointer to each recurse-
function-prototype, which we might also be able to use in other
recurse-applications.
recurse() itself had to be augmented with a recurse_samedev-flag, which
basically prevents recurse from leaving the current device.
Now, let's take a closer look at the audit:
1) Removing the now unnecessary util-functions push, pop, xrealpath,
rename print() to printpath(), localize some global variables.
2) Only pass the block count to nblks instead of the entire stat-
pointer.
3) Fix estrtonum to use the minimum of LLONG_MAX and SIZE_MAX.
4) Use idiomatic argv+argc-loop
5) Report proper exit-status.
After a short correspondence with Otto Moerbeek it turned out
mallocarray() is only in the OpenBSD-Kernel, because the kernel-
malloc doesn't have realloc.
Userspace applications should rather use reallocarray with an
explicit NULL-pointer.
Assuming reallocarray() will become available in c-stdlibs in the
next few years, we nip mallocarray() in the bud to allow an easy
transition to a system-provided version when the day comes.
A function used only in the OpenBSD-Kernel as of now, but it surely
provides a helpful interface when you just don't want to make sure
the incoming pointer to erealloc() is really NULL so it behaves
like malloc, making it a bit more safer.
Talking about *allocarray(): It's definitely a major step in code-
hardening. Especially as a system administrator, you should be
able to trust your core tools without having to worry about segfaults
like this, which can easily lead to privilege escalation.
How do the GNU coreutils handle this?
$ strings -n 4611686018427387903
strings: invalid minimum string length -1
$ strings -n 4611686018427387904
strings: invalid minimum string length 0
They silently overflow...
In comparison, sbase:
$ strings -n 4611686018427387903
mallocarray: out of memory
$ strings -n 4611686018427387904
mallocarray: out of memory
The first out of memory is actually a true OOM returned by malloc,
whereas the second one is a detected overflow, which is not marked
in a special way.
Now tell me which diagnostic error-messages are easier to understand.
Stateless and I stumbled upon this issue while discussing the
semantics of read, accepting a size_t but only being able to return
ssize_t, effectively lacking the ability to report successful
reads > SSIZE_MAX.
The discussion went along and we came to the topic of input-based
memory allocations. Basically, it was possible for the argument
to a memory-allocation-function to overflow, leading to a segfault
later.
The OpenBSD-guys came up with the ingenious reallocarray-function,
and I implemented it as ereallocarray, which automatically returns
on error.
Read more about it here[0].
A simple testcase is this (courtesy to stateless):
$ sbase-strings -n (2^(32|64) / 4)
This will segfault before this patch and properly return an OOM-
situation afterwards (thanks to the overflow-check in reallocarray).
[0]: http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man3/calloc.3
The HLP-changes to sbase have been a great addition of functionality,
but they kind of "polluted" the enmasse() and recurse() prototypes.
As this will come in handy in the future, knowing at which "depth"
you are inside a recursing function is an important functionality.
Instead of having a special HLP-flag passed to enmasse, each sub-
function needs to provide it on its own and can calculate results
based on the current depth (for instance, 'H' implies 'P' at
depth > 0).
A special case is recurse(), because it actually depends on the
follow-type. A new flag "recurse_follow" brings consistency into
what used to be spread across different naming conventions (fflag,
HLP_flag, ...).
This also fixes numerous bugs with the behaviour of HLP in the
tools using it.
This has already been suggested by Evan Gates <evan.gates@gmail.com>
and he's totally right about it.
So, what's the problem?
I wrote a testing program asshole.c with
int
main(void)
{
execl("/path/to/sbase/echo", "echo", "test");
return 0;
}
and checked the results with glibc and musl. Note that the
sentinel NULL is missing from the end of the argument list.
glibc calculates an argc of 5, musl 4 (instead of 2) and thus
mess up things anyway.
The powerful arg.h also focuses on argv instead of argc as well,
but ignoring argc completely is also the wrong way to go.
Instead, a more idiomatic approach is to check *argv only and
decrement argc on the go.
While at it, I rewrote yes(1) in an argv-centric way as well.
All audited tools have been "fixed" and each following audited
tool will receive the same treatment.
Previously, it was not possible to use
sha1sum test.c | sha1sum -c
because the program would not differenciate between an empty
argument and a non-specified argument.
Moreover, why not allow this?
sha1sum -c hashlist1 hashlist2
Digging deeper I found that using function pointers and a
modification in the crypt-backend might simplify the program
a lot by passing the argument-list to both cryptmain and
cryptcheck.
Allowing more than one list-file to be specified is also
consistent with what the other implementations support,
so we not only have simpler code, we also do not silently
break if there's a script around passing multiple files to
check.
This is a useful behavior if you want to reorder the lines,
because otherwise you might end up with originally two lines
on one, e.g.
$ echo -ne "foo\nbar" | sort
barfoo
We should never mix FILE I/O with raw I/O. Going from raw I/O
to FILE I/O is fine but doing the opposite is extremely tricky and
only works under certain conditions (unbuffered stream + no call
to ungetc()).
The -d option is a GNU extension and is equivalent to its "-P
--preserve=links" options.
Since we don't implement the --preserve=links functionality anyway (it
means preserve hard links between files), just call it -P, which is
specified by POSIX.
Additionally, there is no need to check for cp_Pflag again before
copying the symlink itself because the only way the mode in the stat
will indicate a symlink is if we used lstat (which we only do if -P is
specified).
