Commit Graph

1704 Commits

Author SHA1 Message Date
Eivind Uggedal
d4f7ecd334 uniq: always store previous line length
A bug was introduced in the NUL support refactor leading to
the length of the previous line only being saved if the
previous line was shorter than the current line. If triggered
this lead to copying too much data into the previous line buffer.

Behavior before:

	printf '1234\na\n' | ./uniq
	1234
	a
	4

Behavior after:

	printf '1234\na\n' | ./uniq
	1234
	a
2016-03-11 15:38:36 +00:00
FRIGN
515525997c Fix linecmp() to return correct values 2016-03-11 15:38:36 +00:00
FRIGN
0b87cd4c61 Properly use delimlen instead of len in paste(1) 2016-03-10 13:39:49 +00:00
FRIGN
7fd88bdc6f xargs(1) already supports NUL characters 2016-03-10 08:48:09 +00:00
FRIGN
5b7df455f8 Change paste(1) a bit to prepare libutf changes 2016-03-10 08:48:09 +00:00
FRIGN
e537186ba4 Support NUL containing lines in uniq(1) 2016-03-10 08:48:09 +00:00
FRIGN
b58884312c Support NUL containing lines in nl(1) and reuse the line-buffer(!) 2016-03-10 08:48:09 +00:00
FRIGN
c25996924b Support NUL containing lines in join(1)
while also simplyfing the line-field-parser.
2016-03-10 08:48:09 +00:00
FRIGN
d7741c7725 Support NUL containing lines in fold(1) and reuse the line-buffer(!) 2016-03-10 08:48:09 +00:00
FRIGN
dc5190eab1 logger(1) is NUL compliant
because we can't call the syslog-syscall without becoming unportable.
The syslog() libc-function does not accept a length though, so we're
fucked. Anyway, let's carry on.
2016-03-10 08:48:09 +00:00
FRIGN
00b579566a Use the UTF8_POINT macro in cols(1)
No need for libutf any more
2016-03-10 08:48:09 +00:00
FRIGN
cc3aefa488 Support NUL containing lines in cut(1) 2016-03-10 08:48:09 +00:00
FRIGN
5ad71a466b Error out when giving an empty delimiter to sort(1) 2016-03-10 08:48:09 +00:00
FRIGN
3debc5e064 Add linecmp() 2016-03-10 08:48:09 +00:00
FRIGN
698a14b1da Indent README headings properly 2016-03-10 08:48:09 +00:00
FRIGN
45b42516d4 Support NUL containing lines in head(1)
Very trivial this time.
2016-03-10 08:48:09 +00:00
FRIGN
886fca3fd6 Support NUL containing lines in tail(1)
This was rather simple this time.
2016-03-10 08:48:09 +00:00
FRIGN
0fa5a3e5bb Rename struct linebufline to struct line and add linecmp()
This simplifies the handling in sort(1) and comm(1) quite a bit.
2016-03-10 08:48:09 +00:00
FRIGN
54d3f3b3a5 Rename linecmp and line-structs in join(1) and sort(1)
We will steal the names for the global functions.
2016-03-10 08:48:09 +00:00
FRIGN
d585d4b028 No need for += when res is 0 anyway 2016-03-10 08:48:09 +00:00
FRIGN
eebba22577 Support NUL containing lines in comm(1) 2016-03-10 08:48:09 +00:00
FRIGN
691c54750f Don't mark cols(1) as NUL-compliant as of yet
Thing is, we don't yet have an updated utflen() ready for use. Although
prints the lines completely, it does not add a proper spacing between
each entry. This will be fixed later.
2016-03-10 08:48:09 +00:00
FRIGN
9d120b7b32 Actually move past the field separator
Previously, sort(1) failed on key-based sorting and was caught in an
infinite loop with the c-flag.
2016-03-10 08:48:09 +00:00
FRIGN
0e25f09b56 Remove debug info 2016-03-10 08:48:09 +00:00
FRIGN
19c0ca9830 Properly increment line lenght on edge-case in getlines() 2016-03-10 08:48:09 +00:00
FRIGN
eb9bda8787 Support NUL-containing lines in sort(1)
For sort(1) we need memmem(), which I imported from OpenBSD.
Inside sort(1), the changes involved working with the explicit lengths
given by getlines() earlier and rewriting some of the functions.

Now we can handle NUL-characters in the input just fine.
2016-03-10 08:48:09 +00:00
FRIGN
e4810f1cdb Support NUL-containing lines in cols(1)
This required an architectural change in getlines() by also storing
the line length.
2016-03-10 08:48:09 +00:00
FRIGN
2d7d2261a0 Really force 1 entry per line with -1 in ls(1)
if a filename contains a \n, it would not be limited to one line.
We use the qflag to replace the control characters with '?'s.
2016-03-02 18:44:32 +00:00
FRIGN
97ce9ea586 Fix -s in tr(1)
Forgot that in case there is a second argument given with -s you
probably want to have your characters substituted.
I changed it so that shortly before "deploying" we check if the
"to be written"-Rune is equal to the last Rune, and proceed as
needed.
2016-03-02 09:31:11 +00:00
sin
ea0d16e928 Revert "fix length after '\' getline string termination"
Caused a regression in sed, revert until we investigate further.
2016-03-01 15:24:32 +00:00
Hiltjo Posthuma
fa18379a05 uuencode: style: just use pointer *b
it doesn't make sense to use b[2] here anyway, and it is incorrect (should be b[3]).
2016-03-01 15:24:32 +00:00
Hiltjo Posthuma
a51b01ff90 uudecode: dont return pointer to local variable 2016-03-01 15:24:32 +00:00
Quentin Rameau
d04402b6ea cat: fix u flag
Our libutil concat() always uses a buffer for input wich is obviously
not compatible with unbuffered io. Here is a local uconcat() which
naively copies input to stdout char by char.
2016-03-01 11:14:42 +00:00
Quentin Rameau
3da450e203 printf: replace strtonum with strtol functions in conversions
Use strtol and strtoul respectively for d, i and o, u, x, X conversions.
This way we can convert other bases than 10, which strtonum doesn't
provide.
Also don't exit on conversion error but display a warning, set a return
error code, and continue.
2016-03-01 11:14:42 +00:00
Quentin Rameau
243cdb6669 printf: remove unnecessary string size test 2016-03-01 11:14:42 +00:00
FRIGN
b83916ca27 Add symbol to mark NUL handling in each tool (and where it is missing)
The assumption of NUL-terminated strings is actually quite a good one in
most cases. You don't have to worry about paths, because they may not
contain NUL.
Same applies to arguments passed to you. Unless you have to unescape,
there is no way for you to receive a NUL.

There are two important exceptions though, and it's important that we
address them, or else we get unexpected behaviour:

	1) All tools using unescape() have to be strict about delimlen.
	   Else they end up for instance unescaping
	   	'\\0abc'
	   to
	   	'\0abc',
	   which in C's string-vision is an empty string.

	2) All tools doing line wrenching and putting them out
	   again as lines again.
	   puts() will cut each line containing NULs off at the first
	   occurence.
2016-02-26 09:54:46 +00:00
FRIGN
a88906b423 Rever the strmem() addition and add a TODO element
strmem() was not very well thought out. The thing is the following:
If the string contains a zero character, we want to match it, and not
stop right there in place.

The "real" solution is to use memmem() where needed and replace all
functions that assume zero-terminated-strings from standard input, which
could lead to early string-breakoffs.
This requires a strict tracking of string lengths.
2016-02-26 09:54:46 +00:00
FRIGN
3396088666 Implement strmem() and use it in join(1)
We want our delimiters to also contain 0 characters and have them
handled gracefully.
To accomplish this, I wrote a function strmem(), which looks for a
certain, arbitrarily long memory subset in a given string.
memmem() is a GNU extension and forces you to call strlen every time.
2016-02-26 09:54:46 +00:00
Hiltjo Posthuma
e8eeb19fcd fix length after '\' getline string termination 2016-02-26 09:54:46 +00:00
Hiltjo Posthuma
58257275f0 tsort.1: use mandoc literal block for example 2016-02-26 09:54:46 +00:00
sin
9a2b8d3531 tsort style fixes
getline() cannot return 0 so no need to guard against that.
2016-02-24 15:57:33 +00:00
FRIGN
102baab4c4 Use en* instead of e* function family in tsort(1)
The wtf macros overwrote the semantics, for more clarity this
has been fixed.
2016-02-24 15:28:01 +00:00
FRIGN
50b30bbd69 Remove wtf 2016-02-24 15:21:05 +00:00
FRIGN
79e8e330cb Fix wc(1) output for large files
Previously, we used the System V output format:
	"%7d%7d%7d %s\n"
The problem here is, that if any number has more than six digits, the
result looks like one big number, as we don't mandate spaces.

POSIX says the output format should rather be
	"%d %d %d %s\n"
but in this case we wouldn't get consistent results.

To serve both camps, I changed it to the following:
	"%6d %6d %6d %s\n"
This won't change the output for normal values, but also
prevent the output of large files to be ambiguous.
2016-02-24 14:45:20 +00:00
Mattias Andrée
a392cd475e add sha512-224sum (SHA512/224) and sha512-256sum (SHA512/256)
Signed-off-by: Mattias Andrée <maandree@kth.se>
2016-02-24 10:40:57 +00:00
Mattias Andrée
ae1da536bb add sha224sum and sha384sum
Signed-off-by: Mattias Andrée <maandree@kth.se>
2016-02-24 10:15:16 +00:00
FRIGN
6adb9b8ccd Fix compilation error 2016-02-21 08:52:48 +00:00
FRIGN
0d97bd3f57 Change the note in printf.1 on octal escapes 2016-02-21 08:52:48 +00:00
FRIGN
70adb1252d Do a range check on the resulting octal 2016-02-21 08:52:48 +00:00
FRIGN
bbe10559e6 Remove the octal note in printf.1 2016-02-21 08:52:48 +00:00