printstat and awkprintf are very clear: print statement errors
are fatal.
In Jan 2020 [1], to prevent fatal print errors from masquerading
as fclose warnings, every WARNING in closefile and closeall became
FATAL. This broke awk's close and getline functions.
close no longer returns if there's an error, unless the stream
doesn't exist.
getline read errors still return -1, but they are no longer
ignorable. Eventually, one of the closing functions will inspect the
stream with ferror and call FATAL.
In Jul 2020 [2], fatal stdout write errors which had been detectable by
closefile for a few months became invisible, a consequence of switching
standard streams from fclose (which reports flush errors) to freopen
(which ignores them). The Jan 2020 changes which broke getline and
close were themselves partially broken.
The solution is to finish printing before closing. That is to flush
and ferror every stream opened for writing before calling fclose,
pclose, or freopen. A failure to write print statement data is
fatal. A failure to close a flushed stream is a warning. They must
be handled separately.
Every redirected print statement is finished in printstat or awkprintf.
The same is not true of unredirected print statements. To finish
these, stdout must be flushed at some point after the final such
statement. Any problem with that flush is fatal.
Though only stdout needs it, let's defensively finish every stream
opened for writing, so this bug won't recur if someone changes how
redirected streams are flushed.
Write errors on stderr by the implementation are never fatal. When
closing, we only warn of them. Write errors from an application
attempting a redirected print to /dev/stderr are as immediately fatal
as every other redirected print statement.
[1] fed1a562c3
[2] b82b649aa6
* In closeall(), skip stdin and flush std{err,out} instead of closing.
Otherwise awk could fclose(stdin) twice (it may appear more than once)
and closing stderr means awk cannot report errors with other streams.
For example, "awk 'BEGIN { getline < "-" }' < /dev/null" will call
fclose(stdin) twice, with undefined results.
* If closefile() is called on std{in,out,err}, freopen() /dev/null instead.
Otherwise, awk will continue trying to perform I/O on a closed stdio
stream, the behavior of which is undefined.
Commit 0d8778bbbb reintroduced a
regression that was fixed in commit
97a4b7ed21. The length of SUBSEP needs to
be rechecked after calling execute(), in case SUBSEP itself has been
changed.
Co-authored-by: Tim van der Molen <tim@kariliq.nl>
The optimization in commit 1d6ddfd9c0
reintroduced the regression that was fixed in commit
e26237434f.
Co-authored-by: Tim van der Molen <tim@kariliq.nl>
POSIX specifies a dprintf function that operates on an fd instead of
a stdio stream. Using upper case for macros is more idiomatic too.
We no longer need to use an extra set of parentheses for debugging
printf statements.
The errcheck() function treats an errno of ERANGE or EDOM as something
to report, so make sure errno is set to zero before invoking a
function to check so that a previous such errno value won't result
in a false positive. This could happen simply due to input line fields
that looked enough like floating-point input to trigger ERANGE.
Reported by Jordan Geoghegan, fix from Philip Guenther.
* LC_NUMERIC radix issue.
According to https://pubs.opengroup.org/onlinepubs/7990989775/xcu/awk.html
The period character is the character recognized in processing awk
programs. Make it so that during output we also print the period
character, since this is what other awk implementations do, and it
makes sense from an interoperability point of view.
* print "T.builtin" in the error message
* Fix backslash continuation line handling.
* Keep track of RS processing so we apply the regex properly only once
per record.
* Add a test for german case folding.
* Add a function to copy a string with a string with a larger allocation
(to be used by the case folding routines)
* Add printf attributes to the printf-like functions and fix one format
warning
* Cleanup the tempfree macro
* make more functions static
* rename fp to frp (FRame Pointer) to avoid shadowing with fp (File Pointer).
* add more const
* fix indent in UPLUS case
* add locale-aware case folding
* make nfiles size_t
* fix bugs in file closing:
- compare fclose to EOF and pclose to -1
- use nfiles instead of FOPEN_MAX in closeall
- don't close files we did not open (0,1,2) fpurge/fflush instead
* - use NUL instead of 0 for char comparisons
- add ISWS() macro
- use continue; instead of ;
* Check for existance of the german locale before using it.
* Add missing parentheses, thanks Arnold.
* sprinkle const, static
* account for lineno in unput
* Add an EMPTY string that is used when a non-const empty string is needed.
* make inputFS static and dynamically allocated
* Simplify and in the process avoid -Wwritable-strings
* make fs const to avoid -Wwritable-strings
An input/output error indicates a fatal condition, even if it
occurs when closing a file. Awk should not return success on I/O
error, but treat I/O errors as it already treats write errors.
Test case:
$ (trap '' PIPE; awk 'BEGIN { print "hi"; }'; echo "E $?" >&2) | :
awk: i/o error occurred while closing /dev/stdout
source line number 1
E 2
The test case pipes a line into a dummy command that reads no
input, with SIGPIPE ignored so we rely on awk's own I/O checking.
No write error is detected, because the pipe is buffered; the
broken pipe is only detected as an I/O error on closing stdout.
Before this commit, "E 0" was printed (indicating status 0/success)
because an I/O error merely produced a warning. A shell script
was unable to detect the I/O error using the exit status.
Further simplify printf % parsing by eating the length specifiers
during the copy phase, and substitute 'j' when finalizing the format.
Add some more tests for this.
* More cleanups:
- sprinkle const
- add a macro (setptr) that cheats const to temporarily NUL terminate strings
remove casts from allocations
- use strdup instead of strlen+strcpy
- use x = malloc(sizeof(*x)) instead of x = malloc(sizeof(type of *x)))
- add -Wcast-qual (and casts through unitptr_t in the two macros we
cheat (xfree, setptr)).
* More cleanups:
- add const
- use bounded sscanf
- use snprintf instead of sprintf
* More cleanup:
- use snprintf/strlcat instead of sprintf/strcat
- use %j instead of %l since we are casting to intmax_t/uintmax_t
* Merge the 3 copies of the code that evaluated array strings with separators
and convert them to keep track of lengths and use memcpy instead of strcat.
* Fix formats for 32 bit machines broken by previous commit.
We use intmax_t to provide maximum range for both 32 and 64 bit machines.
* More cleanups:
- sprinkle const
- add a macro (setptr) that cheats const to temporarily NUL terminate strings
remove casts from allocations
- use strdup instead of strlen+strcpy
- use x = malloc(sizeof(*x)) instead of x = malloc(sizeof(type of *x)))
- add -Wcast-qual (and casts through unitptr_t in the two macros we
cheat (xfree, setptr)).
* More cleanups:
- add const
- use bounded sscanf
- use snprintf instead of sprintf
* More cleanup:
- use snprintf/strlcat instead of sprintf/strcat
- use %j instead of %l since we are casting to intmax_t/uintmax_t
* Merge the 3 copies of the code that evaluated array strings with separators
and convert them to keep track of lengths and use memcpy instead of strcat.
- sprinkle const
- add a macro (setptr) that cheats const to temporarily NUL terminate strings
remove casts from allocations
- use strdup instead of strlen+strcpy
- use x = malloc(sizeof(*x)) instead of x = malloc(sizeof(type of *x)))
- add -Wcast-qual (and casts through unitptr_t in the two macros we
cheat (xfree, setptr)).
While none of them is considered even near to cryptographic
level, random(3) is a better random generator than rand(3).
Use random(3) for awk as is done in other systems.
Thanks to Chenguang Li for discussing this in the lists
and submitting the patch upstream.
PR: 193147
MFC after: 5 weeks
git-svn-id: svn+ssh://svn.freebsd.org/base/head@271879 ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
Also remove a redundant semicolon.
Also had to rebase on upstream pull.
git-svn-id: svn+ssh://svn.freebsd.org/base/head@301289 ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
Miguel Pineiro Jr
Todd C. Miller
Arnold Robbins
Chris
Tim van der Molen
zoulasc
Martijn Dekker
M. Warner Losh
pfg
Cody Mello
onetrueawk