92f9e8a9be
I botched readrec's definition of a record, when I implemented
RS regular expression support. This is the relevant hunk from the
old diff:
```
- return c == EOF && rr == buf ? 0 : 1;
+ isrec = *buf || !feof(inf);
+ dprintf( ("readrec saw <%s>, returns %d\n", buf, isrec) );
+ return isrec;
```
Problem #1
Unlike testing with EOF, `*buf || !feof(inf)` is blind to stdio
errors. This can cause an infinite loop whose each iteration fabricates
an empty record.
The following demonstration uses standard terminal access control
policy to produce a persistent error condition. Note that the "i/o
error" message does not come from readrec(). It's produced much later
by closeall() at shutdown.
```
$ trap '' SIGTTIN && awk 'END {print NR}' &
[1] 33517
$ # After fg, type ^D
$ fg
trap '' SIGTTIN && awk 'END {print NR}'
13847376
awk: i/o error occurred on /dev/stdin
input record number 13847376, file
source line number 1
```
Each time awk tries to read the terminal from the background,
while ignoring SIGTTIN, the read fails with EIO, getc returns EOF,
the stream's end-of-file indicator remains clear, and `!feof`
erroneously promotoes the empty buffer to an empty record. So long
as the error persists, the stream's position does not advance and
end-of-file is never set.
Problem #2:
When RS is a regex, `*buf || !feof(inf)` can't see an empty record's
terminator at the end of a stream.
```
$ echo a | awk 1 RS='a\n'
$
```
That pipeline should have found one empty record and printed a blank
line, but `*buf || !feof(inf)` considers reaching the end of the
stream the conclusion of a fruitless search. That's only correct when
the terminator is a single character, because a regex RS search can
set the end-of-file marker even when it succeeds.
The Fix
`isrec` must be 0 **iff** no record is found. The correct definition
of "no record" is a failure to find a record terminator and a
failure to find any data (possibly from a final, unterminated
record). Conceptually, for any RS:
```
isrec = (noTERM && noDATA) ? 0 : 1
```
noDATA is an expression that's true if `buf` is empty, false otherwise.
When RS is null or a single character, noTERM is an expression
that is true when the sought after character is not found, false
otherwise. Since the search for a single character can only end with
that character or EOF, noTERM is `c == EOF`.
```
isrec = (c == EOF && rr == buf) ? 0 : 1
```
When RS is a regular expression: noTERM is an expression that is
true if a match for RS is not found, false otherwise. This is simply
the inverse of the result of the function that conducts the search,
`!found`.
```
isrec = (found == 0 && *buf == '\0') ? 0 : 1
```
|
||
|---|---|---|
| bugs-fixed | ||
| testdir | ||
| .gitignore | ||
| awk.1 | ||
| awk.h | ||
| awkgram.y | ||
| b.c | ||
| ChangeLog | ||
| FIXES | ||
| lex.c | ||
| lib.c | ||
| LICENSE | ||
| main.c | ||
| makefile | ||
| maketab.c | ||
| parse.c | ||
| proto.h | ||
| README.md | ||
| REGRESS | ||
| run.c | ||
| TODO | ||
| tran.c | ||
The One True Awk
This is the version of awk described in The AWK Programming Language,
by Al Aho, Brian Kernighan, and Peter Weinberger
(Addison-Wesley, 1988, ISBN 0-201-07981-X).
Copyright
Copyright (C) Lucent Technologies 1997
All Rights Reserved
Permission to use, copy, modify, and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that the copyright notice and this permission notice and warranty disclaimer appear in supporting documentation, and that the name Lucent Technologies or any of its entities not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission.
LUCENT DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL LUCENT OR ANY OF ITS ENTITIES BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
Distribution and Reporting Problems
Changes, mostly bug fixes and occasional enhancements, are listed
in FIXES. If you distribute this code further, please please please
distribute FIXES with it.
If you find errors, please report them to bwk@cs.princeton.edu. Please also open an issue in the GitHub issue tracker, to make it easy to track issues. Thanks.
Submitting Pull Requests
Pull requests are welcome. Some guidelines:
-
Please do not use functions or facilities that are not standard (e.g.,
strlcpy(),fpurge()). -
Please run the test suite and make sure that your changes pass before posting the pull request. To do so:
- Save the previous version of
awksomewhere in your path. Call itnawk(for example). - Run
oldawk=nawk make check > check.out 2>&1. - Search for
BADorerrorin the result. In general, look over it manually to make sure there are no errors.
- Save the previous version of
-
Please create the pull request with a request to merge into the
stagingbranch instead of into themasterbranch. This allows us to do testing, and to make any additional edits or changes after the merge but before merging tomaster.
Building
The program itself is created by
make
which should produce a sequence of messages roughly like this:
yacc -d awkgram.y
conflicts: 43 shift/reduce, 85 reduce/reduce
mv y.tab.c ytab.c
mv y.tab.h ytab.h
cc -c ytab.c
cc -c b.c
cc -c main.c
cc -c parse.c
cc maketab.c -o maketab
./maketab >proctab.c
cc -c proctab.c
cc -c tran.c
cc -c lib.c
cc -c run.c
cc -c lex.c
cc ytab.o b.o main.o parse.o proctab.o tran.o lib.o run.o lex.o -lm
This produces an executable a.out; you will eventually want to
move this to some place like /usr/bin/awk.
If your system does not have yacc or bison (the GNU
equivalent), you need to install one of them first.
NOTE: This version uses ANSI C (C 99), as you should also. We have
compiled this without any changes using gcc -Wall and/or local C
compilers on a variety of systems, but new systems or compilers
may raise some new complaint; reports of difficulties are
welcome.
This compiles without change on Macintosh OS X using gcc and
the standard developer tools.
You can also use make CC=g++ to build with the GNU C++ compiler,
should you choose to do so.
The version of malloc that comes with some systems is sometimes
astonishly slow. If awk seems slow, you might try fixing that.
More generally, turning on optimization can significantly improve
awk's speed, perhaps by 1/3 for highest levels.
A Note About Maintenance
NOTICE! Maintenance of this program is on a ``best effort'' basis. We try to get to issues and pull requests as quickly as we can. Unfortunately, however, keeping this program going is not at the top of our priority list.
Last Updated
Fri Dec 25 16:53:34 EST 2020