Major cleanups of the preprocessor. In particular, the
block-allocation of Token is pretty ridiculous since nearly every
token requires a text allocation anyway. Change the definition of
Token so that only very long tokens (48+ characters on 64-bit systems)
need to be stored out of line.
If malloc() preserves alignment (XXX: glibc doesn't) then this means
that each Token will fit in a cache line.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
8571f06061