Instead of an array of strings, just have a character array; that
reduces the size of canned macros by up to 30%, and we only did
sequential access anyway.
ctype functions take an *int*, which the user is expected to have
taken the input character from getc() and friends, or taken a
character and cast it to (unsigned char).
We don't care about EOF (-1), so use macros that cast to (unsigned
char) for us.
Move the handling of "extra" macros (i.e. output format macros) into
the macros.pl mechanism. This allows us to change the format of the
internal macro store in the future - e.g. to a single byte store
without redundant pointers.
Also, stop using indicies into a long array when there is no good
reason to not just use different arrays.
Automatically generate a %define as the first string in the include
block, and just pick the string out of it from that %define statement
to verify existence. That way we eliminate any use of toupper() --
all case-insensitivity in NASM uses tolower()/nasm_tolower().
Automatically provide an include guard for %use packages; the macro
__USE_package__ is automatically defined, and inclusion is suppressed
if it is already defined.
Compress macros.c by representing macro directives with a single byte.
We can do this because we only use the ASCII character range inside
the standard macro files.
Note: we could save significant additional space by not having a
pointer array, and instead relying on the fact that we sweep
sequentially through the output array.
Add a builtin equivalent to the %include directive called %use.
%use includes a standard macro file compiled into the binary; these
come from the macros/ directory in the source code.
The idea here is to be able to provide optional macro packages with
the distribution, without adding complex host filesystem dependencies.
On some platforms, tolower() is implemented as a function call, in
order to handle locale support. We never change locales, so can the
result of tolower() into a table, so we don't have to sit through the
function call every time.
~1.3% overall performance improvement on a macro-heavy benchmark under
Linux x86-64.
- Add %warning directive
- Only unquote an %error or %warning string if it is the only thing on
the directive line.
- Don't expand macros inside a quoted string, even for %error.
Make strings a proper, first-class token type, instead of relying on
the "TOKEN_NUM with tv_charptr" hack. Only convert a string to a
number if requested in an expression context; this also makes it
possible to actually issue a warning when it overflows.
The handling of %? in multi-line macros was broken when the macro name
was preceeded by a label; it would expand to the label instead of the
macro name. This was particularly serious since this was used in
the macro implementation of INCBIN.
The dependency list tail pointer wasn't actually updated correctly.
Fix that. We may want to make this a structure of some sort to make
the code a bit cleaner, but this seems to be the cleanest hack for
now.
Ownership of the filename string was a bit fuzzy, with the result that
we were freeing it even though it was retained for use by __FILE__.
Clean up a number of other memory management issues with the new
quoting code, and change the stdscan implementation to one pass over
the string.
Introduce new preprocessor directives %depend and %pathsearch, and
make incbin a standard macro using these filenames. This lets us
remove the code that makes incbin search the path.
Call expand_smacros() early instead of expand_smacros_in_string()
late. expand_smacros_in_string() seems like a prodigiously bad idea
and a sheer brainfart in my opinion.
Add the -MP option to emit phony targets. Since this means each
header file has to be visited more than once, change the
implementation to use an internal list of all the dependencies, and
centralize the emission of the dependency files.
Implement the dependency options:
-MF: set the file to which dependencies are written.
-MD: generate dependencies in parallel with compilation.
-MT: set the name of the dependency target.
-MQ: same as -MT, but *attempt* to quote it for Makefile safety.
struct hash_table, a fixed-sized structure, is now allocated by the
caller. This lets us integrate it into the Context structure, thus
avoiding an additional dynamically allocated object for no good
reason.
Add some minor code collapsing: make it more obvious that all that
differs is a pointer value, rather than relying on the compiler to do
tail merging.
The compiler is free to store enum pp_token_type into any size integer
small enough to contain all the values up to 2^n-1 for the smallest n
which contains all the values. Force it to size it to integer size,
since we use it to hold macro positional parameters.
%? - substitute the macro name as invoked
%?? - substitute the macro name as defined
In particular:
%idefine keyword $%?
... can be used to make a new keyword "disappear".
Normally, contexts aren't used with a large number of macros, but in
case someone does, do use hash tables for those as well. This
simplifies the code somewhat, since *all* handling of macros is now
done via hash tables.
Future note: consider if it wouldn't be better to allow struct
hash_table to be allocated by the caller, instead of being allocated
by the hash table routine.
Fix buffer overflow in preproc.c due to an incorrect test. In the
code:
for (r = p, s = ourcopy; *r; r++) {
if (r >= p+MAX_KEYWORD)
return tokval->t_type = TOKEN_ID; /* Not a keyword */
*s++ = tolower(*r);
}
*s = '\0';
... the test really needs to be >= since for the pass where there are
equal:
a) a nonzero byte means we have > MAX_KEYWORD characters, and
b) s = ourcopy+MAX_KEYWORD; but if the test doesn't trigger,
we can write one more character *plus* the null byte, overflowing
ourcopy.
First cut at AVX machinery support. The only instruction implemented
is VPERMIL2PS, and it's probably buggy. I'm checking this in with the
hope that other people can start helping out with (a) testing this,
and (b) adding instructions.
NDISASM support is not there yet.
Correct the implementation of %arg and %local.
It's questionable how much they make sense for 64-bit mode; even in
32-bit mode one normally make references off the stack pointer instead
of the base pointer (frame pointer), but that requires keeping track
of the stack pointer offset.
Correct the handling of floating-point tokens in the preprocessor.
The preprocessor scanner and the main scanner really are painfully
divergent for no good reason.
Proper use of bool and enum makes code easier to debug. Do more of
it. In particular, we really should stomp out any residual uses of
magic constants that aren't enums or, in some cases, even #defines.
Per the comment:
* In a MMacro describing a `%rep' block, the `in_progress' field
* isn't merely boolean, but gives the number of repeats left to
* run.
This fixes the "global" directive not getting recognized, since it
repeats over all its arguments.
Both C and C++ have "bool", "true" and "false" in lower case; C
requires <stdbool.h> for this, in C++ it is an inherent type built
into the compiler. Use those instead of the old macros; emulate with
a simple typedef enum if unavailable.
Concentrate compiler dependencies to compiler.h; make sure compiler.h
is included first in every .c file (since some prototypes may depend
on the presence of feature request macros.)
Actually use the conditional inclusion of various functions (totally
broken in previous releases.)
The parent-pointer-based freeing loop in %undef should not advance the
parent pointer when a node is freed, since that will result accessing
freed memory.
Implement the -MG option, to generate dependencies in the presence of
generated files. In the end, we probably need to support the full
gamut of GCC-like dependency-generation options.
Document that %+ needs a space after it due to collision with %+1
syntax for multiline macro arguments; make it issue an error message
rather than crashing.
Switch the preprocessor over to using the hash table library. On my
system, this improves the runtime of the output of test/pref/macro.pl
from over 600 seconds to 7 seconds.
Macros have an odd mix of case-sensitive and case-insensitive
behaviour, plus there are matching parameters for arguments, etc. As
a result, we use case-insensitive hash tables and use a linked list to
store all the possible isomorphs.
