When the user generates an error via %warning, %error, or %fatal,
treat is as any other error message. The attempt at making them stand
out really looked ugly when the preprocessor adds additional tracing
information.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
We always need to process %+ at least once, but we also always need to
reprocess smacros after pasting. The solution to this is to make sure
we always reprocess %+ after the first expansion pass.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Revert to the earlier behavior of not expanding %+ until the final
phase of smacro expansion. However, the previous code has:
if (expanded && paste_tokens(&thead, true)) {
... which would inhibit paste_tokens() if expanded was false on the
first iteration. However, if expand_mmac_params is not expanding %+,
then we cannot bypass this expansion. Thus use:
pasted = paste_tokens(&thead, true);
if (expanded && pasted) {
... instead.
This seems to work with both Syslinux and x264 usage, and therefore
hopefully should be compatible with earlier versions of NASM.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
*To the best of my knowledge*, we now have authorization from everyone
who has significantly contributed to NASM in the past. As such,
change the license to the 2-clause BSD license.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Add copyright headers to the *.c/*.h files in the main directory. For
files where I'm sure enough that we have all the approvals, I have
given them the 2-BSD license, the others have been given the "LGPL for
now" license header. Most of them can probably be changed after
auditing.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
When processing an unparsable TASM argument, convert it to %if 0 which
is guaranteed to not happen, rather than %ifdef BOGUS.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Preexisting code seems to rely on %+ being processed even during early
token replacement, e.g. Syslinux contains the following code:
%macro superb 1
bx %+ %1 equ SuperInfo+($-superblock)*8+4
bs %+ %1 equ $
zb 1
%endmacro
... which is expected to work when invoked as:
superb Media
As a result, set handle_paste_tokens to true at all times; assuming
this turns out to be the way things are we can really just remove it
as an option.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Unify the token-pasting code between the macro expansion and the
preprocessor parameter case. Parameterize whether or not to handle %+
tokens during expansion (%+ tokens have late binding semantics.)
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Recognize $ and $$ as TOKEN_OTHER; they aren't really either
TOK_NUMBER nor TOK_ID, even though we have traditionally considered
them TOK_NUMBER.
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
"+" can be a separate token that ends up having to get pulled into the
middle of a floating-point constant. It's not even that strange.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Especially when token pasting involves floating-point numbers, we can
have some really strange effects from token pasting: for example,
pasting the two tokens "xyzzy" and "1e+10" ends up with *three*
tokens: "xyzzy1e" "+" "10". The easiest way to deal with this is to
explicitly combine the string and then run tokenize() on it.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
The documentation says that constructs with %$...$foo can be used
to access macros from deeper in the context stack. From what
I can tell, that has never actually worked, since we'd enter names
like %$foo into the context-local macro name table. Instead, only
insert the tail of the macro name into the context-local table;
expand get_ctx to also return a pointer to the macro name proper;
this is rather straightforward since we'd usually save away that
name at the point get_ctx is called anyway.
These two really need to be done together, in order for constructs
such as %[%1] to work properly. Furthermore, fix a token-pasting bug
in expand_mmac_params().
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Use expand_id() for the argument to %use, instead of expand_smacro().
This really makes more sense for a "naked" argument. This is a
semantic change, but is unlikely to break any real code.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Allow the %pop directive to take an identifier (an assert on the
context name); unify the parsing parts of %push, %repl, and %pop.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Fix the case where the terminal token pastes with the first token of
the unmodified sequence. This is a really ugly version; we need to
merge the two instances plus the one in expand_mmac_params().
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
When locating the end of a %[...] construct, we need to end up with
the pointer pointing to the terminating character.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Linked lists where an element may be deleted or substituted during
processing can be subtle to deal with. Fix the iteration conditions
in this particular case.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Add a new builtin macro, __PASS__, which is either 1 (for a
preparatory pass), 2 (for a final pass, including preprocessor only),
or 0 (for dependency generation.) This might be useful in special
contexts.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Since the error directives, including %warning, are now issued in the
final pass only, it is important that we do *not* pass ERR_PASS1 with
%warning. Rather than playing even more ugly games in error(),
require ERR_PASS1 to be passed in with warnings elsewhere in the
preprocessor, just like the rest of the system.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
When allocating the buffer for an mmacro list, we apparently failed to
guarantee space for the terminating NULL. This almost certainly
caused the crash described in BR 2048950, and quite possibly BR
1284169.
Checkin a26433db68 incorrectly changed a
few break;s in do_directive() that were *inside loops* to returns.
This broke single-line macros as well as %exitrep; fix.
Don't use a redundant "const" for macros_t (which is const unsigned
char), since OpenWatcom doesn't like it, and I believe it is incorrect
per the C standard.
Add %un[i]macro, and a few stylistic cleanups.
Note: unlike %undef, %un[i]macro takes an argument specification,
which must *exactly* match the macro being undefined. Similarly,
%unimacro has to be used to undefine a macro defined with %imacro, and
vice versa.
Apply updated version of fix submitted with feature request 803785.
This fix causes %exitrep to terminate only the innermost %rep block,
and also allows the count for nested blocks to be calculated in the
containing block.
I noticed there was no sane way to concatenate the contents of quoted
strings, so add the %strcat directive.
These really need to become preprocessor functions at some stage.
Instead of an array of strings, just have a character array; that
reduces the size of canned macros by up to 30%, and we only did
sequential access anyway.
ctype functions take an *int*, which the user is expected to have
taken the input character from getc() and friends, or taken a
character and cast it to (unsigned char).
We don't care about EOF (-1), so use macros that cast to (unsigned
char) for us.
Move the handling of "extra" macros (i.e. output format macros) into
the macros.pl mechanism. This allows us to change the format of the
internal macro store in the future - e.g. to a single byte store
without redundant pointers.
Also, stop using indicies into a long array when there is no good
reason to not just use different arrays.
Automatically generate a %define as the first string in the include
block, and just pick the string out of it from that %define statement
to verify existence. That way we eliminate any use of toupper() --
all case-insensitivity in NASM uses tolower()/nasm_tolower().
Automatically provide an include guard for %use packages; the macro
__USE_package__ is automatically defined, and inclusion is suppressed
if it is already defined.
Compress macros.c by representing macro directives with a single byte.
We can do this because we only use the ASCII character range inside
the standard macro files.
Note: we could save significant additional space by not having a
pointer array, and instead relying on the fact that we sweep
sequentially through the output array.
Add a builtin equivalent to the %include directive called %use.
%use includes a standard macro file compiled into the binary; these
come from the macros/ directory in the source code.
The idea here is to be able to provide optional macro packages with
the distribution, without adding complex host filesystem dependencies.
On some platforms, tolower() is implemented as a function call, in
order to handle locale support. We never change locales, so can the
result of tolower() into a table, so we don't have to sit through the
function call every time.
~1.3% overall performance improvement on a macro-heavy benchmark under
Linux x86-64.
- Add %warning directive
- Only unquote an %error or %warning string if it is the only thing on
the directive line.
- Don't expand macros inside a quoted string, even for %error.
Make strings a proper, first-class token type, instead of relying on
the "TOKEN_NUM with tv_charptr" hack. Only convert a string to a
number if requested in an expression context; this also makes it
possible to actually issue a warning when it overflows.
The handling of %? in multi-line macros was broken when the macro name
was preceeded by a label; it would expand to the label instead of the
macro name. This was particularly serious since this was used in
the macro implementation of INCBIN.
The dependency list tail pointer wasn't actually updated correctly.
Fix that. We may want to make this a structure of some sort to make
the code a bit cleaner, but this seems to be the cleanest hack for
now.
Ownership of the filename string was a bit fuzzy, with the result that
we were freeing it even though it was retained for use by __FILE__.
Clean up a number of other memory management issues with the new
quoting code, and change the stdscan implementation to one pass over
the string.
Introduce new preprocessor directives %depend and %pathsearch, and
make incbin a standard macro using these filenames. This lets us
remove the code that makes incbin search the path.
Call expand_smacros() early instead of expand_smacros_in_string()
late. expand_smacros_in_string() seems like a prodigiously bad idea
and a sheer brainfart in my opinion.
Add the -MP option to emit phony targets. Since this means each
header file has to be visited more than once, change the
implementation to use an internal list of all the dependencies, and
centralize the emission of the dependency files.
Implement the dependency options:
-MF: set the file to which dependencies are written.
-MD: generate dependencies in parallel with compilation.
-MT: set the name of the dependency target.
-MQ: same as -MT, but *attempt* to quote it for Makefile safety.
struct hash_table, a fixed-sized structure, is now allocated by the
caller. This lets us integrate it into the Context structure, thus
avoiding an additional dynamically allocated object for no good
reason.
Add some minor code collapsing: make it more obvious that all that
differs is a pointer value, rather than relying on the compiler to do
tail merging.
The compiler is free to store enum pp_token_type into any size integer
small enough to contain all the values up to 2^n-1 for the smallest n
which contains all the values. Force it to size it to integer size,
since we use it to hold macro positional parameters.
%? - substitute the macro name as invoked
%?? - substitute the macro name as defined
In particular:
%idefine keyword $%?
... can be used to make a new keyword "disappear".
Normally, contexts aren't used with a large number of macros, but in
case someone does, do use hash tables for those as well. This
simplifies the code somewhat, since *all* handling of macros is now
done via hash tables.
Future note: consider if it wouldn't be better to allow struct
hash_table to be allocated by the caller, instead of being allocated
by the hash table routine.
Fix buffer overflow in preproc.c due to an incorrect test. In the
code:
for (r = p, s = ourcopy; *r; r++) {
if (r >= p+MAX_KEYWORD)
return tokval->t_type = TOKEN_ID; /* Not a keyword */
*s++ = tolower(*r);
}
*s = '\0';
... the test really needs to be >= since for the pass where there are
equal:
a) a nonzero byte means we have > MAX_KEYWORD characters, and
b) s = ourcopy+MAX_KEYWORD; but if the test doesn't trigger,
we can write one more character *plus* the null byte, overflowing
ourcopy.
First cut at AVX machinery support. The only instruction implemented
is VPERMIL2PS, and it's probably buggy. I'm checking this in with the
hope that other people can start helping out with (a) testing this,
and (b) adding instructions.
NDISASM support is not there yet.
Correct the implementation of %arg and %local.
It's questionable how much they make sense for 64-bit mode; even in
32-bit mode one normally make references off the stack pointer instead
of the base pointer (frame pointer), but that requires keeping track
of the stack pointer offset.
