mstk.mstk reflects %rep conditions as well as actual expanded
macros. However, in_progress is undefined for %rep loops; we instead
want to look at the underlying mmacro, if there is one.
Discovered trying to compile x264.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
%ifdef should accept any argument count. However, requiring
a macro structure return means we have to use the wildcard
argument number (-1), not 0 meaning exactly 0 arguments.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Legacy NASM behavior is (quite frankly the sane one) that a comma
inside a set of parentheses do not split smacro arguments, unless
explicitly using braces to enforce this behavior. Revert to legacy
behavior, which again, is arguably the more correct.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
A trailing comma at the end of an mmacro call is an empty
argument, and so we can't terminate the argument-processing loop. The
only case where skip_white() returning NULL where we are allowed to
terminate the loop is in the case of nparams == 0, i.e. the macro call
has no arguments at all.
Reported-by: gabriele balducci <balducci@units.it>
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
We must not call nasm_error_hold_push() twice... the obvious
leak of the error stack caused all kinds of errors to be suppressed.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Don't issue smacro expansion warnings until we are sure we are
actually *done* with the smacro expansion. The last pass of
expand_smacro_noreset() gets to commit warnings.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
If an include file exists, but cannot be opened, that is still a
critical error.
However, downgrade this from a fatal to a nonfatal error. There really
isn't any reason to stop cold here.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
If the rest of the line is consumed, we may never see tafter, so we
have to test for end of line at line 5412. We already do at 5397, so
it clearly should have been there all along.
Reported-by: <puppet@zju.edu.cn>
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Don't set "defining" until the macro definition is successfully parsed
and we know for sure that we are going to define the macro.
Together with:
a762cd4e54 BR 3392668: preproc: test for macro in TOK_LOCAL_SYMBOL
... this addresses BR 3392668.
Reported-by: <puppet@zju.edu.cn>
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
TOK_LOCAL_SYMBOL is only applicable inside a macro; otherwise error
out just like we do for TOK_MMACRO_PARAM.
This *partially* addresses BR 3392668.
Reported-by: <puppet@zju.edu.cn>
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Determining whether we should warn on defining a single-line macro, with a
name and a certain number of parameters, call a helper function,
smacro_defined(). It does not always return the address of the definition
structure.
Fix the code to be cautiously accessing the definition structure.
Fixes: e91f5cc132 ("preproc: fix %undef of macro aliases, and add
%ifdefalias")
Reported-by: Dale Curtis <dalecurtis@chromium.org>
Link: https://bugzilla.nasm.us/show_bug.cgi?id=3392659
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
Mistreating the macro-parameter, just equivalent to the given
argument number, leads to casting an unnecessary error. Fix to
assemble the conditional code correctly.
Fixes: de7acc3a46 ("preproc: defer %00, %? and %??
expansion for nested macros, cleanups")
Reported-by: C. Masloch <pushbx@ulukai.org>
Link: https://bugzilla.nasm.us/show_bug.cgi?id=3392660
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
The code looked to be unintentionally always nullifying the
token pointer at first place in handling those macro-parameters.
Remove it to avoid segfault.
Fixes: de7acc3a46 ("preproc: defer %00, %? and %??
expansion for nested macros, cleanups")
Reported-by: C. Masloch <pushbx@ulukai.org>
Link: https://bugzilla.nasm.us/show_bug.cgi?id=3392640
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
The code used to stuck in going through whitespace tokens.
Fix to increment towards on the next in the loop.
Reported-by: C. Masloch <pushbx@ulukai.org>
Link: https://bugzilla.nasm.us/show_bug.cgi?id=3392630
Suggested-by: C. Masloch <pushbx@ulukai.org>
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
The code has been fixed to print the corresponding line
numbers of %rep blocks correctly, but only for the first
iteration. For the subsequent iterations, the current line
number on the expansion needs to be explicitly reset again.
Fixes: ab6f831955 ("listing: when listing lines in macros
and rep blocks, show the actual line")
Reported-by: C. Masloch <pushbx@ulukai.org>
Link: https://bugzilla.nasm.us/show_bug.cgi?id=3392626
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
Macro aliases can legitimately point to nonexistent
macros. Furthermore, %undef should remove the pointed-at macro, not
the alias. This led to an infinite loop in the existing code; fix
that.
Add an %ifdefalias directive to test for the existence of an alias.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
1. %xdefine was broken because the code used i as a loop, which is
a standard use for the name i. To avoid that confusion in the
future, use "op" rather than "i" to hold the directive constant.
2. Once (1) was fixed, the smacro expansion code would fail because of
parameter token numbers being indistinguishable between the ongoing
expansion and the %xdefine parameters. In a first pass, mark the
parameters with a new TOK_XDEF_PARAM token number, and change them
to proper parameter token numbers in a second pass, which is now
moved into define_smacro() which is where it arguably belongs.
3. Add a few tests for token pasting and xdefine with and without
parameters.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Technically, this is not necessary, because make_tok_qstr_len()
doesn't rely on NUL termination, and in fact it *can't*, since the
string might contain embedded NULs, but tacking on a NUL is good for
debugging if nothing else. That means reserving space for it!
Reported-by: C. Masloch <pushbx@ulukai.org>
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Fix incorrectly running off the end of the intended string for %strcat
and %substr.
This is a modified version of a patch contributed by C. Masloch.
Reported-by: C. Masloch <pushbx@ulukai.org>
Originally-by: C. Masloch <pushbx@ulukai.org>
Link: https://bugzilla.nasm.us/show_bug.cgi?id=3392599#c11
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
The only way in which
%xdefine(xxx) yyyy zzzz
differs from
%define(xxx) yyyy %[zzzz]
is that in the former case macro arguments get preserved, even if
they are macros defined elsewhere. Revert to that behavior.
Reported-by: C. Masloch <pushbx@ulukai.org>
Link: https://bugzilla.nasm.us/show_bug.cgi?id=3392623
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
%exitrep should should stop emitting code immediately, not just
terminate the loop when we hit %endrep. There is a bunch of hacky code
that special-cases that using istk->in_progress == 0.
The handling of the tail of %exitrep, %include and non-emitting
conditionals using entirely different mechanisms is just dumb. They
need to be unified.
Link: https://bugzilla.nasm.us/show_bug.cgi?id=3392612
Reported-by: Jason Hood <jadoxa@yahoo.com.au>
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Major cleanups of the preprocessor. In particular, the
block-allocation of Token is pretty ridiculous since nearly every
token requires a text allocation anyway. Change the definition of
Token so that only very long tokens (48+ characters on 64-bit systems)
need to be stored out of line.
If malloc() preserves alignment (XXX: glibc doesn't) then this means
that each Token will fit in a cache line.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
paste_tokens() would not null-terminate the buffer before passing it
to tokenize(), resulting in garbage or a memory overwrite.
In several places the next pointers got confused; sometimes causing a
circular list and sometimes an invalid pointer.
Some minor code cleanups while fixing things, too...
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
expand_one_smacro() would corrupt the end of the list if a macro
expanded to another macro with arguments, which was also the last part
of the expansion.
Instead of doing all that testing with ttail, just scan forward at the
end to find the tail pointer; it is O(n) regardless.
Clean up the handling of tokens: use inline functions rather than odd
macros that sometimes modify their arguments and sometimes don't, and
fold some common code into new functions.
The tok_is() and tok_isnt() functions always are used with single
characters, so make it explicitly so (and remove the local hacks used
in some places.)
Allow using nasm_malloc() rather than blocked Tokens; this makes tools
like valgrind more useful in their reports.
For the future, consider making Tokens a separate memory allocation
immediately followed by the text, instead of using a pointer; we
allocate space for the string in almost every case anyway. Also
consider making it a doubly linked list...
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
%ifusable tests to see if a certain %use package is available in this
version of NASM.
%ifusing tests if a certain %use packages is already loaded.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Move the parsing of %use package names to a separate routine, and stop
using get_id() for that purpose -- get_id() is wrong in a number of
ways.
This also means we can drop the error string argument to get_id().
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Since pp_error_list_macros() was introduced, the only need for
pp_verror() is to suppress error messages in certain contexts. Replace
this function with a preprocessor callback,
preproc->pp_suppress_error(), so we can drop the nasm_verror()
function pointer entirely.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Enough users expect the namespace starting with underscore to be safe
for symbols. Change our private namespace from __foo__ to
__?foo?__. Use %defalias to provide backwards compatiblity (by using
%defalias instead of %define, we handle the case properly where the
user changes the value.)
Add a preprocessor directive:
%aliases off
... to disable all smacro aliases and thereby making the namespace
clean.
Finally, fix infinite recursion when seeing %? or %?? due to
paste_tokens(). If we don't paste anything, the expansion is done.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
If we adjust nparams due to default or greedy arguments, we need to
re-terminate the params[] array.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
The smacro argument list cannot be preceded by whitespace, or we
wouldn't be able to define no-argument smacros the expansion of which
starts with (.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Correctly handle empty mmacro arguments that still have preceding
whitespace tokens.
Default mmacro parameters are obtained by count_mmac_params() so they,
too, need to be shifted over by one.
Add an option to list mmacro calls with arguments. Name this -Lm;
remove the old -Lm option to -Ls since it is related to single-line
macros.
Trivially optimize the case where an mmacro is called from within
itself: if all possible mmacros are excluded by loop removal, there is
no need to delve into the mmac processing code.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Fix the (severely broken handling of) varadic macros.
Add a conditional comma operator "%,". This expands to a comma unless
followed by a null expansion of some sort, which allows suppressing
the comma before an empty argument (usually varadic, but not
necessarily.)
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Add a few more types of smacro arguments, and clean stuff up in
process.
If declared with an &, an smacro parameter will be quoted as a string.
If declared with a +, it is a greedy/varadic parameter.
If declared with an !, don't strip whitespace and braces (useful with &).
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
BR 3392603: When doing nested macro definitions, we need %00, %? and
%?? expansion to be deferred to actual expansion time, just as the
other parameters.
Do major cleanups to the mmacro expansion code.
Reported-by: Alexandre Audibert <alexandre.audibert@outlook.fr>
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
BR 3392602: mmacros should not nest unless so explicitly specified.
Reported-by: C. Masloch <pushbx@38.de>
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Since we use 127 not 0 for end of line in stdmac packages now, we
can't simply use the __USE_*__ macro as a string to test for a %use
package. Keep an internal array of state instead.
Fix the stripping of comments from lines in macro files.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Handle up to 160 directives for stdmac compression. This is done by
allowing the directive numbers to wrap around (128-255, 0-31), using
127 for end of line, and forcing any whitespace character to be space.
Make macros.c a bit more legible by using #defines for the byte codes;
strictly for the benefit of the human reader.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Simplify the handling of conditionals; remove the PPC_* types.
Automate the generation of case-sensitive versus case-insensitive
directives, and make it so the bulk of the code doesn't have to worry
about it.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Introduce "alias smacros", which are the smacro equivalent of
symlinks; when used with the various smacro-defining and undefining
directives, they affect the macro they are aliased to. Only explicit
%defalias, %idefalias, and %undefalias affect them.
This is intended for being able to rename macros while retaining the
legacy names.
This patch also removes an *astonishing* amount of duplicated
code:
1. Every caller to defined_smacro() and undef_smacro() would call
get_ctx() to mangle the macro name; push that into those functions.
2. Common code to get an smacro identifier.
3. Every code path that returns DIRECTIVE_FOUND also has to do
free_tlist(origline); make it do so.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
tline got advanced a token too far, with the obvious results that the
facility name got truncated. Skip whitespace *after* expand_smacro(),
not before.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
expand_smacro() consumes its input, so we need to truncate the input
list so we can call free_tline(origline) safely.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Add listing options:
-Lb to show builtin macro packages
-Lf to override .nolist
Do some cleanups in the process, in particular generalize read_line()
between stdmac and file alternatives.
When processing stdmac, create an istk entry for it. This means stdmac
can be identified by istk->fp == NULL. At some future date there could
even be a function pointer to an appropriate read function.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
Additional listing options:
-Ld to display counts in decimal
-Lp to output a list file in every pass (to make sure one exists)
Clean up the help output and make it comprehensive. The -hf and -y
options are no longer necessary, although they are supported for
backwards compatiblity.
Fix macro-levels so it actually count descent levels; a new
macro-tokens limit introduced for the actual token limit.
Slightly simplify the limits code.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
When printing lines coming from %rep blocks and macros, show the line
number corresponding to the line actually being printed.
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
It turns out that in tokenize() we would sometimes truncate a token
string by inserting a NUL into the input string, expecting new_Token()
to pick it up using strlen(). With explicit lengths, that no longer
works, but there is a better solution anyway: instead of inserting
NUL characters, keep track of where the token actually ends and feed
the correct length to new_Token().
This triggered a buffer overflow in detoken(), add a debug level 2
assert for this condition. Use a relatively high debug level, because
strlen() is fairly expensive, and this is an extremely
performance-critical path.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Add an -L option for additional listing information. Currently
supported is -Le, which emits each line after processing through the
preprocessor, and -Lm, which displays each single-line macro defined
or undefined.
NASM doesn't preserve the names of unused arguments, nor does it have
any technical reason to do so. Instead of adding complexity to save
them, make unnamed parameters official by specifying an empty string
in the argument list.
This has the additional advantage that () is now simply considered a
single empty argument, which means that NASM should now properly
handle things like:
%define myreg() eax
mov edx,myreg()
... similar to how the C preprocessor allows an empty macro argument
list which is distinct from a macro with no arguments whatsoever.
Signed-off-by: H. Peter Anvin (Intel) <hpa@zytor.com>
