...when we know we have a working compiler. This will reduce
the risk of faulty edits by exposing them rather than hiding
them as "unresolved". It also harmonizes behavior with that of
run_sim_test.
* c/c.exp: Mark C tests failing compilation test errors.
Calls to basename were added here as part of commit
e1e1ae6e9b "sim: testsuite: fix objdir handling", but that
commit missed adding "#include <libgen.h>" or the equivalent
GNU extension, see basename(3). Fixing that shows a logical
error in the change to openpf1.c; the non-/-prefixed
code-path was changed instead of the "/"-prefixed code-path,
which is the one executed after that commit.
For "newlib" these tests failed linking after that commit.
Recent newlib has the (asm-renamed) GNU-extension-variant of
basename, but we're better off not using it at all.
Unfortunately, compilation failures for C tests run by the
machinery in c.exp are currently just marked "unresolved",
in contrast to C and assembler tests run by calling
run_sim_test.
The interaction of calling with the full program-path vs.
use of --sysroot exposes a consistency problem: when
--sysroot is used, argv[0] isn't the path by which the
program can find itself. It's undecided whether argv[0] for
the program running in the simulator should be edited
(related to the naked argument to the simulator before
passing on to the simulated program) to remove a leading
--sysroot. Either way, such a change would be out of scope
for this commit.
* c/stat3.c (mybasename): New macro. Use it instead of basename.
* c/openpf1.c: Correct basename-related change and update related
comment.
Apply the new run_sim_test option "require" as in "#require
simoption --cris-900000xx" for all tests using that option.
This allows a clean test-suite-run for a build with
--disable-sim-hardware, where that option is not supported,
by skipping those tests as "untested".
sim/testsuite/cris:
* asm/io1.ms, asm/io2.ms, asm/io3.ms, asm/io6.ms,
asm/io7.ms: Call "#require: simoption --cris-900000xx".
Simulator features can be present or not, typically
depending on different-valued configure options, like
--enable-sim-hardware[=off|=on]. To avoid failures in
test-suite-runs when testing such configurations, a new
predicate is needed, as neither "target", "progos" nor
"mach" fits cleanly.
The immediate need was to check for presence of a simulator
option, but rather than a specialized "requires-simoption:"
predicate I thought I'd handle the general (parametrized)
need, so here's a generic predicate machinery and a (first)
predicate to use together with it; checking whether a
particular option is supported, by looking at "run --help"
output. This was inspired by the check_effective_target_
machinery in the gcc test-suite.
Multiple "requires: <requirement> <parameter>" form a list of
predicates (with parameters), to be used as a conjunction.
sim/testsuite:
* lib/sim-defs.exp (sim_check_requires_simoption): New function.
(run_sim_test): Support "requires: <requirement> <parameter>".
For reasons that remain largely to be investigated (besides
the apparent lack of synchronization between two processes),
this test fails randomly, with two different sets of common
outputs. Curiously, that doesn't happen for the other
similar tests. There's a comment that mentions this, though
that doesn't make it a sustainable part of a test-suite.
(Known-blinking tests should be disabled until fixed.)
sim/testsuite/cris:
* hw/rv-n-cris/irq1.ms: Disable by use of a never-matched
"progos" value.
Commit a39487c668 "sim: cris: use -sim with C tests for cris-elf
targets" caused " -sim" to be appended to CFLAGS_FOR_TARGET for
cris*-*-elf, where testing had until then relied on
"RUNTESTFLAGS=--target_board=cris-sim" being passed when running "make
check-sim", adding the right options. While "-sim" happens to work,
the baseboard-file cris-sim.exp uses "-sim3" so for consistency use
that instead.
Then commit b42f20d2ac "sim: testsuite: drop most specific istarget
checks" caused " -sim" to be appended for *all* targets, which just
doesn't work. For example, for crisv32-linux-gnu, that's not a
recognized option and will cause a dejagnu error and further testing
in c.exp will be aborted.
While cris-sim.exp appends "-static" for *-linux-gnu, further changes
in the test-suite have caused "linux"-specific tests to break, so that
part will be tended to separately.
But, save and restore CFLAGS_FOR_TARGET around the modification and
use where needed, to not have the CRIS-specific modification affect a
continuing test-run (possibly for other targets).
sim/testsuite/cris:
* c/c.exp (CFLAGS_FOR_TARGET): Replace appended option " -sim"
with " -sim3", but do it conditionally for newlib targets. Save
and restore CFLAGS_FOR_TARGET in saved_CFLAGS_FOR_TARGET such
that it doesn't affect the value of CFLAGS_FOR_TARGET outside
c.exp.
If we don't set this variable, it doesn't exist, and using "#progos:"
in an assembler-file will cause an error rather than just skipping the
test, viz:
Running /src/sim/testsuite/cris/hw/rv-n-cris/rvc.exp ...
ERROR: tcl error sourcing /src/sim/testsuite/cris/hw/rv-n-cris/rvc.exp.
ERROR: can't read "global_cc_os": no such variable
while executing
"if { $opts(progos) != "" && $opts(progos) != $global_cc_os } {
untested $subdir/$name
return
}"
(procedure "run_sim_test" line 102)
Neither the commit introducing progos, nor the top comment
in run_sim_test, mentions progos as intended only for C
tests, or that its use must be gated on $global_cc_works !=
0, so (not) setting it in the no-working-compiler path seems
just overlooked.
Allowing it to be used for assembler tests makes it usable
for e.g. an always-false predicate and in expressions in
.exp files without gating on $global_cc_works != 0.
With this patch, global_cc_os is set to "", just as for "unknown OS".
sim/testsuite:
* lib/sim-defs.exp (sim_init_toolchain): Set global_cc_os also when
no working target C compiler is found.
This commit brings all the changes made by running gdb/copyright.py
as per GDB's Start of New Year Procedure.
For the avoidance of doubt, all changes in this commits were
performed by the script.
The ## marker tells automake to not include the comment in its
generated output, so use that in most places where the comment
only makes sense in the inputs.
Running frv's allinsn in serial is quite slow due to the sheer number
of tests it contains. By splitting it up and running in parallel, the
execution time on my system goes from ~100sec to ~60sec.
We'll rely on the toolchain probing to determine whether each arch's
tests can be run rather the current configure target. This allows
testing all of the ports in a multitarget configuration.
For now, we don't reformat the files entirely to make it easier to
review, and in case we need to make adjustments. Once this feels
like it's stable, we can flatten the code a bit by removing the if
statement entirely.
Break up the dejagnu logic so that we can parallelize the testsuite.
This takes a page from gcc & gdb where each .exp is run in isolation
instead of in serial.
For most targets, this doesn't make much of a difference as they only
have a single .exp. A few (like cris & frv) have multiple .exp though
and will see a bit of a speed up.
The real gain is when testing a multitarget build. This way we can
run all the targets in parallel and cut the execution time a bit.
On my system, it goes from ~155sec to ~100sec.
We can gain further speedups by splitting up some of the larger .exp
files into smaller groups. We'll do that in a followup though.
Leverage the new per-port toolchain settings to initialize the env
for eeach set of tests. This allows us to run all the tests in a
multitarget build if the user sets up the vars. If they don't, we
can still skip all the tests.
Gas does not support multitarget builds -- it still only supports
a single input & output format. ld is a bit better, but requires
manual flags to select the right output. This makes it impossible
to run the complete testsuite in a multitarget build.
To address this limitation, create a suite of FOR_TARGET variables
so these can be set to precompiled as & ld programs. It requires
a bit of setup ahead of time, but it's a one-time cost, and makes
running the full testsuite at once much easier.
As we setup more reliable CC_FOR_TARGET variables for each target, the
bfin way of overriding it to stuff custom CFLAGS doesn't scale well.
Add a dedicated CFLAGS_FOR_TARGET_init setting that each set of tests
can setup if they want to add custom options.
If the board info sets the sim to a basename that is found via $PATH
(which is the default dejagnu behavior), the logic here to use its
dirname to find rvdummy fails because it looks for `./rvdummy`. So
switch it to always use the local build of rvdummy which is the one
we want to be testing against in the first place.
If we get a request for testing against a different setup, we can
figure out & document the needs at that point, and then setup some
config knobs to control it.
Add missing $(EXEEXT) to dependencies on bits-gen. These are actually
build-only tools, but automake doesn't allow for build & host tools, so
the rules are re-using EXEEXT.
We usually test against the newlib/libgloss environment, but for a
few ports that also support Linux apps, we want to test that logic
too. A lot of the C code is written such that it works with either
newlib/libgloss or glibc/linux toolchains, but we have some tests
that end up being Linux-specific. Cris has been using the target
tuple as a rough proxy for this (where cris*-*-elf is assumed to be
newlib/libgloss, and everything else is glibc/linux), but that is a
bit too rough, and it doesn't work in a multitarget build.
So lets create a few stub files that we can do compile tests with
to detect the different setups, and then let tests declare which
one they require (if they require any at all).
Both bfin & cris ports test the C compiler to see if it works, but in
their own way. Unify the checks in the common code so we can leverage
them in more ports in the future, and collapse the bfin & cris code.
The sim_init function was called by runtest for each test when --tool
was set to sim. When we changed to --tool '' to collapse the testsuite
dir, the init function was no longer called on every test. However, it
was still being called explicitly by config/default.exp. It's not clear
why that explicit call ever existed since, in the past, it meant it was
redundant.
Lets drop the single sim_init call in config/default.exp and move it out
to all our tests. This replicates the runtest behavior so we can setup
variables on a per-test basis which allows us to recollapse the sim_path
logic back. We'll also leverage this in the future for toolchain setup.
Also add a few comments clarifying the overall runtime behavior.
If the cris sim hasn't been built yet, trying to run its testsuite
will hang indefinitely. The common sim APIs already have this, so
copy it over to the cris forks of the test+run functions.
The tests assume that the cwd is the objdir directory and write its
intermediates to there all the time. When using runtest's --objdir
setting though, this puts the files in the wrong place. This isn't
a big problem currently as we never change --objdir, but in order to
support parallel test execution, we're going to start setting that
option, so clean up the code ahead of time.
We also have to tweak some of the cris tests which were making
assumptions about the argv[0] value.
Now that all the other toolchain settings have been renamed to match
the dejagnu settings of XXX_FOR_TARGET, rename global_sim_options to
SIMFLAGS_FOR_TARGET too.
Only a few tests actually use global_ld_options, but we can replace the
sim-specific settings with the dejagnu common LDFLAGS_FOR_TARGET and get
the same result.
Only a few tests actually use global_as_options, but we can replace the
sim-specific settings with the dejagnu common ASFLAGS_FOR_TARGET and get
the same result.
Nothing in the testsuite is using this setting, so let's drop it.
Any code that wants to set compiler flags can use CFLAGS_FOR_TARGET
instead to get the same effect.
These haven't been used in over 20 years. The sim testsuite used to
run these tools itself directly, but back in ~1999 it switched to the
dejagnu helpers (e.g. target_assemble & target_link), and the dejagnu
logic only utilizes XXX_FOR_TARGET variables. Punt them here to avoid
confusion with dead code.
The v850 testsuite code has been testing the $opt variable, but this
was never actually set anywhere globally or v850-specific. Instead,
this was a random variable leaking out of the sh testsuite code. As
far as I can tell, it has always been this way. That means the code
only ever tested the v850 cpu target (which is the default).
This failure can be easily seen in practice by running the v850 code
in isolation and seeing it crash:
$ runtest v850/allinsns.exp
...
Running target unix
Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target.
Using ../../../sim/testsuite/config/default.exp as tool-and-target-specific interface file.
WARNING: Assuming target board is the local machine (which is probably wrong).
You may need to set your DEJAGNU environment variable.
Running ../../../sim/testsuite/v850/allinsns.exp ...
ERROR: tcl error sourcing ../../../sim/testsuite/v850/allinsns.exp.
ERROR: tcl error code TCL LOOKUP VARNAME opt
ERROR: can't read "opt": no such variable
while executing
"switch -regexp -- $opt {
Backing up a bit, the reason for this logic in the first place is
because the common sim testsuite code makes an assumption about the
assembler options with cpu_option -- the option and its value are
always separated by an =. This is not the case with v850. So tweak
the core sim logic a bit to support omitting the = so that we can
switch v850 to the standard all_machs setting and avoid opt entirely.
The common framework supports $srcdir & $subdir replacements already,
so replace the custom @srcdir@ logic with those. Since the replace
happens in slurp_options that cris already uses, we don't have any
logic to port over there. We have to duplicate that into the cris
slurp_rv helper though.
This tag is used to force tests to be built dynamically (i.e. without
-static linking). This is because cris-sim.exp in dejagnu turns on
static linking in ldflags.
The default configs and runtest flags shouldn't load these boards.
If these settings are still needed, we should figure out a different
way of suppressing the stock settings wholesale. We want these to
all pass out of the box with little to no configuration so that they
can run in a multitarget build.
With dropping "dynamic", it'll be easier to merge the custom cris
test logic with the common sim test logic.
Provide explicit control over the program's environment with the
basic set/unset/clear options. These are a bit clunky to use,
but they're functional.
The env set operation is split out into a separate function as it'll
be used in the next commit.
With these in place, we can adjust the custom cris testsuite to use
the now standard options and not its one-off hack.
The point of this test is to just make sure the usage text is shown,
not the exact details of the usage text. So shorten the output test
to match the beginning. This fixes breakage when the output changed
slightly to include [--].
The #dest field used in the cris testsuite is a bit of hack to set the
argv[0] for the tests to read out later on. Now that the sim has an
option to set argv[0] explicitly, we don't need this custom field, so
let's drop it to harmonize the testsuites a little.
Set the mach to the right value all the time, and update xfail to
say the test fails on all targets. WIth multitarget testing, the
idea of target here doesn't make much sense.
Since gcc dropped support for a.out starting with 4.4.0 in 2009, it's
been impossible to verify this code actually still works. Since it
crashes in ld, and it uses a config option that no other tests uses
and we want to remove, drop the test to avoid all the trouble.
There's no need to sync testutils.inc with remote hosts. The one
we have in the source tree is all we need and only thing we test.
Delete it to simplify.
Objcopy was used to create a binary file of just the executable code
since the environment requires code to based at address 0. We can
accomplish the same thing with the -Ttext=0 flag, so switch to that
to get rid of custom logic.
Switch from the centralized list in the exp file to each test declaring
its own requirements which they're already (mostly) doing. This will
increase coverage slightly by running more tests in more configurations
since the hardcoded exp list was a little out of date.
We have to mark the psh* tests as shdsp only (to match what the exp
file was doing), mark the fsca & fsrra tests as failing (since they
weren't even being run by the exp file), and to fix the expected
output & status of the fail test.
Now that ChangeLog entries are no longer used for sim patches,
this commit renames all relevant sim ChangeLog to ChangeLog-2021,
similar to what we would do in the context of the "Start of New
Year" procedure.
The purpose of this change is to avoid people merging ChangeLog
entries by mistake when applying existing commits that they are
currently working on.
Also throw in a .gitignore entry to keep people from adding new
ChangeLog files anywhere in the sim tree.
We don't want to handle Linux syscalls when in the virtual environment,
just the user environment, so adjust the Linux traps logic to check for
that specifically (instead of just skipping the operating environment).
Also tweak some testcases to explicitly specify the environment they run
under rather than relying on the default matching their needs. This gets
the tests passing for all m32r targets.
