If the board info sets the sim to a basename that is found via $PATH
(which is the default dejagnu behavior), the logic here to use its
dirname to find rvdummy fails because it looks for `./rvdummy`. So
switch it to always use the local build of rvdummy which is the one
we want to be testing against in the first place.
If we get a request for testing against a different setup, we can
figure out & document the needs at that point, and then setup some
config knobs to control it.
Add missing $(EXEEXT) to dependencies on bits-gen. These are actually
build-only tools, but automake doesn't allow for build & host tools, so
the rules are re-using EXEEXT.
We usually test against the newlib/libgloss environment, but for a
few ports that also support Linux apps, we want to test that logic
too. A lot of the C code is written such that it works with either
newlib/libgloss or glibc/linux toolchains, but we have some tests
that end up being Linux-specific. Cris has been using the target
tuple as a rough proxy for this (where cris*-*-elf is assumed to be
newlib/libgloss, and everything else is glibc/linux), but that is a
bit too rough, and it doesn't work in a multitarget build.
So lets create a few stub files that we can do compile tests with
to detect the different setups, and then let tests declare which
one they require (if they require any at all).
Both bfin & cris ports test the C compiler to see if it works, but in
their own way. Unify the checks in the common code so we can leverage
them in more ports in the future, and collapse the bfin & cris code.
The sim_init function was called by runtest for each test when --tool
was set to sim. When we changed to --tool '' to collapse the testsuite
dir, the init function was no longer called on every test. However, it
was still being called explicitly by config/default.exp. It's not clear
why that explicit call ever existed since, in the past, it meant it was
redundant.
Lets drop the single sim_init call in config/default.exp and move it out
to all our tests. This replicates the runtest behavior so we can setup
variables on a per-test basis which allows us to recollapse the sim_path
logic back. We'll also leverage this in the future for toolchain setup.
Also add a few comments clarifying the overall runtime behavior.
If the cris sim hasn't been built yet, trying to run its testsuite
will hang indefinitely. The common sim APIs already have this, so
copy it over to the cris forks of the test+run functions.
The tests assume that the cwd is the objdir directory and write its
intermediates to there all the time. When using runtest's --objdir
setting though, this puts the files in the wrong place. This isn't
a big problem currently as we never change --objdir, but in order to
support parallel test execution, we're going to start setting that
option, so clean up the code ahead of time.
We also have to tweak some of the cris tests which were making
assumptions about the argv[0] value.
Now that all the other toolchain settings have been renamed to match
the dejagnu settings of XXX_FOR_TARGET, rename global_sim_options to
SIMFLAGS_FOR_TARGET too.
Only a few tests actually use global_ld_options, but we can replace the
sim-specific settings with the dejagnu common LDFLAGS_FOR_TARGET and get
the same result.
Only a few tests actually use global_as_options, but we can replace the
sim-specific settings with the dejagnu common ASFLAGS_FOR_TARGET and get
the same result.
Nothing in the testsuite is using this setting, so let's drop it.
Any code that wants to set compiler flags can use CFLAGS_FOR_TARGET
instead to get the same effect.
These haven't been used in over 20 years. The sim testsuite used to
run these tools itself directly, but back in ~1999 it switched to the
dejagnu helpers (e.g. target_assemble & target_link), and the dejagnu
logic only utilizes XXX_FOR_TARGET variables. Punt them here to avoid
confusion with dead code.
The v850 testsuite code has been testing the $opt variable, but this
was never actually set anywhere globally or v850-specific. Instead,
this was a random variable leaking out of the sh testsuite code. As
far as I can tell, it has always been this way. That means the code
only ever tested the v850 cpu target (which is the default).
This failure can be easily seen in practice by running the v850 code
in isolation and seeing it crash:
$ runtest v850/allinsns.exp
...
Running target unix
Using /usr/share/dejagnu/baseboards/unix.exp as board description file for target.
Using /usr/share/dejagnu/config/unix.exp as generic interface file for target.
Using ../../../sim/testsuite/config/default.exp as tool-and-target-specific interface file.
WARNING: Assuming target board is the local machine (which is probably wrong).
You may need to set your DEJAGNU environment variable.
Running ../../../sim/testsuite/v850/allinsns.exp ...
ERROR: tcl error sourcing ../../../sim/testsuite/v850/allinsns.exp.
ERROR: tcl error code TCL LOOKUP VARNAME opt
ERROR: can't read "opt": no such variable
while executing
"switch -regexp -- $opt {
Backing up a bit, the reason for this logic in the first place is
because the common sim testsuite code makes an assumption about the
assembler options with cpu_option -- the option and its value are
always separated by an =. This is not the case with v850. So tweak
the core sim logic a bit to support omitting the = so that we can
switch v850 to the standard all_machs setting and avoid opt entirely.
The common framework supports $srcdir & $subdir replacements already,
so replace the custom @srcdir@ logic with those. Since the replace
happens in slurp_options that cris already uses, we don't have any
logic to port over there. We have to duplicate that into the cris
slurp_rv helper though.
This tag is used to force tests to be built dynamically (i.e. without
-static linking). This is because cris-sim.exp in dejagnu turns on
static linking in ldflags.
The default configs and runtest flags shouldn't load these boards.
If these settings are still needed, we should figure out a different
way of suppressing the stock settings wholesale. We want these to
all pass out of the box with little to no configuration so that they
can run in a multitarget build.
With dropping "dynamic", it'll be easier to merge the custom cris
test logic with the common sim test logic.
Provide explicit control over the program's environment with the
basic set/unset/clear options. These are a bit clunky to use,
but they're functional.
The env set operation is split out into a separate function as it'll
be used in the next commit.
With these in place, we can adjust the custom cris testsuite to use
the now standard options and not its one-off hack.
The point of this test is to just make sure the usage text is shown,
not the exact details of the usage text. So shorten the output test
to match the beginning. This fixes breakage when the output changed
slightly to include [--].
The #dest field used in the cris testsuite is a bit of hack to set the
argv[0] for the tests to read out later on. Now that the sim has an
option to set argv[0] explicitly, we don't need this custom field, so
let's drop it to harmonize the testsuites a little.
Set the mach to the right value all the time, and update xfail to
say the test fails on all targets. WIth multitarget testing, the
idea of target here doesn't make much sense.
Since gcc dropped support for a.out starting with 4.4.0 in 2009, it's
been impossible to verify this code actually still works. Since it
crashes in ld, and it uses a config option that no other tests uses
and we want to remove, drop the test to avoid all the trouble.
There's no need to sync testutils.inc with remote hosts. The one
we have in the source tree is all we need and only thing we test.
Delete it to simplify.
Objcopy was used to create a binary file of just the executable code
since the environment requires code to based at address 0. We can
accomplish the same thing with the -Ttext=0 flag, so switch to that
to get rid of custom logic.
Switch from the centralized list in the exp file to each test declaring
its own requirements which they're already (mostly) doing. This will
increase coverage slightly by running more tests in more configurations
since the hardcoded exp list was a little out of date.
We have to mark the psh* tests as shdsp only (to match what the exp
file was doing), mark the fsca & fsrra tests as failing (since they
weren't even being run by the exp file), and to fix the expected
output & status of the fail test.
Now that ChangeLog entries are no longer used for sim patches,
this commit renames all relevant sim ChangeLog to ChangeLog-2021,
similar to what we would do in the context of the "Start of New
Year" procedure.
The purpose of this change is to avoid people merging ChangeLog
entries by mistake when applying existing commits that they are
currently working on.
Also throw in a .gitignore entry to keep people from adding new
ChangeLog files anywhere in the sim tree.
We don't want to handle Linux syscalls when in the virtual environment,
just the user environment, so adjust the Linux traps logic to check for
that specifically (instead of just skipping the operating environment).
Also tweak some testcases to explicitly specify the environment they run
under rather than relying on the default matching their needs. This gets
the tests passing for all m32r targets.
On a host without installed libbfd, this patch fixes the following
"make check-sim" errors for both pru cross target, and native x86_64:
In file included from ../../../binutils/sim/common/sim-basics.h:131,
from testsuite/common/bits32m0.c:13:../../../binutils/sim/../include/gdb/callback.h:55:10: fatal error: bfd.h: No such file or directory
55 | #include "bfd.h"
| ^~~~~~~
Signed-off-by: Dimitar Dimitrov <dimitar@dinux.eu>
When in the virtual environment, have brki 8 trigger libgloss syscalls
like other ports. This also matches the ABI that Linux uses for its
syscalls (ignoring the syscall table differences).
Exit status 77 is common (including the autotools world) to indicate
"skip this test". Add support for mapping that to "unsupported" as
that's the closest in the dejagnu world.
If the port hasn't been enabled, don't try to run its tests. Making
this dynamic simplifies the test harnesses and avoids duplicating a
bunch of target tuple checks.
Provide a simple example simulator for people porting to new targets
to use as a reference. This one has the advantage of being used by
people and having a fun program available for it.
It doesn't require a special target -- the example simulators can be
built for any existing port.
Now that we have the common automake build with support for build-time
programs working, we can integrate the common tests into the default
`make check` flow.
This is needed to move to automake & its dejagnu-provided logic,
and eventually by the unified sim logic. The $arch is used only
to figure out which `run` program to use when running tests, and
as we move to a single top-level build, we can delete this and
use sim/run directly.
This is a hand-written implementation that should have fairly complete
coverage for the base integer instruction set ("i"), and for the atomic
("a") and integer multiplication+division ("m") extensions. It also
covers 32-bit & 64-bit targets.
The unittest coverage is a bit weak atm, but should get better.
This wasn't supposed to be in here when it was first merged as we
had specifically disabled it for all the tests (and ADI has papers
in place w/the FSF). Clean up this one.
Now that all port tests live under testsuite/sim/*/, and none live
in testsuite/ directly, flatten the structure by moving all of the
dirs under testsuite/sim/ to testsuite/ directly.
We need to stop passing --tool to dejagnu so that it searches all
dirs and not just ones that start with "sim". Since we have no
other dirs in this tree, and no plans to add any, should be fine.
Now that we've moved all ports to dejagnu & testsuite/sim/, the only
thing the testsuite/configure script has been doing is filling in the
sim_arch field in the testsuite/Makefile. We can simply let the top
sim/configure script do that for us now. This simplifies & speeds up
the build a bit by killing an entire configure script.
This is the only target using a dir directly under testsuite/. All
others use sim/<arch>/ instead. Relocate it so all targets look the
same, and so we can leverage the common test harness.
We drop loop.s in the process because it was never referenced and
was just 2 lines of code.
All other test files are moved & have directives added to the top so
that the test harness can invoke them correctly.
No tests were ever added in here in the ~22 years since it was first
created. Seems unlikely any tests will be added at this rate, and
the sim/mips/ testdir already has some (light) coverage for this
target. So punt the tree.
The frv-elf subdir contained five tests:
* cache: A cache test of some sort.
* exit47: A program to test exit status of 47 from sim.
* grloop: Some basic limited loop test program.
* hello: Standard "hello world" output program.
* loop: An infinite loop program.
The loop.s test is never referenced anywhere, and is all of 2 lines.
Anyone who really needs a while(1); test case and re-implement it
themselves locally.
The cache.s code isn't referenced anywhere because it requires some
custom args to the run program, and when this testcase was added, we
didn't have any support for that. We do now, so we can add a header
to enable that. Turns out the code crashes even with those, so turn
around and mark it xfail. Maybe someone someday will care.
That leaves the small exit47, grloop, and hello tests. Now that the
sim test harness supports testing for custom exit status, we can move
them all to sim/frv/ to maintain test coverage.
The remaining differences between frv-elf & sim/frv are:
* frv-elf/ runs for frv-*-elf while sim/frv/ runs for frv*-*-*.
* frv-elf/ runs "*.s" files while sim/frv/ only has .cgs and such.
On closer inspection, these are also meaningless distinctions:
* There is nothing specific to the tests that require an *-elf
target. Normally that would mean newlib+libgloss type stuff,
but there's no such requirement in frv-elf/.
* The ".s" suffix is the standard "this is an assembly file" suffix.
Since FRV is a CGEN target, we can reuse the existing convention of
".ms" to mean "miscellaneous .s" as in "this is an assembly file,
and run/bucket its test results in the miscellaneous category".
So moving frv-elf/{cache,exit47,grloop,hello}.s to sim/frv/*.ms makes
sense and simplifies things quite a bit for the target while also
slightly increasing the coverage for some tuples.
The m32r-elf subdir contained three tests:
* exit47: A program to test exit status of 47 from sim.
* hello: Standard "hello world" output program.
* loop: An infinite loop program.
There's already a sim/m32r/hello.ms test that does exactly the same
thing as m32r-elf/hello.s, so we can delete that.
The loop.s test is never referenced anywhere, and is all of 2 lines.
Anyone who really needs a while(1); test case and re-implement it
themselves locally.
That leaves the single exit47 test. Now that the sim test harness
supports testing for custom exit status, we can easily move that to
sim/m32r/exit47.ms to maintain test coverage.
The remaining differences between m32r-elf & sim/m32r are:
* m32r-elf/ runs for m32r-*-elf while sim/m32r/ runs for m32r*-*-*.
* m32r-elf/ runs "*.s" files while sim/m32r/ runs "*.ms" files.
On closer inspection, these are also meaningless distinctions:
* There is nothing specific to the tests that require an *-elf
target. Normally that would mean newlib+libgloss type stuff,
but there's no such requirement in m32r-elf/.
* The ".s" suffix is the standard "this is an assembly file"
suffix. Turns out ".ms" is just how sim/m32r/ (and a few other
CGEN based targets) categorize/bucket test cases. It simply
means "miscellaneous .s" as in "this is an assembly file, and
run/bucket its test results in the miscellaneous category".
So moving m32r-elf/exit47.s to sim/m32r/exit47.ms makes sense and
simplifies things quite a bit for the target while also slightly
increasing the coverage for some tuples.
