Go to file
Andrew Burgess c3d64d467d gdb/testsuite: handle invalid .exp names passed in TESTS
I ran some tests like:

  $ make check-gdb TESTS="gdb.base/break.exp"

then, then I went to rerun the tests later, I managed to corrupt the
command line, like this:

  $ make check-gdb TESTS="gdb.base/breakff.exp"

the make command did exit with an error, but DejaGnu appeared to
report that every test passed!  The tail end of the output looks like
this:

  Illegal Argument "no-matching-tests-found"
  try "runtest --help" for option list
  		=== gdb Summary ===

  # of expected passes		115
  /tmp/build/gdb/gdb version  13.0.50.20220831-git -nw -nx -iex "set height 0" -iex "set width 0" -data-directory /tmp/build/gdb/testsuite/../data-directory

  make[3]: *** [Makefile:212: check-single] Error 1
  make[3]: Leaving directory '/tmp/build/gdb/testsuite'
  make[2]: *** [Makefile:161: check] Error 2
  make[2]: Leaving directory '/tmp/build/gdb/testsuite'
  make[1]: *** [Makefile:1916: check] Error 2
  make[1]: Leaving directory '/tmp/build/gdb'
  make: *** [Makefile:13565: check-gdb] Error 2

For a while, I didn't spot that DejaGnu had failed at all, I saw the
115 passes, and thought everything had run correctly - though I was
puzzled that make was reporting an error.

What happens is that in gdb/testsuite/Makefile, in the check-single
rule, we first run DejaGnu, then run the dg-add-core-file-count.sh
script, and finally, we use sed to extract the results from the
gdb.sum file.

In my case, with the invalid test name, DejaGnu fails, but the
following steps are still run, the final result, the 115 passes, is
then extracted from the pre-existing gdb.sum file.

If I use 'make -jN' then the 'check-parallel' rule, rather than the
'check-single' rule is used.  In this case the behaviour is slightly
different, the tail end of the output now looks like this:

  No matching tests found.

  make[4]: Leaving directory '/tmp/build/gdb/testsuite'
  find: ‘outputs’: No such file or directory
  Usage: ../../../src/gdb/testsuite/../../contrib/dg-extract-results.py [-t tool] [-l variant-list] [-L] log-or-sum-file ...

      tool           The tool (e.g. g++, libffi) for which to create a
                     new test summary file.  If not specified then output
                     is created for all tools.
      variant-list   One or more test variant names.  If the list is
                     not specified then one is constructed from all
                     variants in the files for <tool>.
      sum-file       A test summary file with the format of those
                     created by runtest from DejaGnu.
      If -L is used, merge *.log files instead of *.sum.  In this
      mode the exact order of lines may not be preserved, just different
      Running *.exp chunks should be in correct order.
  find: ‘outputs’: No such file or directory
  Usage: ../../../src/gdb/testsuite/../../contrib/dg-extract-results.py [-t tool] [-l variant-list] [-L] log-or-sum-file ...

      tool           The tool (e.g. g++, libffi) for which to create a
                     new test summary file.  If not specified then output
                     is created for all tools.
      variant-list   One or more test variant names.  If the list is
                     not specified then one is constructed from all
                     variants in the files for <tool>.
      sum-file       A test summary file with the format of those
                     created by runtest from DejaGnu.
      If -L is used, merge *.log files instead of *.sum.  In this
      mode the exact order of lines may not be preserved, just different
      Running *.exp chunks should be in correct order.
  make[3]: Leaving directory '/tmp/build/gdb/testsuite'
  make[2]: Leaving directory '/tmp/build/gdb/testsuite'
  make[1]: Leaving directory '/tmp/build/gdb'

Rather than DejaGnu failing, we now get a nice 'No matching tests
found' message, followed by some other noise.  This other noise is
first `find` failing, followed by the dg-extract-results.py script
failing.

What happens here is that, in the check-parallel rule, the outputs
directory is deleted before DejaGnu is invoked.  Then we try to run
all the tests, and finally we use find and dg-extract-results.py to
combine all the separate .sun and .log files together.  However, if
there are no tests run then the outputs/ directory is never created,
so the find command and consequently the dg-extract-results.py script,
fail.

This commit aims to fix the following issues:

 (1) For check-single and check-parallel rules, don't run any of the
 post-processing steps if DejaGnu failed to run.  This will avoid all
 the noise after the initial failure of DejaGnu,

 (2) For check-single ensure that we don't accidentally report
 previous results, this is related to the above, but is worth calling
 out as a separate point, and

 (3) For check-single, print the 'No matching tests found' message
 just like we do for a parallel test run.  This makes the parallel and
 non-parallel testing behaviour more similar, and I think is clearer
 than the current 'Illegal Argument' error message.

Points (1) and (2) will be handled by moving the post processing steps
inside an if block within the recipe.  For check-single I propose
deleting the gdb.sum and gdb.log files before running DejaGnu, this is
similar (I think) to how we delete the outputs/ directory in the
check-parallel rule.

For point (3) I plan to split the check-single rule in two, the
existing check-single will be renamed do-check-single, then a new
check-single rule will be added.  The new check-single rule can either
depend on the new do-check-single, or will ensure the 'No matching
tests found' message is printed when appropriate.
2022-10-02 17:43:55 +01:00
bfd Automatic date update in version.in 2022-10-02 00:00:18 +00:00
binutils objcopy: avoid "shadowing" of remove() function name 2022-09-30 10:55:02 +02:00
config egrep in binutils 2022-09-28 13:37:31 +09:30
contrib
cpu
elfcpp
etc
gas RISC-V: Relax "fmv.[sdq]" requirements 2022-09-30 15:10:27 +00:00
gdb gdb/testsuite: handle invalid .exp names passed in TESTS 2022-10-02 17:43:55 +01:00
gdbserver Renenerate {gdb,gdbserver}/configure 2022-09-28 13:06:06 +01:00
gdbsupport gdbsupport: move fileio_errno_to_host to fileio.{h,cc} and rename 2022-09-21 14:11:03 -04:00
gnulib
gold egrep in binutils 2022-09-28 13:37:31 +09:30
gprof Add -B to the help output from gprof, and add suitable documentation. 2022-09-29 13:12:37 +01:00
gprofng gprofng: fix cppcheck warnings 2022-09-29 22:00:02 -07:00
include LoongArch: Update ELF e_flags handling according to specification. 2022-09-30 14:00:47 +08:00
intl egrep in binutils 2022-09-28 13:37:31 +09:30
ld RISC-V: re-arrange opcode table for consistent alias handling 2022-09-30 10:19:00 +02:00
libbacktrace
libctf libctf: Add ZSTD_LIBS to LIBS so that ac_cv_libctf_bfd_elf can be true 2022-09-26 20:41:42 -07:00
libdecnumber
libiberty
opcodes RISC-V: Relax "fmv.[sdq]" requirements 2022-09-30 15:10:27 +00:00
readline
sim sim: Link ZSTD_LIBS 2022-09-27 11:42:32 -07:00
texinfo
zlib
.cvsignore
.editorconfig
.gitattributes
.gitignore
ar-lib
ChangeLog Maintainer mode: wrong gettext version? 2022-09-08 10:03:04 +01:00
compile
config-ml.in
config.guess
config.rpath
config.sub
configure binutils, gdb: support zstd compressed debug sections 2022-09-26 19:50:13 -07:00
configure.ac binutils, gdb: support zstd compressed debug sections 2022-09-26 19:50:13 -07:00
COPYING
COPYING3
COPYING3.LIB
COPYING.LIB
COPYING.LIBGLOSS
COPYING.NEWLIB
depcomp
djunpack.bat
install-sh
libtool.m4
lt~obsolete.m4
ltgcc.m4
ltmain.sh
ltoptions.m4
ltsugar.m4
ltversion.m4
MAINTAINERS
Makefile.def
Makefile.in
Makefile.tpl
makefile.vms
missing
mkdep
mkinstalldirs
move-if-change
multilib.am
README
README-maintainer-mode Maintainer mode: wrong gettext version? 2022-09-08 10:03:04 +01:00
setup.com
src-release.sh
symlink-tree
test-driver
ylwrap

		   README for GNU development tools

This directory contains various GNU compilers, assemblers, linkers, 
debuggers, etc., plus their support routines, definitions, and documentation.

If you are receiving this as part of a GDB release, see the file gdb/README.
If with a binutils release, see binutils/README;  if with a libg++ release,
see libg++/README, etc.  That'll give you info about this
package -- supported targets, how to use it, how to report bugs, etc.

It is now possible to automatically configure and build a variety of
tools with one command.  To build all of the tools contained herein,
run the ``configure'' script here, e.g.:

	./configure 
	make

To install them (by default in /usr/local/bin, /usr/local/lib, etc),
then do:
	make install

(If the configure script can't determine your type of computer, give it
the name as an argument, for instance ``./configure sun4''.  You can
use the script ``config.sub'' to test whether a name is recognized; if
it is, config.sub translates it to a triplet specifying CPU, vendor,
and OS.)

If you have more than one compiler on your system, it is often best to
explicitly set CC in the environment before running configure, and to
also set CC when running make.  For example (assuming sh/bash/ksh):

	CC=gcc ./configure
	make

A similar example using csh:

	setenv CC gcc
	./configure
	make

Much of the code and documentation enclosed is copyright by
the Free Software Foundation, Inc.  See the file COPYING or
COPYING.LIB in the various directories, for a description of the
GNU General Public License terms under which you can copy the files.

REPORTING BUGS: Again, see gdb/README, binutils/README, etc., for info
on where and how to report problems.