The testsuite used slightly different logic in the code employed to
decide whether a test should be skipped due to a limitation of the
selected shell, and in the code running the test itself.
* tests/autotest.at (Syntax error): Adjust.
(parallel syntax error): Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Done via 'make update-copyright', since all files are effectively
modified and distributed this year via public version control.
* all files: Update copyright year.
Done via 'make update-copyright', since all files are effectively
modified and distributed this year via public version control.
* all files: Update copyright year.
Together with Linux's binfmt-misc feature, Wine can be used to test
cross-compiled programs as if they were native. However, the shell
will not perform the "magic" addition of the .exe extension after a
program name when searching for an executable. These simple patches
let the user work around this by specifying $EXEEXT in the AT_CHECK
and AT_TESTED argument. (More care is needed because of carriage
returns, but this is beyond the scope of this series).
* tests/autotest.at (AT_INIT): Expand contents of $at_tested.
(AT_TESTED): Quote each program that is passed to the function.
* lib/autotest/general.m4 (C unit tests): Add AT_TESTED invocation
and keyword.
* doc/autoconf.texi (Writing testsuites): Document usage of variables
in AT_TESTED.
* NEWS: Document change.
Signed-off-by: Paolo Bonzini <bonzini@gnu.org>
* lib/autotest/general.m4 (AT_DATA_UNQUOTED): New macro, paralleling
AT_DATA but not quoting the contents.
* doc/autoconf.texi (Writing Testsuites): Document it.
* tests/autotest.at (AT_DATA_UNQUOTED): Test it.
Signed-off-by: Paolo Bonzini <bonzini@gnu.org>
* lib/autotest/general.m4 (AT_DATA): Special case an empty data
file, since zsh botches empty here-docs.
* doc/autoconf.texi (Writing Testsuites) <AT_DATA>: Document that
this allows empty contents.
* tests/autotest.at (AT_DATA): New test.
Reported by Ralf Wildenhues.
Signed-off-by: Eric Blake <eblake@redhat.com>
* doc/autoconf.texi (Writing Testsuites): Document that COMMANDS
is run in a subshell, but RUN-IF-FAIL and RUN-IF-PASS are not.
* tests/autotest.at (AT@&t@_CHECK execution environment): New
test.
* NEWS: Update.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* lib/autotest/general.m4 (AT_INIT): Remove definition of
AT_groups_all. Initialize at_groups from at_help_all, with
newlines instead of spaces separating test groups numbers.
Adjust all code to newlines.
* NEWS: Update.
* tests/autotest.at (Huge testsuite): New test.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* lib/autotest/general.m4 (AT_INIT) <at_fn_group_postprocess>:
Exit after an unexpected passing test if $at_errexit.
* tests/autotest.at (errexit): Also try tests that xpass, skip,
xfail, or fail hard.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* lib/autotest/general.m4 (AT_INIT) <at_fn_group_postprocess>:
Don't cleanup the group directory when a test unexpectedly passes.
* tests/autotest.at (Cleanup): Check that an unexpected pass leaves
the test group directory intact.
Signed-off-by: Peter Rosin <peda@lysator.liu.se>
* tests/autotest.at (colored test results): Use exit not
Exit. Fixes test failure with dash 0.5.4.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* tests/autotest.at (parallel args but non-working mkfifo):
New test, to expose the failure v2.66-23-g991183c avoided.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* doc/autoconf.texi (Writing Testsuites) <AT_BANNER>: Update
description.
* lib/autotest/general.m4 (AT_INIT) <at_fn_banner>: Set banner
to single space, not empty line, once printed. For empty
banners, print a single empty line to separate them from a
previous test group category.
* tests/autotest.at (Banners): Insert another test group; adjust
tests accordingly. Extend test to cover semantic change.
* NEWS: Update.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* lib/autotest/general.m4 (_AT_ARG_OPTION): Fix translation of
hyphens to underscores when turning option names to variables.
Avoid macro name concatenation garbage with trailing `dnl'.
(AT_ARG_OPTION, AT_ARG_OPTION_ARG): Overhaul macro description.
The OPTIONS are space-separated, not comma-separated. The
negative form of AT_ARG_OPTION is prefixed with `--no-'.
* tests/autotest.at (AT@&t@_ARG_OPTION, AT@&t@_ARG_OPTION_ARG):
New tests.
* NEWS: Update.
* doc/autoconf.texi (Writing Testsuites): Document AT_ARG_OPTION
and AT_ARG_OPTION_ARG.
(testsuite Invocation): Call the thingies passed to the
testsuite options, not arguments. Note that the testsuite
author may add further package-specific options.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* lib/autotest/general.m4 (HELP_TUNING_BEGIN): New diversion.
(HELP_TUNING, HELP_OTHER, HELP_END): Bump diversion numbers.
(AT_INIT): Accept
--color and --color=never|auto|always. If desired, colorize
test results and testsuite summary on standard output.
[HELP_TUNING]: Divert content instead to ...
[HELP_TUNING_BEGIN]: ... this diversion, m4_wrapped until the
end, when we know whether AT_COLOR_TESTS has been specified.
(AT_COLOR_TESTS): New macro, set the default for color to auto.
* doc/autoconf.texi (Writing Testsuites): Document it.
(testsuite Invocation): Document --color* options.
* tests/local.at: Call AT_COLOR_TESTS for Autoconf's testsuite.
* tests/autotest.at (color test results): New test, mirroring
color.test from Automake.
* NEWS: Update.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* lib/autotest/general.m4 (AT_INIT): New variable $at_recheck.
Escape hyphen in $at_dir early. Accept command line switch
--recheck. Set $at_suite_log early, based on --directory
switch; with --recheck, include the list of FAILed and XPASSed
tests from old testsuite.log file in $at_groups. Document
--recheck in --help output.
* tests/autotest.at (recheck): New test.
* doc/autoconf.texi (testsuite Invocation): Document --recheck.
* NEWS: Update.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* tests/autotest.at (AT_CHECK_AT_PREP): Create a default
atconfig file in the directory of the testsuite.
(AT_CHECK_AT_TITLE): Also check that `./micro-suite -v' output
does not contain empty $at_srcdir expansion.
(srcdir propagation): Remove the atconfig file generated by
AT_CHECK_AT_PREP. Check each suite invocation for $at_srcdir
expansion.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* lib/autotest/general.m4 (AT_CLEANUP): Actually print test
title in verbose output. Fixes AUTOCONF-2.57-101-gc102ed8
regression.
* tests/autotest.at (AT_CHECK_AT_TITLE): Amend macro to check
for test title in -v output.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* tests/autotest.at (Erlang Eunit unit tests): Use "no" as
value-if-not-found for Erlang tools.
* tests/erlang.at: Likewise. Also, use AS_EXIT instead of plain
exit.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
* tests/autotest.at (Hard fail): Check more restrictively for
passed tests, so that hostnames recorded in the log file do not
wrongly match.
Report by Ludovic Courtès.
Signed-off-by: Ralf Wildenhues <Ralf.Wildenhues@gmx.de>
These are lightweight versions of AT_CHECK that automatically
add the equivalent of ! in front of the command and change a
failure exit status to 77 resp. 99. They expand to just
two lines of shell code at the expense of not supporting
tracing (but then so does AT_XFAIL_IF).
2009-07-12 Paolo Bonzini <bonzini@gnu.org>
* NEWS: Mention AT_SKIP_IF and AT_FAIL_IF.
* doc/autoconf.texi (Autotest): Document them.
* lib/autotest/general.m4 (_AT_LINE_ESCAPED, AT_SKIP_IF,
AT_FAIL_IF, _AT_CHECK_EXIT): New.
(AT_CHECK): Use _AT_LINE_ESCAPED.
* tests/autotest.st: Add tests for AT_SKIP_IF and AT_FAIL_IF.
Use AT_SKIP_IF.
* tests/local.st: Use AT_SKIP_IF.
* tests/autotest.at (AT_SKIP_PARALLEL_TESTS): Skip test if set -m
is not supported.
Reported by Ralf Wildenhues.
Signed-off-by: Eric Blake <ebb9@byu.net>
* lib/autotest/general.m4 (AT_INIT) <AT_JOB_FIFO_FD>: Avoid <>;
instead open write descriptor in each group and read descriptor in
main driver.
* tests/autotest.at (AT_SKIP_PARALLEL_TESTS): Relax condition.
Signed-off-by: Eric Blake <ebb9@byu.net>
* lib/autotest/general.m4 (AT_INIT) <driver loop>: Avoid syntax
errors on shells that don't recognize <>.
* tests/autotest.at (AT_SKIP_PARALLEL_TESTS): Also skip parallel
tests for this reason. Skip based on the shell to be tested,
not the shell driving the testsuite.
(parallel syntax error): Rearrange similar to previous patch.
(parallel test execution): Defer skip until after serial tests.
Signed-off-by: Eric Blake <ebb9@byu.net>
