Add stdout-nolog and ignore-nolog to AT_CHECK.

* lib/autotest/general.m4 (AT_DIFF_STDERR(stderr-nolog))
(AT_DIFF_STDERR(ignore-nolog), AT_DIFF_STDOUT(stdout-nolog))
(AT_DIFF_STDOUT(ignore-nolog)): New macros.
* tests/autotest.at (Logging): New test.
* doc/autoconf.texi (Writing Testsuites) <AT_CHECK>: Document the
new logging actions.
* NEWS: Likewise.
Reported by Ralf Wildenhues.

Signed-off-by: Eric Blake <ebb9@byu.net>
This commit is contained in:
Eric Blake 2009-04-13 07:10:32 -06:00
parent e8f34dca8c
commit c241057ad4
5 changed files with 83 additions and 4 deletions

View File

@ -1,5 +1,15 @@
2009-04-15 Eric Blake <ebb9@byu.net>
Add stdout-nolog and ignore-nolog to AT_CHECK.
* lib/autotest/general.m4 (AT_DIFF_STDERR(stderr-nolog))
(AT_DIFF_STDERR(ignore-nolog), AT_DIFF_STDOUT(stdout-nolog))
(AT_DIFF_STDOUT(ignore-nolog)): New macros.
* tests/autotest.at (Logging): New test.
* doc/autoconf.texi (Writing Testsuites) <AT_CHECK>: Document the
new logging actions.
* NEWS: Likewise.
Reported by Ralf Wildenhues.
Teach AT_CHECK about hard failures.
* lib/autotest/general.m4 (AT_INIT) <at_fn_check_skip>
<at_fn_check_status, at_fn_group_postprocess>: Handle hard

3
NEWS
View File

@ -13,7 +13,8 @@ GNU Autoconf NEWS - User visible changes.
** The macro AT_CHECK now understands the concept of hard failure. If
a test exits with an unexpected status 99, cleanup actions for the
test are inhibited and the test is treated as a failure regardless
of AT_XFAIL_IF.
of AT_XFAIL_IF. It also understands the new directives
ignore-nolog, stdout-nolog, and stderr-nolog.
** The autotest macro AT_CHECK_NOESCAPE is now documented.

View File

@ -22424,9 +22424,18 @@ arguments.
@table @samp
@item ignore
The content of the output is ignored, but still captured in the test
group log (if the test group later fails, the test group log is then
copied into the overall testsuite log). This is valid for both
@var{stdout} and @var{stderr}.
group log (if the testsuite is run with option @option{-v}, the test
group log is displayed as the test is run; if the test group later
fails, the test group log is also copied into the overall testsuite
log). This action is valid for both @var{stdout} and @var{stderr}.
@item ignore-nolog
The content of the output is ignored, and nothing is captured in the log
files. If @var{commands} are likely to produce binary output (including
long lines) or large amounts of output, then logging the output can make
it harder to locate details related to subsequent tests within the
group, and could potentially corrupt terminal display of a user running
@command{testsuite -v}.
@item stdout
For the @var{stdout} parameter, capture the content of standard output
@ -22441,6 +22450,14 @@ Like @samp{stdout}, except that it only works for the @var{stderr}
parameter, and the standard error capture file will be named
@file{stderr}.
@item stdout-nolog
@itemx stderr-nolog
Like @samp{stdout} or @samp{stderr}, except that the captured output is
not duplicated into the test group log. This action is particularly
useful for an intermediate check that produces large amounts of data,
which will be followed by another check that filters down to the
relevant data, as it makes it easier to locate details in the log.
@item expout
For the @var{stdout} parameter, compare standard output contents with
the previously created file @file{expout}, and list any differences in

View File

@ -2019,8 +2019,11 @@ dnl COMMANDS may contain parameter expansions; expand them at runtime.
# faster than using m4_case, and these are called very frequently.
m4_define([AT_DIFF_STDERR(stderr)],
[echo stderr:; tee stderr <"$at_stderr"])
m4_define([AT_DIFF_STDERR(stderr-nolog)],
[echo stderr captured; cp "$at_stderr" stderr])
m4_define([AT_DIFF_STDERR(ignore)],
[echo stderr:; cat "$at_stderr"])
m4_define([AT_DIFF_STDERR(ignore-nolog)])
m4_define([AT_DIFF_STDERR(experr)],
[$at_diff experr "$at_stderr" || at_failed=:])
m4_define([AT_DIFF_STDERR()],
@ -2028,8 +2031,11 @@ m4_define([AT_DIFF_STDERR()],
m4_define([AT_DIFF_STDOUT(stdout)],
[echo stdout:; tee stdout <"$at_stdout"])
m4_define([AT_DIFF_STDOUT(stdout-nolog)],
[echo stdout captured; cp "$at_stdout" stdout])
m4_define([AT_DIFF_STDOUT(ignore)],
[echo stdout:; cat "$at_stdout"])
m4_define([AT_DIFF_STDOUT(ignore-nolog)])
m4_define([AT_DIFF_STDOUT(expout)],
[$at_diff expout "$at_stdout" || at_failed=:])
m4_define([AT_DIFF_STDOUT()],

View File

@ -287,6 +287,51 @@ AT_CHECK_AT_TEST([errexit],
AT_CHECK([grep "1 .* inhibited subsequent" stderr], [], [ignore])],
[--errexit])
AT_CHECK_AT([Logging],
[[AT_INIT([artificial test suite])
dnl intentionally write failing tests, to see what gets logged
AT_SETUP([one])
AT_CHECK([echo magicstring01], [1], [ignore])
AT_CLEANUP
AT_SETUP([two])
AT_CHECK([echo magicstring02 >&2], [1], [], [ignore])
AT_CLEANUP
AT_SETUP([three])
AT_CHECK([echo magicstring03], [1], [ignore-nolog])
AT_CLEANUP
AT_SETUP([four])
AT_CHECK([echo magicstring04 >&2], [1], [], [ignore-nolog])
AT_CLEANUP
AT_SETUP([five])
AT_CHECK([echo magicstring05], [1], [stdout])
AT_CLEANUP
AT_SETUP([six])
AT_CHECK([echo magicstring06 >&2], [1], [], [stderr])
AT_CLEANUP
AT_SETUP([seven])
AT_CHECK([echo magicstring07], [1], [stdout-nolog])
AT_CLEANUP
AT_SETUP([eight])
AT_CHECK([echo magicstring08 >&2], [1], [], [stderr-nolog])
AT_CLEANUP
AT_SETUP([nine])
echo magicstring09 > expout
AT_CHECK([echo magicstring09], [1], [expout])
AT_CLEANUP
AT_SETUP([ten])
echo magicstring10 > experr
AT_CHECK([echo magicstring10 >&2], [1], [], [experr])
AT_CLEANUP
]], [], [1], [], [ignore], [],
[AT_CHECK([$CONFIG_SHELL ./micro-suite], [1], [ignore-nolog], [ignore-nolog])
AT_CHECK([grep '^magicstring' micro-suite.log], [],
[[magicstring01
magicstring02
magicstring05
magicstring06
]])])
AT_CHECK_AT_TEST([Cleanup],
[AT_CHECK([test ! -f cleanup.success && test ! -f cleanup.failure])
AT_CHECK_NOESCAPE([exit $value], [ignore], [$output],