When test cases were added with ADD_ALL_TESTS_NOSUBTEST(), all those
iteration verdicts were summarized as if it was one single case. This
modification gets each iteration verdict displayed separately instead.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/14498)
Tests randomisation wasn't reliably printing the seed used on failure in the CIs.
Fixes: #13572
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13573)
Fixes#8322
The leak-checking (and backtrace option, on some platforms) provided
by crypto-mdebug and crypto-mdebug-backtrace have been mostly neutered;
only the "make malloc fail" capability remains. OpenSSL recommends using
the compiler's leak-detection instead.
The OPENSSL_DEBUG_MEMORY environment variable is no longer used.
CRYPTO_mem_ctrl(), CRYPTO_set_mem_debug(), CRYPTO_mem_leaks(),
CRYPTO_mem_leaks_fp() and CRYPTO_mem_leaks_cb() return a failure code.
CRYPTO_mem_debug_{malloc,realloc,free}() have been removed. All of the
above are now deprecated.
Merge (now really small) mem_dbg.c into mem.c
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10572)
Actually supply a chain and then test:
1) A successful check of both the ee and chain certs
2) A failure to check the ee cert
3) A failure to check a chain cert
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/9442)
Implement the GNU C library's random(3) pseudorandom number generator.
The algorithm is described: https://www.mscs.dal.ca/~selinger/random/
The rationale is to make the tests repeatable across differing platforms with
different underlying implementations of the random(3) library call.
More specifically: when executing tests with random ordering.
[extended tests]
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9463)
Commit c5f7a99645 broke the test framework such that some tests might
fail, but the test framework still gives a PASS result overall.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9056)
From a Unix point of view, some other platform families have certain
quirks. Windows command prompt doesn't expand globs into actual file
names, so we must do this. VMS has some oddity with argv pointer size
that can cause crashes if you're not careful (by copying it to a less
surprising pointer size array).
The fixups already exist and are used in the apps/ code. However, the
testutil code started using the opt routines from apps/ without
including the non-Unix fixups. This change fixes that.
For VMS' sake, libtestutil gets an app_malloc() shim, to avoid sucking
in all of apps/apps.c.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8381)
(introduced by commit 9186016582, which added -Wstrict-prototypes)
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/6555)
It's argued that /WX allows to keep better focus on new code, which
motivates its comeback...
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4721)
Apart from ssltest_old.c, the test suite relied on e_os.h for the
OSSL_NELEM macro and nothing else.
The ssltest_old.c also requires EXIT and some socket macros.
Create a new header to define the OSSL_NELEM macro and use that instead.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4186)
that needed test_main now works using the same infrastructure as tests that used
register_tests.
This meant:
* renaming register_tests to setup_tests and giving it a success/failure return.
* renaming the init_test function to setup_test_framework.
* renaming the finish_test function to pulldown_test_framework.
* adding a user provided global_init function that runs before the test frame
work is initialised. It returns a failure indication that stops the stest.
* adding helper functions that permit tests to access their command line args.
* spliting the BIO initialisation and finalisation out from the test setup and
teardown.
* hiding some of the now test internal functions.
* fix the comments in testutil.h
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3953)
and bignums. These have been refactored into their own file, along with
their error displays. The formatting follows the output format used
on error, except that bignums of sixty four bits or less are displayed
in a more compact one line form.
Added a TEST_note function for producing output without file and line
information.
Update the three tests that call BN_print so they use the new test
infrastructure instead.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3655)
In some cases, testutil outputs subtests like this:
1..6 # Subtest: progname
The standard set by Test::More (because there really is no actual
standard yet) gives this display:
# Subtest: progname
1..6
Until the standard is actually agreed upon, let's do it like
Test::More.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3446)
Also added a internal error printing callback to be used both with
ERR_print_errors_cb() and with CRYPTO_mem_leaks_cb
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3345)
These functions aren't meant to be used directly by the test programs,
reflect that by making the declarations a little harder to reach, but
still available enough if there's a need to override them.
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3345)
This includes better signals of skips and subtests according to TAP 12,
and flushing stdout and stderr at the end of every test function to
make sure we get the output in good order.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3309)
With the perl test framework comes the output format TAP
(Test Anything Protocol, see http://testanything.org/) with
extra extension for subtests. This change extends that same
output format to any test program using testutil.
In this implementation, each test program is seen as a full test that
can be used as a subtest. The perl framework passes on the subtest
level to the test programs with the environment variable
HARNESS_OSSL_LEVEL. Furthermore, and series of tests added with
ADD_ALL_TESTS is regarded as another subtest level.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3296)
It's now built as a static library, and greatly simplified for test
programs, which no longer need to include test_main_custom.h or
test_main.h and link with the corresponding object files.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3243)
