Commit Graph

28 Commits

Author SHA1 Message Date
Richard Levitte
b73c5e054a testutil: add the possibility to set the current test title
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3445)
2017-05-11 20:40:23 +02:00
Pauli
03d8e9cb43 Add test_test tests for bignums.
Add relative tests for bignums.

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3405)
2017-05-09 21:30:29 +02:00
Pauli
dc352c1937 Add BN support to the test infrastructure.
This includes support for:

- comparisions between pairs of BIGNUMs
- comparisions between BIGNUMs and zero
- equality comparison between BIGNUMs and one
- equality comparisons between BIGNUMs and constants
- parity checks for BIGNUMs

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3405)
2017-05-09 21:30:29 +02:00
Richard Levitte
68e49bf223 testutil: Add OpenSSL error stack printing wrapper TEST_openssl_errors
Also added a internal error printing callback to be used both with
ERR_print_errors_cb() and with CRYPTO_mem_leaks_cb

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3345)
2017-04-28 15:59:46 +02:00
Richard Levitte
c5657cb710 testutil: make subtest_level() internal
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3345)
2017-04-28 15:59:46 +02:00
Richard Levitte
579d0fabcd testutil: Move printing function declarations to "internal" header
These functions aren't meant to be used directly by the test programs,
reflect that by making the declarations a little harder to reach, but
still available enough if there's a need to override them.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3345)
2017-04-28 15:59:46 +02:00
Richard Levitte
208d721a00 TAPify testutil
With the perl test framework comes the output format TAP
(Test Anything Protocol, see http://testanything.org/) with
extra extension for subtests.  This change extends that same
output format to any test program using testutil.

In this implementation, each test program is seen as a full test that
can be used as a subtest.  The perl framework passes on the subtest
level to the test programs with the environment variable
HARNESS_OSSL_LEVEL.  Furthermore, and series of tests added with
ADD_ALL_TESTS is regarded as another subtest level.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3296)
2017-04-25 15:43:04 +02:00
Richard Levitte
f044cd05a1 Avoid using BIO streams in bioprinttest.c
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3243)
2017-04-24 18:09:01 +02:00
Richard Levitte
4db40c94c3 Refactor the test framework testutil
It's now built as a static library, and greatly simplified for test
programs, which no longer need to include test_main_custom.h or
test_main.h and link with the corresponding object files.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3243)
2017-04-24 18:09:01 +02:00
Rich Salz
adcd8e37db Convert more tests
ct_test,evp_extra_test,wpackettest,packettest
Add strncmp TEST wrappers
And make some style/consistency fixes to ct_test
Silence travis; gcc bug?

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3234)
2017-04-18 16:33:15 -04:00
Pauli
d063add7cb Guarantee single argument evaluation for test macros.
Add test case that checks some of them.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3208)
2017-04-13 08:33:12 -04:00
Pauli
298f40e1f1 Make test marcos for true/false checks reliable for all integral types.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3154)
2017-04-10 19:44:02 -04:00
Pauli
2fae041d6c Test infrastructure additions.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3011)
2017-03-29 08:51:43 +10:00
Bernd Edlinger
01b76c2c5d Add -Wundef to --strict-warnings options.
Avoid a -Wundef warning in refcount.h
Avoid a -Wundef warning in o_str.c
Avoid a -Wundef warning in testutil.h
Include internal/cryptlib.h before openssl/stack.h
to avoid use of undefined symbol OPENSSL_API_COMPAT.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2712)
2017-02-24 09:21:59 +01:00
Emilia Kasper
e364c3b24e Add main() test methods to reduce test boilerplate.
Simple tests only need to implement register_tests().
Tests that need a custom main() should implement test_main(). This will
be wrapped in a main() that performs common setup/teardown (currently
crypto-mdebug).

Note that for normal development, enable-asan is usually
sufficient for detecting leaks, and more versatile.

enable-crypto-mdebug is stricter as it will also
insist that all static variables be freed. This is useful for debugging
library init/deinit; however, it also means that test_main() must free
everything it allocates.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-11-09 16:07:16 +01:00
Emilia Kasper
d836d71b2d Simplify tests part 2
1) Remove some unnecessary fixtures
2) Add EXECUTE_TEST_NO_TEARDOWN shorthand when a fixture exists but has
no teardown.
3) Fix return values in ct_test.c (introduced by an earlier refactoring,
oops)

Note that for parameterized tests, the index (test vector) usually holds all the
customization, and there should be no need for a separate test
fixture. The CTS test is an exception: it demonstrates how to combine
customization with parameterization.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-11-07 16:55:16 +01:00
Emilia Kasper
308b876da9 Don't create fixtures for simple tests
The test fixtures are (meant to be) useful for sharing common
setup. Don't bother when we don't have any setup/teardown.

This only addresses simple tests. Parameterized tests (ADD_ALL_TESTS)
will be made more user-friendly in a follow-up.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-11-04 15:05:37 +01:00
Emilia Kasper
ffd3d0ef34 TEST_check macro: don't end with semi
Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-11-02 11:34:25 +01:00
Emilia Kasper
d61f00780a Add TEST_check
Like OPENSSL_assert, but also prints the error stack before exiting.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-08-10 14:41:21 +02:00
Emilia Kasper
ce2cdac278 SSL test framework: port NPN and ALPN tests
Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-07-19 14:17:48 +02:00
Rich Salz
440e5d805f Copyright consolidation 02/10
Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-05-17 14:20:27 -04:00
Emilia Kasper
ababe86b96 testutil: return 1 on success
Require that test methods return 1 on success (not 0). This is more
customary for OpenSSL.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-04-05 17:05:40 +02:00
Emilia Kasper
453dfd8d5e New SSL test framework
Currently, SSL tests are configured via command-line switches to
ssltest.c. This results in a lot of duplication between ssltest.c and
apps, and a complex setup. ssltest.c is also simply old and needs
maintenance.

Instead, we already have a way to configure SSL servers and clients, so
we leverage that. SSL tests can now be configured from a configuration
file. Test servers and clients are configured using the standard
ssl_conf module. Additional test settings are configured via a test
configuration.

Moreover, since the CONF language involves unnecessary boilerplate, the
test conf itself is generated from a shorter Perl syntax.

The generated testcase files are checked in to the repo to make
it easier to verify that the intended test cases are in fact run; and to
simplify debugging failures.

To demonstrate the approach, min/max protocol tests are converted to the
new format. This change also fixes MinProtocol and MaxProtocol
handling. It was previously requested that an SSL_CTX have both the
server and client flags set for these commands; this clearly can never work.

Guide to this PR:
 - test/ssl_test.c - test framework
 - test/ssl_test_ctx.* - test configuration structure
 - test/handshake_helper.* - new SSL test handshaking code
 - test/ssl-tests/ - test configurations
 - test/generate_ssl_tests.pl - script for generating CONF-style test
   configurations from perl inputs

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-04-05 13:44:46 +02:00
Rich Salz
349807608f Remove /* foo.c */ comments
This was done by the following
        find . -name '*.[ch]' | /tmp/pl
where /tmp/pl is the following three-line script:
        print unless $. == 1 && m@/\* .*\.[ch] \*/@;
        close ARGV if eof; # Close file to reset $.

And then some hand-editing of other files.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2016-01-26 16:40:43 -05:00
Matt Caswell
0f113f3ee4 Run util/openssl-format-source -v -c .
Reviewed-by: Tim Hudson <tjh@openssl.org>
2015-01-22 09:20:09 +00:00
Tim Hudson
1d97c84351 mark all block comments that need format preserving so that
indent will not alter them when reformatting comments

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
2014-12-30 22:10:26 +00:00
Mike Bland
5e3de8e609 test/testutil.c test registry functions.
These help standardize the structure of main() and result reporting.

Reviewed-by: Tim Hudson <tjh@openssl.org>
2014-07-19 19:24:35 +01:00
Mike Bland
3ead9f3798 Create test/testutil.h for unit test helper macros
Defines SETUP_TEST_FIXTURE and EXECUTE_TEST, and updates ssl/heartbeat_test.c
using these macros. SETUP_TEST_FIXTURE makes use of the new TEST_CASE_NAME
macro, defined to use __func__ or __FUNCTION__ on platforms that support those
symbols, or to use the file name and line number otherwise. This should fix
several reported build problems related to lack of C99 support.
2014-06-10 19:20:25 +01:00