openssl/test
Matt Caswell 4e297b7441 Make the rand_crng code OPENSSL_CTX aware
This is in preparation for moving this code inside the FIPS module.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9039)
2019-06-07 12:04:34 +01:00
..
certs Add test cases for SM2 cert verification 2019-03-30 08:31:25 +09:00
ct
d2i-tests
ocsp-tests Fix OCSP_basic_verify() cert chain construction in case bs->certs is NULL 2017-08-16 14:32:38 -04:00
ossl_shim Build: Change all _NO_INST to use attributes instead. 2019-01-22 12:35:39 +01:00
recipes Move digests to providers 2019-06-04 12:09:50 +10:00
smime-certs Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ssl-tests Add option to disable Extended Master Secret 2019-02-15 10:11:18 +00:00
testutil Really fail if we have a test failure 2019-06-03 13:27:11 +01:00
aborttest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
aesgcmtest.c FIPS AES_GCM IV gen changes 2019-03-07 07:30:04 +10:00
afalgtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asn1_decode_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asn1_encode_test.c fix truncation of integers on 32bit AIX 2019-03-11 14:45:18 +01:00
asn1_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asn1_string_table_test.c typo ANS1 -> ASN1 2018-12-11 20:57:31 +10:00
asn1_time_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asynciotest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
asynctest.c add an additional async notification communication method based on callback 2019-01-27 12:27:17 +00:00
bad_dtls_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
bftest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
bio_callback_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
bio_enc_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
bio_memleak_test.c Add test for the BIO_s_mem rdwr->rdonly->rdwr use-case 2019-04-16 10:50:30 +01:00
bioprinttest.c fix truncation of integers on 32bit AIX 2019-03-11 14:45:18 +01:00
bn_internal_test.c removed BN_clear NULL checks 2019-03-20 09:17:42 +10:00
bn_rand_range.h Test of uniformity of BN_rand_range output. 2019-05-29 09:54:29 +10:00
bntest.c Test of uniformity of BN_rand_range output. 2019-05-29 09:54:29 +10:00
bntests.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
build.info Move digests to providers 2019-06-04 12:09:50 +10:00
CAss.cnf
CAssdh.cnf
CAssdsa.cnf
CAssrsa.cnf
casttest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
CAtsa.cnf Added support for ESSCertIDv2 2017-05-03 09:04:23 +02:00
chacha_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
cipher_overhead_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
cipherbytes_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
cipherlist_test.c Ignore cipher suites when setting cipher list 2019-02-14 13:54:56 +00:00
ciphername_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
clienthellotest.c Correctly initialise PACKET to zero in the tests to avoid possible problems 2019-03-29 20:52:01 +10:00
cms-examples.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
cmsapitest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
conf_include_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
constant_time_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
context_internal_test.c Instead of global data store it in an OPENSSL_CTX 2019-05-02 22:42:09 +01:00
crltest.c Fix a memory leak with di2_X509_CRL reuse 2019-01-31 19:10:57 +01:00
ct_test.c fix truncation of integers on 32bit AIX 2019-03-11 14:45:18 +01:00
ctype_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
curve448_internal_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
d2i_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
danetest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
danetest.in Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
danetest.pem
destest.c add missing const 2018-12-09 22:02:48 -05:00
dhtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_data_ctr.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_data_hash.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_data_hmac.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_data.h Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_test.c Skip the no derivation functions when in FIPS mode because they are not 2019-05-09 20:21:09 +10:00
drbgtest.c Make the rand_crng code OPENSSL_CTX aware 2019-06-07 12:04:34 +01:00
drbgtest.h Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
dsa_no_digest_size_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
dsatest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
dtls_mtu_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
dtlstest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
dtlsv1listentest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ec_internal_test.c [test] unit test for field_inv function pointer in EC_METHOD 2019-02-17 21:02:36 +02:00
ecdsatest.c Check for negative return for signature size.Addresses Coverity 1442933 2019-03-04 13:39:09 +10:00
ecdsatest.h [test] modernize ecdsatest and extend ECDSA sign KATs 2019-02-26 17:59:51 +02:00
ecstresstest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
ectest.c Fix no-ec2m 2019-04-23 10:39:47 +01:00
enginetest.c Coverity fixes 2019-06-06 09:34:17 +10:00
errtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
evp_extra_test.c Fixed typo in code comment. 2019-06-07 10:47:19 +10:00
evp_kdf_test.c Added X963KDF API 2019-05-16 11:43:41 +10:00
evp_pkey_dparams_test.c fix timeout in 30-test_evp_pkey_dparam 2019-05-29 09:50:19 +10:00
evp_test.c Move digests to providers 2019-06-04 12:09:50 +10:00
evp_test.h Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
exdatatest.c Add CRYPTO_alloc_ex_data() 2019-02-16 00:29:20 +01:00
exptest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
fatalerrtest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
generate_buildtest.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
generate_ssl_tests.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
gmdifftest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
gosttest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
handshake_helper.c Fix end-point shared secret for DTLS/SCTP 2019-02-01 11:57:19 +00:00
handshake_helper.h Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
hmactest.c Cleaner disposal of ephemeral engine ids and names 2018-12-09 22:02:48 -05:00
ideatest.c add missing const in cast 2018-12-09 22:02:49 -05:00
igetest.c Deprecate AES_ige_encrypt() and AES_bi_ige_encrypt() 2019-04-12 14:22:41 +01:00
lhash_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
md2test.c Add a legacy provider and put MD2 in it 2019-04-09 10:24:43 +01:00
mdc2_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
mdc2test.c Move digests to providers 2019-06-04 12:09:50 +10:00
memleaktest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
modes_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ocspapitest.c OCSP: fix memory leak in OCSP_url_svcloc_new method. 2019-05-27 08:11:50 +10:00
ossl_test_endian.h Fix test builds. 2019-03-28 15:02:19 +10:00
P1ss.cnf
P2ss.cnf
p_test.c p_get_params() [test/p_test.c]: clear the OSSL_PARAM buffers for every test 2019-05-01 13:19:18 +02:00
packettest.c Correctly initialise PACKET to zero in the tests to avoid possible problems 2019-03-29 20:52:01 +10:00
params_api_test.c Params API: {utf8,octet}_ptr need to know the data size 2019-04-09 14:10:23 +02:00
params_conversion_test.c Params conversion tests. 2019-05-21 07:42:54 +10:00
params_test.c test/params_test.c : Adjust tests to check utf8_ptr sizes 2019-04-09 14:10:23 +02:00
pbelutest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
pemtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
pkcs7-1.pem
pkcs7.pem
pkey_meth_kdf_test.c Added new EVP/KDF API. 2019-02-13 12:11:49 +01:00
pkey_meth_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
pkits-test.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
poly1305_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
property_test.c Optional property query support. 2019-05-22 10:45:39 +10:00
provider_internal_test.c For provider tests, don't define a OPENSSL_NO_ macro 2019-04-04 13:39:33 +02:00
provider_internal_test.conf.in Add test for the provider configuration module 2019-04-03 11:42:48 +02:00
provider_test.c For provider tests, don't define a OPENSSL_NO_ macro 2019-04-04 13:39:33 +02:00
rc2test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rc4test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rc5test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rdrand_sanitytest.c Remove unnecessary trailing whitespace 2019-02-05 16:25:11 +01:00
README issue-8493: Fix for filenames with newlines using openssl dgst 2019-03-30 11:22:51 +10:00
README.external Remove unnecessary trailing whitespace 2019-02-05 16:25:11 +01:00
README.ssltest.md Session resume broken switching contexts 2017-10-04 10:21:08 +10:00
recordlentest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
rsa_complex.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rsa_mp_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rsa_sp800_56b_test.c removed BN_clear NULL checks 2019-03-20 09:17:42 +10:00
rsa_test.c FIPS 186-4 RSA Generation & Validation 2019-03-12 12:00:52 +00:00
run_tests.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
sanitytest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
secmemtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
serverinfo2.pem Add a SERVERINFOV2 format test file 2017-05-03 14:37:42 +01:00
serverinfo.pem
servername_test.c Correctly initialise PACKET to zero in the tests to avoid possible problems 2019-03-29 20:52:01 +10:00
session.pem Don't store the ticket nonce in the session 2018-06-07 10:58:35 +01:00
shibboleth.pfx
shlibloadtest.c Introduce a no-pinshared option 2019-01-04 13:19:39 +00:00
siphash_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
sm2_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
sm4_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
smcont.txt
sparse_array_test.c Make the sparse array code use ossl_uintmax_t as its index rather than size_t. 2019-03-06 13:50:54 +10:00
srptest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ssl_cert_table_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ssl_test_ctx_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
ssl_test_ctx_test.conf Implement Maximum Fragment Length TLS extension. 2017-11-05 17:46:48 +01:00
ssl_test_ctx.c Fix end-point shared secret for DTLS/SCTP 2019-02-01 11:57:19 +00:00
ssl_test_ctx.h Fix end-point shared secret for DTLS/SCTP 2019-02-01 11:57:19 +00:00
ssl_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
ssl_test.tmpl
sslapitest.c Write a test for receiving a KeyUpdate (update requested) while writing 2019-06-03 11:51:14 +01:00
sslbuffertest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
sslcorrupttest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
ssltest_old.c Ignore cipher suites when setting cipher list 2019-02-14 13:54:56 +00:00
ssltestlib.c Write a test for receiving a KeyUpdate (update requested) while writing 2019-06-03 11:51:14 +01:00
ssltestlib.h Write a test for receiving a KeyUpdate (update requested) while writing 2019-06-03 11:51:14 +01:00
Sssdsa.cnf
Sssrsa.cnf
stack_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
sysdefault.cnf Apply system_default configuration on SSL_CTX_new(). 2018-03-19 10:22:49 -04:00
sysdefaulttest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
test_test.c Test skip option. 2019-05-09 20:21:09 +10:00
test.cnf
testcrl.pem
testdsa.pem
testdsapub.pem
testec-p256.pem
testecpub-p256.pem
testp7.pem
testreq2.pem
testrsa.pem
testrsapub.pem
testsid.pem
testutil.h Test skip option. 2019-05-09 20:21:09 +10:00
testx509.pem
threadstest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
time_offset_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
tls13ccstest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
tls13encryptiontest.c Collapse ssl3_state_st (s3) into ssl_st 2019-04-29 17:26:09 +01:00
tls13secretstest.c Remove unnecessary trailing whitespace 2019-02-05 16:25:11 +01:00
uitest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
Uss.cnf
v3-cert1.pem
v3-cert2.pem
v3ext.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
v3nametest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
verify_extra_test.c Make X509_set_sm2_id consistent with other setters 2019-04-09 20:44:42 +08:00
versions.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
wpackettest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
x509_check_cert_pkey_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
x509_dup_cert_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
x509_internal_test.c Join the x509 and x509v3 directories 2019-05-29 09:32:50 +02:00
x509_time_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
x509aux.c constify *_dup() and *i2d_*() and related functions as far as possible, introducing DECLARE_ASN1_DUP_FUNCTION 2019-03-06 16:10:09 +00:00

How to add recipes
==================

For any test that you want to perform, you write a script located in
test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and
{name} is a unique name of your choice.

Please note that if a test involves a new testing executable, you will need to
do some additions in test/Makefile.  More on this later.


Naming conventions
=================

A test executable is named test/{name}test.c

A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two
digit number and {name} is a unique name of your choice.

The number {nn} is (somewhat loosely) grouped as follows:

00-04  sanity, internal and essential API tests
05-09  individual symmetric cipher algorithms
10-14  math (bignum)
15-19  individual asymmetric cipher algorithms
20-24  openssl commands (some otherwise not tested)
25-29  certificate forms, generation and verification
30-35  engine and evp
60-79  APIs
   70  PACKET layer
80-89  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
90-98  misc
99     most time consuming tests [such as test_fuzz]


A recipe that just runs a test executable
=========================================

A script that just runs a program looks like this:

    #! /usr/bin/perl

    use OpenSSL::Test::Simple;

    simple_test("test_{name}", "{name}test", "{name}");

{name} is the unique name you have chosen for your test.

The second argument to `simple_test' is the test executable, and `simple_test'
expects it to be located in test/

For documentation on OpenSSL::Test::Simple, do
`perldoc util/perl/OpenSSL/Test/Simple.pm'.


A recipe that runs a more complex test
======================================

For more complex tests, you will need to read up on Test::More and
OpenSSL::Test.  Test::More is normally preinstalled, do `man Test::More' for
documentation.  For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'.

A script to start from could be this:

    #! /usr/bin/perl

    use strict;
    use warnings;
    use OpenSSL::Test;

    setup("test_{name}");

    plan tests => 2;                # The number of tests being performed

    ok(test1, "test1");
    ok(test2, "test1");

    sub test1
    {
        # test feature 1
    }

    sub test2
    {
        # test feature 2
    }


Changes to test/build.info
==========================

Whenever a new test involves a new test executable you need to do the
following (at all times, replace {NAME} and {name} with the name of your
test):

* add {name} to the list of programs under PROGRAMS_NO_INST

* create a three line description of how to build the test, you will have
to modify the include paths and source files if you don't want to use the
basic test framework:

    SOURCE[{name}]={name}.c
    INCLUDE[{name}]=.. ../include
    DEPEND[{name}]=../libcrypto libtestutil.a

Generic form of C test executables
==================================

    #include "testutil.h"

    static int my_test(void)
    {
        int testresult = 0;                 /* Assume the test will fail    */
        int observed;

        observed = function();              /* Call the code under test     */
        if (!TEST_int_eq(observed, 2))      /* Check the result is correct  */
            goto end;                       /* Exit on failure - optional   */

        testresult = 1;                     /* Mark the test case a success */
    end:
        cleanup();                          /* Any cleanup you require      */
        return testresult;
    }

    int setup_tests(void)
    {
        ADD_TEST(my_test);                  /* Add each test separately     */
        return 1;                           /* Indicate success             */
    }

You should use the TEST_xxx macros provided by testutil.h to test all failure
conditions.  These macros produce an error message in a standard format if the
condition is not met (and nothing if the condition is met).  Additional
information can be presented with the TEST_info macro that takes a printf
format string and arguments.  TEST_error is useful for complicated conditions,
it also takes a printf format string and argument.  In all cases the TEST_xxx
macros are guaranteed to evaluate their arguments exactly once.  This means
that expressions with side effects are allowed as parameters.  Thus,

    if (!TEST_ptr(ptr = OPENSSL_malloc(..)))

works fine and can be used in place of:

    ptr = OPENSSL_malloc(..);
    if (!TEST_ptr(ptr))

The former produces a more meaningful message on failure than the latter.