openssl/test
Richard Levitte e31eda006f TEST: Add test to exercise OPENSSL_CTX_set0_default()
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12228)
2020-06-28 10:55:51 +02:00
..
certs Add support for unusal 'othername' subjectAltNames 2020-04-25 18:52:30 +03:00
ct Use .cnf for config files, not .conf 2020-03-06 18:25:13 +01:00
d2i-tests
ocsp-tests
ossl_shim Fix two additional instances of the old EVP_MAC_CTX_ functions being used. 2020-06-13 09:18:17 +10:00
recipes Reduce the security bits for MD5 and SHA1 based signatures in TLS 2020-06-27 08:41:40 +02:00
smime-certs
ssl-tests Reduce the security bits for MD5 and SHA1 based signatures in TLS 2020-06-27 08:41:40 +02:00
testutil Update copyright year 2020-06-25 14:13:12 +01:00
aborttest.c
acvp_test.c Make EVP_PKEY_CTX_[get|set]_group_name work for DH too 2020-06-19 10:19:31 +01:00
acvp_test.inc Add ACVP fips module tests 2020-06-17 11:33:16 +10:00
aesgcmtest.c Update copyright year 2020-05-15 14:09:49 +01:00
afalgtest.c
asn1_decode_test.c
asn1_dsa_internal_test.c
asn1_encode_test.c
asn1_internal_test.c
asn1_string_table_test.c
asn1_time_test.c Update copyright year 2020-04-23 13:55:52 +01:00
asynciotest.c Update copyright year 2020-04-23 13:55:52 +01:00
asynctest.c
bad_dtls_test.c The EVP_MAC functions have been renamed for consistency. The EVP_MAC_CTX_* 2020-06-11 11:16:37 +10:00
bftest.c Update copyright year 2020-04-23 13:55:52 +01:00
bio_callback_test.c
bio_enc_test.c
bio_memleak_test.c Update copyright year 2020-05-15 14:09:49 +01:00
bio_prefix_text.c
bioprinttest.c TEST: Adjust test/bioprinttest.c to behave like the testutil routines 2020-06-06 19:18:30 +02:00
bn_internal_test.c Update copyright year 2020-05-15 14:09:49 +01:00
bn_rand_range.h
bntest.c
bntests.pl
build.info Add a test for d2i_AutoPrivateKey_ex with a non-default libctx 2020-06-19 10:34:58 +01:00
ca-and-certs.cnf Cleanup cert config files for tests 2020-06-03 09:56:56 +02:00
casttest.c Update copyright year 2020-04-23 13:55:52 +01:00
CAtsa.cnf
chacha_internal_test.c
cipher_overhead_test.c
cipherbytes_test.c Update copyright year 2020-05-15 14:09:49 +01:00
cipherlist_test.c Update copyright year 2020-05-15 14:09:49 +01:00
ciphername_test.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
clienthellotest.c Update copyright year 2020-04-23 13:55:52 +01:00
cmactest.c Add a CMAC test 2020-06-10 12:58:26 +01:00
cmp_asn_test.c
cmp_client_test.c Improve ossl_cmp_msg_check_received() and rename to ossl_cmp_msg_check_update() 2020-06-13 15:13:21 +02:00
cmp_ctx_test.c Rename OSSL_CMP_CTX_set1_clCert() to OSSL_CMP_CTX_set1_cert() 2020-05-13 19:42:00 +02:00
cmp_hdr_test.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
cmp_msg_test.c Rename OSSL_CMP_CTX_set1_clCert() to OSSL_CMP_CTX_set1_cert() 2020-05-13 19:42:00 +02:00
cmp_protect_test.c Rename OSSL_CMP_CTX_set1_clCert() to OSSL_CMP_CTX_set1_cert() 2020-05-13 19:42:00 +02:00
cmp_server_test.c Chunk 8 of CMP contribution to OpenSSL: CMP server and cmp_mock_srv.c for testing 2020-03-10 16:09:44 +01:00
cmp_status_test.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
cmp_testlib.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
cmp_testlib.h Update copyright year 2020-04-23 13:55:52 +01:00
cmp_vfy_test.c Disable tests in cmp_vfy_test.c that make no sense if FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION 2020-06-22 16:41:24 +02:00
cms-examples.pl
cmsapitest.c Amend references to "OpenSSL license" 2020-04-29 15:27:22 +02:00
conf_include_test.c
confdump.c Update copyright year 2020-05-15 14:09:49 +01:00
constant_time_test.c
context_internal_test.c
crltest.c Update copyright year 2020-05-15 14:09:49 +01:00
ct_test.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
ctype_internal_test.c
curve448_internal_test.c Update copyright year 2020-04-23 13:55:52 +01:00
d2i_test.c Update copyright year 2020-04-23 13:55:52 +01:00
danetest.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
danetest.in
danetest.pem
default-and-fips.cnf Update some nits around the FIPS module 2020-04-24 13:19:16 +02:00
default-and-legacy.cnf
default.cnf
destest.c Update copyright year 2020-04-23 13:55:52 +01:00
dhtest.c Update copyright year 2020-04-23 13:55:52 +01:00
drbg_cavs_data_ctr.c
drbg_cavs_data_hash.c
drbg_cavs_data_hmac.c
drbg_cavs_data.h
drbg_cavs_test.c Update copyright year 2020-05-15 14:09:49 +01:00
drbg_extra_test.c coverity 1462562 Dereference before null check 2020-04-30 20:21:32 +10:00
drbg_extra_test.h Amend references to "OpenSSL license" 2020-04-29 15:27:22 +02:00
drbgtest.c update drbgtest to the provider model 2020-06-24 20:05:42 +10:00
drbgtest.h
dsa_no_digest_size_test.c Update copyright year 2020-04-23 13:55:52 +01:00
dsatest.c Make EVP_PKEY_CTX_[get|set]_group_name work for DH too 2020-06-19 10:19:31 +01:00
dtls_mtu_test.c Update copyright year 2020-05-15 14:09:49 +01:00
dtlstest.c Update copyright year 2020-04-23 13:55:52 +01:00
dtlsv1listentest.c
ec_internal_test.c Update copyright year 2020-04-23 13:55:52 +01:00
ecdsatest.c Rename FIPS_MODE to FIPS_MODULE 2020-04-28 15:37:37 +02:00
ecdsatest.h
ecstresstest.c
ectest.c Move EC_METHOD to internal-only 2020-06-02 11:17:24 +03:00
enginetest.c
errtest.c Update copyright year 2020-06-04 14:33:57 +01:00
evp_extra_test2.c TEST: Add test to exercise OPENSSL_CTX_set0_default() 2020-06-28 10:55:51 +02:00
evp_extra_test.c test: add test for generation of random data in chunks. 2020-06-24 20:07:46 +10:00
evp_fetch_prov_test.c Update copyright year 2020-04-23 13:55:52 +01:00
evp_kdf_test.c Update copyright year 2020-06-25 14:13:12 +01:00
evp_pkey_dparams_test.c
evp_pkey_provided_test.c Make EVP_PKEY_CTX_[get|set]_group_name work for DH too 2020-06-19 10:19:31 +01:00
evp_test.c test: update EVP tests to include DRBG testing 2020-06-24 20:07:46 +10:00
evp_test.h
exdatatest.c Update copyright year 2020-06-04 14:33:57 +01:00
exptest.c
fatalerrtest.c Update copyright year 2020-04-23 13:55:52 +01:00
ffc_internal_test.c Add ACVP fips module tests 2020-06-17 11:33:16 +10:00
filterprov.c Make the naming scheme for dispatched functions more consistent 2020-06-24 22:01:22 +02:00
fips.cnf Update some nits around the FIPS module 2020-04-24 13:19:16 +02:00
generate_buildtest.pl
generate_ssl_tests.pl Update copyright year 2020-04-23 13:55:52 +01:00
gmdifftest.c
gosttest.c Use a non-default libctx in sslapitest 2020-04-19 14:40:55 +01:00
handshake_helper.c Silence gcc false positive warning on alpn_protos_len in test/handshake_helper.c 2020-06-10 10:43:48 +02:00
handshake_helper.h
hexstr_test.c Fix ERR_print_errors so that it matches the documented format in doc/man3/ERR_error_string.pod 2020-05-26 12:44:36 +10:00
hmactest.c Update copyright year 2020-04-23 13:55:52 +01:00
http_test.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
ideatest.c Update copyright year 2020-04-23 13:55:52 +01:00
igetest.c
insta_ca.cert.pem Chunk 11 of CMP contribution to OpenSSL: CMP command-line interface 2020-05-13 19:42:00 +02:00
insta.priv.pem Chunk 11 of CMP contribution to OpenSSL: CMP command-line interface 2020-05-13 19:42:00 +02:00
keymgmt_internal_test.c Update copyright year 2020-04-23 13:55:52 +01:00
legacy.cnf
lhash_test.c
mdc2_internal_test.c Update copyright year 2020-04-23 13:55:52 +01:00
mdc2test.c Update copyright year 2020-04-23 13:55:52 +01:00
memleaktest.c
modes_internal_test.c Update copyright year 2020-04-23 13:55:52 +01:00
namemap_internal_test.c namemap: change ossl_namemap_empty() to do what the documentation says. 2020-06-21 16:49:51 +10:00
ocspapitest.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
ossl_test_endian.h
p_test.c Make the naming scheme for dispatched functions more consistent 2020-06-24 22:01:22 +02:00
packettest.c
param_build_test.c params: add OSSL_PARAM helpers for time_t. 2020-06-24 20:05:41 +10:00
params_api_test.c params: add OSSL_PARAM helpers for time_t. 2020-06-24 20:05:41 +10:00
params_conversion_test.c Update copyright year 2020-04-23 13:55:52 +01:00
params_test.c Update copyright year 2020-04-23 13:55:52 +01:00
pbelutest.c
pemtest.c
pkcs7-1.pem
pkcs7.pem
pkey_meth_kdf_test.c
pkey_meth_test.c
pkits-test.pl
poly1305_internal_test.c
property_test.c Update copyright year 2020-05-15 14:09:49 +01:00
provider_fallback_test.c TEST: Add provider_fallback_test, to test aspects of fallback providers 2020-06-04 16:42:45 +02:00
provider_internal_test.c
provider_internal_test.cnf.in Use .cnf for config files, not .conf 2020-03-06 18:25:13 +01:00
provider_test.c
proxy.cnf Cleanup cert config files for tests 2020-06-03 09:56:56 +02:00
rc2test.c Update copyright year 2020-04-23 13:55:52 +01:00
rc4test.c Update copyright year 2020-04-23 13:55:52 +01:00
rc5test.c Update copyright year 2020-04-23 13:55:52 +01:00
rdrand_sanitytest.c
README Centralise Environment Variables for the tests 2020-05-06 11:40:53 +01:00
README.external GOST external tests 2020-05-23 22:04:21 +03:00
README.ssltest.md Fix issues reported by markdownlint 2020-05-08 16:22:02 +02:00
recordlentest.c Update copyright year 2020-04-23 13:55:52 +01:00
rsa_complex.c
rsa_mp_test.c Update copyright year 2020-04-23 13:55:52 +01:00
rsa_sp800_56b_test.c Add ACVP fips module tests 2020-06-17 11:33:16 +10:00
rsa_test.c Update copyright year 2020-04-23 13:55:52 +01:00
run_tests.pl test/run_tests.pl: Improve indentation parsing workaround for VFO and VFP mode 2020-06-22 16:41:24 +02:00
sanitytest.c
secmemtest.c Update copyright year 2020-04-23 13:55:52 +01:00
serverinfo2.pem
serverinfo.pem
servername_test.c Update copyright year 2020-04-23 13:55:52 +01:00
session.pem
shibboleth.pfx
shlibloadtest.c
siphash_internal_test.c
sm2_internal_test.c Update copyright year 2020-04-23 13:55:52 +01:00
sm4_internal_test.c
smcont.txt
sparse_array_test.c
srptest.c
ssl_cert_table_internal_test.c
ssl_ctx_test.c Update copyright year 2020-05-15 14:09:49 +01:00
ssl_test_ctx_test.c Update copyright year 2020-04-23 13:55:52 +01:00
ssl_test_ctx_test.cnf Use .cnf for config files, not .conf 2020-03-06 18:25:13 +01:00
ssl_test_ctx.c Add a test for renegotiation with EXTMS dropped 2020-06-09 14:11:20 +02:00
ssl_test_ctx.h Update copyright year 2020-06-25 14:13:12 +01:00
ssl_test.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
ssl_test.tmpl
sslapitest.c Reduce the security bits for MD5 and SHA1 based signatures in TLS 2020-06-27 08:41:40 +02:00
sslbuffertest.c Update copyright year 2020-04-23 13:55:52 +01:00
sslcorrupttest.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
ssltest_old.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
ssltestlib.c Fix error path in int create_ssl_ctx_pair() 2020-06-05 10:31:58 +01:00
ssltestlib.h Update copyright year 2020-04-23 13:55:52 +01:00
stack_test.c
sysdefault.cnf
sysdefaulttest.c
test_test.c
test.cnf Add -section option to 'req' command 2020-03-07 12:58:02 +03:00
testcrl.pem
testdsa.pem
testdsapub.pem
testec-p256.pem
testecpub-p256.pem
tested448.pem
tested448pub.pem
tested25519.pem
tested25519pub.pem
testp7.pem
testreq2.pem
testrsa.pem
testrsapub.pem
testsid.pem
testutil.h test: update EVP tests to include DRBG testing 2020-06-24 20:07:46 +10:00
testx509.pem
threadstest.c
time_offset_test.c
tls13ccstest.c Update copyright year 2020-04-23 13:55:52 +01:00
tls13encryptiontest.c Update copyright year 2020-06-25 14:13:12 +01:00
tls13secretstest.c Update copyright year 2020-04-23 13:55:52 +01:00
tls-provider.c Make the naming scheme for dispatched functions more consistent 2020-06-24 22:01:22 +02:00
uitest.c
v3-cert1.pem
v3-cert2.pem
v3ext.c Update copyright year 2020-04-23 13:55:52 +01:00
v3nametest.c Update copyright year 2020-05-15 14:09:49 +01:00
verify_extra_test.c In OpenSSL builds, declare STACK for datatypes ... 2020-04-24 16:42:46 +02:00
versions.c
wpackettest.c WPACKET: don't write DER length when we don't want to 2020-05-04 05:50:06 +02:00
x509_check_cert_pkey_test.c Update copyright year 2020-04-23 13:55:52 +01:00
x509_dup_cert_test.c Update copyright year 2020-04-23 13:55:52 +01:00
x509_internal_test.c
x509_time_test.c Update copyright year 2020-04-23 13:55:52 +01:00
x509aux.c Update copyright year 2020-04-23 13:55:52 +01:00

How to add recipes
==================

For any test that you want to perform, you write a script located in
test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and
{name} is a unique name of your choice.

Please note that if a test involves a new testing executable, you will need to
do some additions in test/build.info. Please refer to the section "Changes to 
test/build.info" below.


Naming conventions
=================

A test executable is named test/{name}test.c

A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two
digit number and {name} is a unique name of your choice.

The number {nn} is (somewhat loosely) grouped as follows:

00-04  sanity, internal and essential API tests
05-09  individual symmetric cipher algorithms
10-14  math (bignum)
15-19  individual asymmetric cipher algorithms
20-24  openssl commands (some otherwise not tested)
25-29  certificate forms, generation and verification
30-35  engine and evp
60-79  APIs:
   60  X509 subsystem
   61  BIO subsystem
   65  CMP subsystem
   70  PACKET layer
80-89  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
90-98  misc
99     most time consuming tests [such as test_fuzz]


A recipe that just runs a test executable
=========================================

A script that just runs a program looks like this:

    #! /usr/bin/perl

    use OpenSSL::Test::Simple;

    simple_test("test_{name}", "{name}test", "{name}");

{name} is the unique name you have chosen for your test.

The second argument to `simple_test' is the test executable, and `simple_test'
expects it to be located in test/

For documentation on OpenSSL::Test::Simple, do
`perldoc util/perl/OpenSSL/Test/Simple.pm'.


A recipe that runs a more complex test
======================================

For more complex tests, you will need to read up on Test::More and
OpenSSL::Test.  Test::More is normally preinstalled, do `man Test::More' for
documentation.  For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'.

A script to start from could be this:

    #! /usr/bin/perl

    use strict;
    use warnings;
    use OpenSSL::Test;

    setup("test_{name}");

    plan tests => 2;                # The number of tests being performed

    ok(test1, "test1");
    ok(test2, "test1");

    sub test1
    {
        # test feature 1
    }

    sub test2
    {
        # test feature 2
    }


Changes to test/build.info
==========================

Whenever a new test involves a new test executable you need to do the
following (at all times, replace {NAME} and {name} with the name of your
test):

* add {name} to the list of programs under PROGRAMS_NO_INST

* create a three line description of how to build the test, you will have
to modify the include paths and source files if you don't want to use the
basic test framework:

    SOURCE[{name}]={name}.c
    INCLUDE[{name}]=.. ../include ../apps/include
    DEPEND[{name}]=../libcrypto libtestutil.a

Generic form of C test executables
==================================

    #include "testutil.h"

    static int my_test(void)
    {
        int testresult = 0;                 /* Assume the test will fail    */
        int observed;

        observed = function();              /* Call the code under test     */
        if (!TEST_int_eq(observed, 2))      /* Check the result is correct  */
            goto end;                       /* Exit on failure - optional   */

        testresult = 1;                     /* Mark the test case a success */
    end:
        cleanup();                          /* Any cleanup you require      */
        return testresult;
    }

    int setup_tests(void)
    {
        ADD_TEST(my_test);                  /* Add each test separately     */
        return 1;                           /* Indicate success             */
    }

You should use the TEST_xxx macros provided by testutil.h to test all failure
conditions.  These macros produce an error message in a standard format if the
condition is not met (and nothing if the condition is met).  Additional
information can be presented with the TEST_info macro that takes a printf
format string and arguments.  TEST_error is useful for complicated conditions,
it also takes a printf format string and argument.  In all cases the TEST_xxx
macros are guaranteed to evaluate their arguments exactly once.  This means
that expressions with side effects are allowed as parameters.  Thus,

    if (!TEST_ptr(ptr = OPENSSL_malloc(..)))

works fine and can be used in place of:

    ptr = OPENSSL_malloc(..);
    if (!TEST_ptr(ptr))

The former produces a more meaningful message on failure than the latter.

Note that the test infrastructure automatically sets up all required environment
variables (such as OPENSSL_MODULES, OPENSSL_CONF etc) for the tests. Individual
tests may choose to override the default settings as required.