openssl/test
Richard Levitte 953315ae60 test/build.info: add missing ../apps/include
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/8227)
2019-02-13 18:59:13 +01:00
..
certs Fix cert with rsa instead of rsaEncryption as public key algorithm 2018-12-31 09:47:12 +01:00
ct
d2i-tests
ocsp-tests Fix OCSP_basic_verify() cert chain construction in case bs->certs is NULL 2017-08-16 14:32:38 -04:00
ossl_shim Build: Change all _NO_INST to use attributes instead. 2019-01-22 12:35:39 +01:00
recipes Added new EVP/KDF API. 2019-02-13 12:11:49 +01:00
smime-certs Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ssl-tests Fix end-point shared secret for DTLS/SCTP 2019-02-01 11:57:19 +00:00
testutil Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
aborttest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
afalgtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asn1_decode_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asn1_encode_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asn1_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asn1_string_table_test.c typo ANS1 -> ASN1 2018-12-11 20:57:31 +10:00
asn1_time_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
asynciotest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
asynctest.c add an additional async notification communication method based on callback 2019-01-27 12:27:17 +00:00
bad_dtls_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
bftest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
bio_callback_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
bio_enc_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
bio_memleak_test.c Fix a memory leak in the mem bio 2019-01-21 17:47:02 +10:00
bioprinttest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
bntest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
bntests.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
build.info test/build.info: add missing ../apps/include 2019-02-13 18:59:13 +01:00
CAss.cnf
CAssdh.cnf
CAssdsa.cnf
CAssrsa.cnf
casttest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
CAtsa.cnf Added support for ESSCertIDv2 2017-05-03 09:04:23 +02:00
chacha_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
cipher_overhead_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
cipherbytes_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
cipherlist_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ciphername_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
clienthellotest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
cms-examples.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
cmsapitest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
conf_include_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
constant_time_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
crltest.c Fix a memory leak with di2_X509_CRL reuse 2019-01-31 19:10:57 +01:00
ct_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ctype_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
curve448_internal_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
d2i_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
danetest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
danetest.in Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
danetest.pem
destest.c add missing const 2018-12-09 22:02:48 -05:00
dhtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_data_ctr.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_data_hash.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_data_hmac.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_data.h Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
drbg_cavs_test.c Fix CID 1434549: Unchecked return value in test/evp_test.c 2019-01-08 16:27:27 +10:00
drbgtest.c Address a bug in the DRBG tests where the reseeding wasn't properly 2019-02-08 16:13:47 +10:00
drbgtest.h Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
dsa_no_digest_size_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
dsatest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
dtls_mtu_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
dtlstest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
dtlsv1listentest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ecdsatest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ecstresstest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
ectest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
enginetest.c Cleaner disposal of ephemeral engine ids and names 2018-12-09 22:02:48 -05:00
errtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
evp_extra_test.c Fix a crash in reuse of i2d_X509_PUBKEY 2019-01-31 19:24:07 +01:00
evp_kdf_test.c Added new EVP/KDF API. 2019-02-13 12:11:49 +01:00
evp_test.c Added new EVP/KDF API. 2019-02-13 12:11:49 +01:00
evp_test.h Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
exdatatest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
exptest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
fatalerrtest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
generate_buildtest.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
generate_ssl_tests.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
gmdifftest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
gosttest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
handshake_helper.c Fix end-point shared secret for DTLS/SCTP 2019-02-01 11:57:19 +00:00
handshake_helper.h Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
hmactest.c Cleaner disposal of ephemeral engine ids and names 2018-12-09 22:02:48 -05:00
ideatest.c add missing const in cast 2018-12-09 22:02:49 -05:00
igetest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
lhash_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
md2test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
mdc2_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
mdc2test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
memleaktest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
modes_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ocspapitest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
P1ss.cnf
P2ss.cnf
packettest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
pbelutest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
pemtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
pkcs7-1.pem
pkcs7.pem
pkey_meth_kdf_test.c Added new EVP/KDF API. 2019-02-13 12:11:49 +01:00
pkey_meth_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
pkits-test.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
poly1305_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rc2test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rc4test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rc5test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rdrand_sanitytest.c Remove unnecessary trailing whitespace 2019-02-05 16:25:11 +01:00
README Fix test documentation. 2017-09-08 13:58:59 -05:00
README.external Remove unnecessary trailing whitespace 2019-02-05 16:25:11 +01:00
README.ssltest.md Session resume broken switching contexts 2017-10-04 10:21:08 +10:00
recordlentest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
rsa_complex.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rsa_mp_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
rsa_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
run_tests.pl Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
sanitytest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
secmemtest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
serverinfo2.pem Add a SERVERINFOV2 format test file 2017-05-03 14:37:42 +01:00
serverinfo.pem
servername_test.c Remove unnecessary trailing whitespace 2019-02-05 16:25:11 +01:00
session.pem Don't store the ticket nonce in the session 2018-06-07 10:58:35 +01:00
shibboleth.pfx
shlibloadtest.c Introduce a no-pinshared option 2019-01-04 13:19:39 +00:00
siphash_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
sm2_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
sm4_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
smcont.txt
sparse_array_test.c Add sparse array data type. 2019-02-12 21:07:29 +10:00
srptest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ssl_cert_table_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
ssl_test_ctx_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
ssl_test_ctx_test.conf Implement Maximum Fragment Length TLS extension. 2017-11-05 17:46:48 +01:00
ssl_test_ctx.c Fix end-point shared secret for DTLS/SCTP 2019-02-01 11:57:19 +00:00
ssl_test_ctx.h Fix end-point shared secret for DTLS/SCTP 2019-02-01 11:57:19 +00:00
ssl_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
ssl_test.tmpl
sslapitest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
sslbuffertest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
sslcorrupttest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
ssltest_old.c Use (D)TLS_MAX_VERSION_INTERNAL internally 2018-12-15 12:52:02 +01:00
ssltestlib.c Cleanup vxworks support to be able to compile for VxWorks 7 2019-01-24 17:55:04 +01:00
ssltestlib.h Make sure we trigger retransmits in DTLS testing 2019-01-24 13:39:38 +00:00
Sssdsa.cnf
Sssrsa.cnf
stack_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
sysdefault.cnf Apply system_default configuration on SSL_CTX_new(). 2018-03-19 10:22:49 -04:00
sysdefaulttest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
test_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
test.cnf
testcrl.pem
testdsa.pem
testdsapub.pem
testec-p256.pem
testecpub-p256.pem
testp7.pem
testreq2.pem
testrsa.pem
testrsapub.pem
testsid.pem
testutil.h Move libapps headers into their own directory 2019-02-13 04:47:11 +01:00
testx509.pem
threadstest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
time_offset_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
tls13ccstest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
tls13encryptiontest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
tls13secretstest.c Remove unnecessary trailing whitespace 2019-02-05 16:25:11 +01:00
uitest.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
Uss.cnf
v3-cert1.pem
v3-cert2.pem
v3ext.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
v3nametest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
verify_extra_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
versions.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
wpackettest.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
x509_check_cert_pkey_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
x509_dup_cert_test.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00
x509_internal_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
x509_time_test.c Following the license change, modify the boilerplates in test/ 2018-12-06 14:19:22 +01:00
x509aux.c Updated test command line parsing to support commmon commands 2019-02-11 15:31:51 +01:00

How to add recipes
==================

For any test that you want to perform, you write a script located in
test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and
{name} is a unique name of your choice.

Please note that if a test involves a new testing executable, you will need to
do some additions in test/Makefile.  More on this later.


Naming conventions
=================

A test executable is named test/{name}test.c

A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two
digit number and {name} is a unique name of your choice.

The number {nn} is (somewhat loosely) grouped as follows:

00-04  sanity, internal and essential API tests
05-09  individual symmetric cipher algorithms
10-14  math (bignum)
15-19  individual asymmetric cipher algorithms
20-24  openssl commands (some otherwise not tested)
25-29  certificate forms, generation and verification
30-35  engine and evp
60-79  APIs
   70  PACKET layer
80-89  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
90-98  misc
99     most time consuming tests [such as test_fuzz]


A recipe that just runs a test executable
=========================================

A script that just runs a program looks like this:

    #! /usr/bin/perl

    use OpenSSL::Test::Simple;

    simple_test("test_{name}", "{name}test", "{name}");

{name} is the unique name you have chosen for your test.

The second argument to `simple_test' is the test executable, and `simple_test'
expects it to be located in test/

For documentation on OpenSSL::Test::Simple, do
`perldoc util/perl/OpenSSL/Test/Simple.pm'.


A recipe that runs a more complex test
======================================

For more complex tests, you will need to read up on Test::More and
OpenSSL::Test.  Test::More is normally preinstalled, do `man Test::More' for
documentation.  For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'.

A script to start from could be this:

    #! /usr/bin/perl

    use strict;
    use warnings;
    use OpenSSL::Test;

    setup("test_{name}");

    plan tests => 2;                # The number of tests being performed

    ok(test1, "test1");
    ok(test2, "test1");

    sub test1
    {
        # test feature 1
    }

    sub test2
    {
        # test feature 2
    }


Changes to test/build.info
==========================

Whenever a new test involves a new test executable you need to do the
following (at all times, replace {NAME} and {name} with the name of your
test):

* add {name} to the list of programs under PROGRAMS_NO_INST

* create a three line description of how to build the test, you will have
to modify the include paths and source files if you don't want to use the
basic test framework:

    SOURCE[{name}]={name}.c
    INCLUDE[{name}]=.. ../include
    DEPEND[{name}]=../libcrypto libtestutil.a

Generic form of C test executables
==================================

    #include "testutil.h"

    static int my_test(void)
    {
        int testresult = 0;                 /* Assume the test will fail    */
        int observed;

        observed = function();              /* Call the code under test     */
        if (!TEST_int_equal(observed, 2))   /* Check the result is correct  */
            goto end;                       /* Exit on failure - optional   */

        testresult = 1;                     /* Mark the test case a success */
    end:
        cleanup();                          /* Any cleanup you require      */
        return testresult;
    }

    int setup_tests(void)
    {
        ADD_TEST(my_test);                  /* Add each test separately     */
        return 1;                           /* Indicate success             */
    }

You should use the TEST_xxx macros provided by testutil.h to test all failure
conditions.  These macros produce an error message in a standard format if the
condition is not met (and nothing if the condition is met).  Additional
information can be presented with the TEST_info macro that takes a printf
format string and arguments.  TEST_error is useful for complicated conditions,
it also takes a printf format string and argument.  In all cases the TEST_xxx
macros are guaranteed to evaluate their arguments exactly once.  This means
that expressions with side effects are allowed as parameters.  Thus,

    if (!TEST_ptr(ptr = OPENSSL_malloc(..)))

works fine and can be used in place of:

    ptr = OPENSSL_malloc(..);
    if (!TEST_ptr(ptr))

The former produces a more meaningful message on failure than the latter.