openssl/test
Richard Levitte 3f7ce7f102 Refactor evp_pkey_make_provided() to do legacy to provider export
Previously, evp-keymgmt_util_export_to_provider() took care of all
kinds of exports of EVP_PKEYs to provider side keys, be it from its
legacy key or from another provider side key.  This works most of the
times, but there may be cases where the caller wants to be a bit more
in control of what sort of export happens when.

Also, when it's time to remove all legacy stuff, that job will be much
easier if we have a better separation between legacy support and
support of provided stuff, as far as we can take it.

This changes moves the support of legacy key to provider side key
export from evp-keymgmt_util_export_to_provider() to
evp_pkey_make_provided(), and makes sure the latter is called from all
EVP_PKEY functions that handle legacy stuff.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/11074)
2020-02-22 01:19:54 +01:00
..
certs Generate new Ed488 certificates 2020-02-11 23:23:42 +01:00
ct
d2i-tests
ocsp-tests
ossl_shim Replace BUF_ string function calls with OPENSSL_ ones 2019-10-17 23:19:59 +02:00
recipes test/recipes/30-test_evp_data/evppkey.txt 2020-02-22 01:07:16 +01:00
smime-certs Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
ssl-tests Check that ed25519 and ed448 are allowed by the security level 2020-02-11 23:24:03 +01:00
testutil Fix common test framework options 2020-02-03 11:41:56 +00:00
aborttest.c
aesgcmtest.c
afalgtest.c
asn1_decode_test.c Update source files for deprecation at 3.0 2019-11-07 11:37:25 +01:00
asn1_dsa_internal_test.c
asn1_encode_test.c Update source files for deprecation at 3.0 2019-11-07 11:37:25 +01:00
asn1_internal_test.c
asn1_string_table_test.c
asn1_time_test.c Add duplication APIs to ASN1_TIME and related types 2020-01-17 11:30:33 +08:00
asynciotest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
asynctest.c Deprecate most of debug-memory 2019-12-14 20:57:35 +01:00
bad_dtls_test.c TLS: use EVP for HMAC throughout libssl. 2020-01-29 19:49:23 +10:00
bftest.c Deprecate Low Level Blowfish APIs 2020-01-08 11:25:25 +00:00
bio_callback_test.c
bio_enc_test.c
bio_memleak_test.c Deprecate most of debug-memory 2019-12-14 20:57:35 +01:00
bio_prefix_text.c TEST: Add test recipe and help program to test BIO_f_prefix() 2019-12-18 19:42:44 +01:00
bioprinttest.c
bn_internal_test.c
bn_rand_range.h
bntest.c Fix some typos 2019-12-11 19:04:01 +01:00
bntests.pl
build.info Deprecate the low level Diffie-Hellman functions. 2020-02-20 19:04:57 +10:00
CAss.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
CAssdh.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
CAssdsa.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
CAssrsa.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
casttest.c Deprecate the Low Level CAST APIs 2020-01-13 13:44:27 +00:00
CAtsa.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
chacha_internal_test.c
cipher_overhead_test.c
cipherbytes_test.c
cipherlist_test.c
ciphername_test.c
clienthellotest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
cmp_asn_test.c chunk 6 of CMP contribution to OpenSSL 2019-12-12 10:57:25 +00:00
cmp_ctx_test.c chunk 7 of CMP contribution to OpenSSL 2020-02-17 07:43:58 +01:00
cmp_hdr_test.c fix various formatting nits in CMP contribution chunks 1-6 found by the new util/check-format.pl 2020-02-17 07:43:58 +01:00
cmp_msg_test.c fix various formatting nits in CMP contribution chunks 1-6 found by the new util/check-format.pl 2020-02-17 07:43:58 +01:00
cmp_protect_test.c chunk 7 of CMP contribution to OpenSSL 2020-02-17 07:43:58 +01:00
cmp_status_test.c chunk 5 of CMP contribution to OpenSSL 2019-10-29 14:17:39 +00:00
cmp_testlib.c add missing load_pkimsg() in test/cmp_testlib.c 2020-01-09 09:51:18 +01:00
cmp_testlib.h add missing load_pkimsg() in test/cmp_testlib.c 2020-01-09 09:51:18 +01:00
cmp_vfy_test.c chunk 7 of CMP contribution to OpenSSL 2020-02-17 07:43:58 +01:00
cms-examples.pl
cmsapitest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
conf_include_test.c Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
confdump.c Make sure we free the CONF structure allocated by confdump 2019-11-15 11:08:18 +00:00
constant_time_test.c
context_internal_test.c
crltest.c
ct_test.c Create a new embeddedSCTs1 that's signed using SHA256 2020-02-05 22:04:37 +01:00
ctype_internal_test.c
curve448_internal_test.c Implement a stricter ECX_KEY type 2020-02-11 22:32:47 +00:00
d2i_test.c Fix common test framework options 2020-02-03 11:41:56 +00:00
danetest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
danetest.in
danetest.pem
default-and-fips.cnf
default-and-legacy.cnf
default.cnf
destest.c Deprecate the low level DES functions. 2020-01-25 09:30:59 +10:00
dhtest.c Deprecate the low level Diffie-Hellman functions. 2020-02-20 19:04:57 +10:00
drbg_cavs_data_ctr.c
drbg_cavs_data_hash.c
drbg_cavs_data_hmac.c
drbg_cavs_data.h
drbg_cavs_test.c
drbgtest.c tests/drbgtest: use new RAND_DRBG callback_data API instead of ex_data 2020-02-07 11:38:57 +01:00
drbgtest.h
dsa_no_digest_size_test.c Deprecate the low level DSA functions. 2020-02-12 08:52:41 +10:00
dsatest.c Deprecate the low level DSA functions. 2020-02-12 08:52:41 +10:00
dtls_mtu_test.c
dtlstest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
dtlsv1listentest.c
ec_internal_test.c Deprecate the ECDSA and EV_KEY_METHOD functions. 2020-02-04 20:02:55 +10:00
ecdsatest.c Fix no-sm2 2020-02-06 13:57:35 +00:00
ecdsatest.h
ecstresstest.c
ectest.c Add self-test for EC_POINT_hex2point 2019-11-13 18:02:51 +02:00
enginetest.c
errtest.c
evp_extra_test.c test/evp_extra_test.c: adapt for RSA signature tests 2020-02-22 01:07:15 +01:00
evp_fetch_prov_test.c Introduce the provider property 2020-02-21 20:17:02 +00:00
evp_kdf_test.c Fix no-cmac and no-camellia 2019-11-14 09:44:18 +00:00
evp_pkey_dparams_test.c
evp_pkey_provided_test.c Implement Provider side Key Management for X25519 and X448 2020-02-11 22:32:56 +00:00
evp_test.c Params: add argument to the _from_text calls to indicate if the param exists. 2020-02-21 13:04:25 +01:00
evp_test.h
exdatatest.c
exptest.c
fatalerrtest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
ffc_internal_test.c Coverity 1458438: fix uninitialised memory access. 2020-02-17 19:29:05 +10:00
fips.cnf
generate_buildtest.pl
generate_ssl_tests.pl TEST: Optionally silence OpenSSL::Test::setup() 2020-02-18 09:45:53 +01:00
gmdifftest.c
gosttest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
handshake_helper.c TLS: use EVP for HMAC throughout libssl. 2020-01-29 19:49:23 +10:00
handshake_helper.h
hmactest.c Deprecate the low level HMAC functions 2020-01-29 19:49:23 +10:00
http_test.c Generalize the HTTP client so far implemented mostly in crypto/ocsp/ocsp_ht.c 2020-02-10 16:49:37 +01:00
ideatest.c Deprecate the low level IDEA functions. 2020-01-19 10:38:49 +10:00
igetest.c Deprecate the AES_ige_*() functions 2019-12-04 17:46:38 +00:00
keymgmt_internal_test.c Refactor evp_pkey_make_provided() to do legacy to provider export 2020-02-22 01:19:54 +01:00
legacy.cnf
lhash_test.c
mdc2_internal_test.c Deprecate the low level MDC2 functions. 2020-01-12 12:02:17 +10:00
mdc2test.c Deprecate the low level MDC2 functions. 2020-01-12 12:02:17 +10:00
memleaktest.c test/memleaktest.c: Modify for use with address/leak sanitizer 2019-12-10 14:16:12 +01:00
modes_internal_test.c Deprecate the low level AES functions 2020-01-06 15:09:57 +00:00
namemap_internal_test.c Modify EVP_CIPHER_is_a() and EVP_MD_is_a() to handle legacy methods too 2020-01-17 08:59:41 +01:00
ocspapitest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
ossl_test_endian.h
P1ss.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
P2ss.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
p_test.c
packettest.c
param_build_test.c Remove unused ossl_param_bld_to_param_ex() function. 2020-02-12 19:45:42 +10:00
params_api_test.c test/params_api_test.c: Correct the checks of OSSL_PARAM_set_BN() 2019-11-03 11:19:04 +01:00
params_conversion_test.c Fix common test framework options 2020-02-03 11:41:56 +00:00
params_test.c
pbelutest.c
pemtest.c
pkcs7-1.pem
pkcs7.pem
pkey_meth_kdf_test.c
pkey_meth_test.c
pkits-test.pl
poly1305_internal_test.c Add ChaCha related ciphers to default provider 2019-10-16 16:18:42 +10:00
property_test.c Properties: make query cache reference count aware. 2019-11-18 18:51:26 +10:00
provider_internal_test.c
provider_internal_test.conf.in
provider_test.c
rc2test.c Deprecate the low level RC2 functions 2020-01-16 07:07:27 +10:00
rc4test.c Deprecate the low level SHA functions. 2020-01-19 10:14:39 +10:00
rc5test.c Deprecate the low level RC5 functions 2020-01-16 07:07:27 +10:00
rdrand_sanitytest.c
README TEST: Add test recipe and help program to test BIO_f_prefix() 2019-12-18 19:42:44 +01:00
README.external
README.ssltest.md
recordlentest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
rsa_complex.c
rsa_mp_test.c Deprecate the low level RSA functions. 2020-02-20 18:58:40 +10:00
rsa_sp800_56b_test.c RSA generation: Use more bits of 1/sqrt(2) 2019-11-09 16:01:54 +01:00
rsa_test.c Deprecate the low level RSA functions. 2020-02-20 18:58:40 +10:00
run_tests.pl
sanitytest.c
secmemtest.c Make secure-memory be a config option 2020-02-14 15:18:27 +01:00
serverinfo2.pem
serverinfo.pem
servername_test.c Fix common test framework options 2020-02-03 11:41:56 +00:00
session.pem
shibboleth.pfx
shlibloadtest.c
siphash_internal_test.c
sm2_internal_test.c Deprecate the ECDSA and EV_KEY_METHOD functions. 2020-02-04 20:02:55 +10:00
sm4_internal_test.c
smcont.txt
sparse_array_test.c
srptest.c
ssl_cert_table_internal_test.c
ssl_ctx_test.c Add ssl_ctx_test to test suite. 2019-11-08 08:23:15 +01:00
ssl_test_ctx_test.c Fix common test framework options 2020-02-03 11:41:56 +00:00
ssl_test_ctx_test.conf
ssl_test_ctx.c
ssl_test_ctx.h
ssl_test.c Fix common test framework options 2020-02-03 11:41:56 +00:00
ssl_test.tmpl
sslapitest.c Fix no-tls1_3 2020-02-06 12:13:47 +00:00
sslbuffertest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
sslcorrupttest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
sslprovidertest.c Add a test for SSL_CTX_new_with_libctx() 2020-02-06 11:59:07 +00:00
ssltest_old.c Deprecate the low level Diffie-Hellman functions. 2020-02-20 19:04:57 +10:00
ssltestlib.c Don't exclude quite so much in a no-sock build 2020-02-21 21:41:56 +01:00
ssltestlib.h
Sssdsa.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
Sssrsa.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
stack_test.c
sysdefault.cnf
sysdefaulttest.c
test_test.c
test.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
testcrl.pem
testdsa.pem
testdsapub.pem
testec-p256.pem
testecpub-p256.pem
tested448.pem More testing for CLI usage of Ed25519 and Ed448 keys 2019-12-11 18:37:53 +01:00
tested448pub.pem More testing for CLI usage of Ed25519 and Ed448 keys 2019-12-11 18:37:53 +01:00
tested25519.pem More testing for CLI usage of Ed25519 and Ed448 keys 2019-12-11 18:37:53 +01:00
tested25519pub.pem More testing for CLI usage of Ed25519 and Ed448 keys 2019-12-11 18:37:53 +01:00
testp7.pem
testreq2.pem
testrsa.pem
testrsapub.pem
testsid.pem
testutil.h Fix common test framework options 2020-02-03 11:41:56 +00:00
testx509.pem
threadstest.c
time_offset_test.c
tls13ccstest.c Fix common test framework options 2020-02-03 11:41:56 +00:00
tls13encryptiontest.c
tls13secretstest.c Explicitly fetch ciphers and digests in libssl 2020-02-06 11:59:07 +00:00
uitest.c
Uss.cnf Remove RANDFILE settings from configuration files 2019-11-24 08:35:14 +01:00
v3-cert1.pem
v3-cert2.pem
v3ext.c Fix common test framework options 2020-02-03 11:41:56 +00:00
v3nametest.c
verify_extra_test.c Fix common test framework options 2020-02-03 11:41:56 +00:00
versions.c
wpackettest.c
x509_check_cert_pkey_test.c Fix common test framework options 2020-02-03 11:41:56 +00:00
x509_dup_cert_test.c Fix common test framework options 2020-02-03 11:41:56 +00:00
x509_internal_test.c
x509_time_test.c coverity 1456639: fix NULL dereference 2020-01-05 18:05:14 +10:00
x509aux.c Fix common test framework options 2020-02-03 11:41:56 +00:00

How to add recipes
==================

For any test that you want to perform, you write a script located in
test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and
{name} is a unique name of your choice.

Please note that if a test involves a new testing executable, you will need to
do some additions in test/build.info. Please refer to the section "Changes to 
test/build.info" below.


Naming conventions
=================

A test executable is named test/{name}test.c

A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two
digit number and {name} is a unique name of your choice.

The number {nn} is (somewhat loosely) grouped as follows:

00-04  sanity, internal and essential API tests
05-09  individual symmetric cipher algorithms
10-14  math (bignum)
15-19  individual asymmetric cipher algorithms
20-24  openssl commands (some otherwise not tested)
25-29  certificate forms, generation and verification
30-35  engine and evp
60-79  APIs:
   60  X509 subsystem
   61  BIO subsystem
   65  CMP subsystem
   70  PACKET layer
80-89  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
90-98  misc
99     most time consuming tests [such as test_fuzz]


A recipe that just runs a test executable
=========================================

A script that just runs a program looks like this:

    #! /usr/bin/perl

    use OpenSSL::Test::Simple;

    simple_test("test_{name}", "{name}test", "{name}");

{name} is the unique name you have chosen for your test.

The second argument to `simple_test' is the test executable, and `simple_test'
expects it to be located in test/

For documentation on OpenSSL::Test::Simple, do
`perldoc util/perl/OpenSSL/Test/Simple.pm'.


A recipe that runs a more complex test
======================================

For more complex tests, you will need to read up on Test::More and
OpenSSL::Test.  Test::More is normally preinstalled, do `man Test::More' for
documentation.  For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'.

A script to start from could be this:

    #! /usr/bin/perl

    use strict;
    use warnings;
    use OpenSSL::Test;

    setup("test_{name}");

    plan tests => 2;                # The number of tests being performed

    ok(test1, "test1");
    ok(test2, "test1");

    sub test1
    {
        # test feature 1
    }

    sub test2
    {
        # test feature 2
    }


Changes to test/build.info
==========================

Whenever a new test involves a new test executable you need to do the
following (at all times, replace {NAME} and {name} with the name of your
test):

* add {name} to the list of programs under PROGRAMS_NO_INST

* create a three line description of how to build the test, you will have
to modify the include paths and source files if you don't want to use the
basic test framework:

    SOURCE[{name}]={name}.c
    INCLUDE[{name}]=.. ../include ../apps/include
    DEPEND[{name}]=../libcrypto libtestutil.a

Generic form of C test executables
==================================

    #include "testutil.h"

    static int my_test(void)
    {
        int testresult = 0;                 /* Assume the test will fail    */
        int observed;

        observed = function();              /* Call the code under test     */
        if (!TEST_int_eq(observed, 2))      /* Check the result is correct  */
            goto end;                       /* Exit on failure - optional   */

        testresult = 1;                     /* Mark the test case a success */
    end:
        cleanup();                          /* Any cleanup you require      */
        return testresult;
    }

    int setup_tests(void)
    {
        ADD_TEST(my_test);                  /* Add each test separately     */
        return 1;                           /* Indicate success             */
    }

You should use the TEST_xxx macros provided by testutil.h to test all failure
conditions.  These macros produce an error message in a standard format if the
condition is not met (and nothing if the condition is met).  Additional
information can be presented with the TEST_info macro that takes a printf
format string and arguments.  TEST_error is useful for complicated conditions,
it also takes a printf format string and argument.  In all cases the TEST_xxx
macros are guaranteed to evaluate their arguments exactly once.  This means
that expressions with side effects are allowed as parameters.  Thus,

    if (!TEST_ptr(ptr = OPENSSL_malloc(..)))

works fine and can be used in place of:

    ptr = OPENSSL_malloc(..);
    if (!TEST_ptr(ptr))

The former produces a more meaningful message on failure than the latter.