openssl/test
Dr. Stephen Henson c749308fc4 Add tests for SHA1 and EC point compression
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2739)
2017-02-25 00:46:45 +00:00
..
certs EC certificate with compression point 2017-02-24 23:52:22 +00:00
ct
d2i-tests add test for CVE-2016-7053 2016-11-10 13:04:11 +00:00
ocsp-tests
ossl_shim Move extension data into sub-structs 2017-01-09 22:26:47 -05:00
recipes X509 time: tighten validation per RFC 5280 2017-02-24 17:37:08 +01:00
smime-certs
ssl-tests Add tests for SHA1 and EC point compression 2017-02-25 00:46:45 +00:00
testlib Update the kex modes tests to check various HRR scenarios 2017-02-14 13:14:25 +00:00
aborttest.c
afalgtest.c
asn1_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
asynciotest.c Let test handshakes stop on certain errors 2017-02-23 19:40:27 +01:00
asynctest.c Add test to show wrong behavior of ASYNC_WAIT_CTX 2017-02-13 15:29:42 +00:00
bad_dtls_test.c Solution proposal for issue #1647. 2016-11-12 22:26:20 -05:00
bftest.c
bio_enc_test.c
bioprinttest.c
bntest.c bntest: do not stop on first fautl encountered 2017-02-01 02:03:29 +01:00
bntests.pl Make bntest be (mostly) file-based. 2016-11-28 12:26:05 -05:00
bntests.txt bntests.txt: add a couple of checks of possibly negative zero 2017-02-01 02:03:29 +01:00
build.info X509 time: tighten validation per RFC 5280 2017-02-24 17:37:08 +01:00
CAss.cnf
CAssdh.cnf
CAssdsa.cnf
CAssrsa.cnf
casttest.c
CAtsa.cnf
cipher_overhead_test.c Add unit test for ssl_cipher_get_overhead() 2016-11-02 14:00:11 +00:00
cipherbytes_test.c Tests for SSL_bytes_to_cipher_list() 2017-02-23 19:40:25 +01:00
cipherlist_test.c update test 2017-02-08 02:16:28 +00:00
clienthellotest.c Fix a warning about an uninit var 2016-11-24 18:02:43 +00:00
cms-examples.pl
constant_time_test.c constant time test: include our internal/numbers.h rather than limits.h 2016-11-05 11:38:29 +01:00
crltest.c GH2176: Add X509_VERIFY_PARAM_get_time 2017-01-12 09:54:09 -05:00
ct_test.c Make sure things get deleted when test setup fails in ct_test.c 2016-11-16 13:54:17 +00:00
d2i_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
danetest.c
danetest.in
danetest.pem
destest.c
dhtest.c
dsatest.c
dtls_mtu_test.c Let test handshakes stop on certain errors 2017-02-23 19:40:27 +01:00
dtlstest.c Let test handshakes stop on certain errors 2017-02-23 19:40:27 +01:00
dtlsv1listentest.c
ecdhtest_cavs.h
ecdhtest.c
ecdsatest.c
ectest.c
enginetest.c
evp_extra_test.c
evp_test.c Call EVP_CipherFinal in CCM mode for tests. 2017-02-08 02:16:27 +00:00
evptests.txt Implementation of the ARIA cipher as described in RFC 5794. 2017-02-21 11:51:45 +01:00
exdatatest.c
exptest.c
generate_buildtest.pl
generate_ssl_tests.pl
gmdifftest.c
handshake_helper.c Tests for SSL early callback 2017-02-23 19:40:26 +01:00
handshake_helper.h Add test support for TLS signature types. 2017-01-30 13:00:17 +00:00
hmactest.c
ideatest.c
igetest.c
md2test.c
md4test.c
md5test.c
mdc2_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
mdc2test.c
memleaktest.c
methtest.c
modes_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
p5_crpt2_test.c
P1ss.cnf
P2ss.cnf
packettest.c
pbelutest.c
pkcs7-1.pem
pkcs7.pem
pkey_meth_test.c Add test to check EVP_PKEY method ordering. 2016-11-20 00:22:02 +00:00
pkits-test.pl Remove trailing whitespace from some files. 2016-10-10 23:36:21 +01:00
poly1305_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
r160test.c
randtest.c
rc2test.c
rc4test.c
rc5test.c
README test/README: clarify last test number group 2017-02-17 20:58:04 +01:00
README.external Fix argument order in documentation 2016-11-04 10:38:54 +00:00
README.ssltest.md Add test support for TLS signature types. 2017-01-30 13:00:17 +00:00
rmdtest.c
rsa_test.c
run_tests.pl
sanitytest.c
secmemtest.c
serverinfo.pem
sha1test.c
sha256t.c
sha512t.c
shibboleth.pfx
shlibloadtest.c Fix no-dso (shlibloadtest) 2016-11-10 10:12:00 +00:00
siphash_internal_test.c Add support for parameterized SipHash 2017-02-01 14:14:36 -05:00
smcont.txt
srptest.c Add SRP test vectors from RFC5054 2016-10-01 13:46:54 +01:00
ssl_test_ctx_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
ssl_test_ctx_test.conf
ssl_test_ctx.c Tests for SSL early callback 2017-02-23 19:40:26 +01:00
ssl_test_ctx.h Tests for SSL early callback 2017-02-23 19:40:26 +01:00
ssl_test.c Add test support for TLS signature types. 2017-01-30 13:00:17 +00:00
ssl_test.tmpl
sslapitest.c Add an sslapitest for early callback 2017-02-23 19:40:27 +01:00
sslcorrupttest.c Let test handshakes stop on certain errors 2017-02-23 19:40:27 +01:00
ssltest_old.c Move extension data into sub-structs 2017-01-09 22:26:47 -05:00
ssltestlib.c Let test handshakes stop on certain errors 2017-02-23 19:40:27 +01:00
ssltestlib.h Let test handshakes stop on certain errors 2017-02-23 19:40:27 +01:00
Sssdsa.cnf
Sssrsa.cnf
test_main_custom.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
test_main_custom.h Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
test_main.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
test_main.h Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
test.cnf
testcrl.pem
testdsa.pem
testdsapub.pem
testec-p256.pem
testecpub-p256.pem
testp7.pem
testreq2.pem
testrsa.pem
testrsapub.pem
testsid.pem
testutil.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
testutil.h Add -Wundef to --strict-warnings options. 2017-02-24 09:21:59 +01:00
testx509.pem
threadstest.c
tls13encryptiontest.c Fix crash in tls13_enc 2017-02-08 11:41:45 +00:00
tls13secretstest.c Test logging TLSv1.3 secrets. 2017-02-02 09:34:00 +00:00
uitest.c UI: fix uitest for VMS 2017-01-12 15:23:15 +01:00
Uss.cnf
v3-cert1.pem
v3-cert2.pem
v3ext.c
v3nametest.c
verify_extra_test.c Fix some extra or missing whitespaces... 2017-01-25 09:06:34 +00:00
wp_test.c
wpackettest.c Add a test for WPACKET_fill_lengths() 2017-01-30 10:18:24 +00:00
x509_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
x509_time_test.c X509 time: tighten validation per RFC 5280 2017-02-24 17:37:08 +01:00
x509aux.c test/x509aux.c: Fix argv loop 2016-09-21 16:19:22 +02:00

How to add recipes
==================

For any test that you want to perform, you write a script located in
test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and
{name} is a unique name of your choice.

Please note that if a test involves a new testing executable, you will need to
do some additions in test/Makefile.  More on this later.


Naming conventions
=================

A test executable is named test/{name}test.c

A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two
digit number and {name} is a unique name of your choice.

The number {nn} is (somewhat loosely) grouped as follows:

00-04  sanity, internal and essential API tests
05-09  individual symmetric cipher algorithms
10-14  math (bignum)
15-19  individual asymmetric cipher algorithms
20-24  openssl commands (some otherwise not tested)
25-29  certificate forms, generation and verification
30-35  engine and evp
60-79  APIs
   70  PACKET layer
80-89  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
90-98  misc
99     most time consuming tests [such as test_fuzz]


A recipe that just runs a test executable
=========================================

A script that just runs a program looks like this:

    #! /usr/bin/perl
    
    use OpenSSL::Test::Simple;
    
    simple_test("test_{name}", "{name}test", "{name}");

{name} is the unique name you have chosen for your test.

The second argument to `simple_test' is the test executable, and `simple_test'
expects it to be located in test/

For documentation on OpenSSL::Test::Simple, do
`perldoc test/testlib/OpenSSL/Test/Simple.pm'.


A recipe that runs a more complex test
======================================

For more complex tests, you will need to read up on Test::More and
OpenSSL::Test.  Test::More is normally preinstalled, do `man Test::More' for
documentation.  For OpenSSL::Test, do `perldoc test/testlib/OpenSSL/Test.pm'.

A script to start from could be this:

    #! /usr/bin/perl
    
    use strict;
    use warnings;
    use OpenSSL::Test;
    
    setup("test_{name}");
    
    plan tests => 2;                # The number of tests being performed
    
    ok(test1, "test1");
    ok(test2, "test1");
    
    sub test1
    {
        # test feature 1
    }
    
    sub test2
    {
        # test feature 2
    }
    

Changes to test/Makefile
========================

Whenever a new test involves a new test executable you need to do the
following (at all times, replace {NAME} and {name} with the name of your
test):

* among the variables for test executables at the beginning, add a line like
  this:

    {NAME}TEST= {name}test

* add `$({NAME}TEST)$(EXE_EXT)' to the assignment of EXE:

* add `$({NAME}TEST).o' to the assignment of OBJ:

* add `$({NAME}TEST).c' to the assignment of SRC:

* add the following lines for building the executable:

    $({NAME}TEST)$(EXE_EXT): $({NAME}TEST).o $(DLIBCRYPTO)
           @target=$({NAME}TEST); $(BUILD_CMD)