Simple tests only need to implement register_tests().
Tests that need a custom main() should implement test_main(). This will
be wrapped in a main() that performs common setup/teardown (currently
crypto-mdebug).
Note that for normal development, enable-asan is usually
sufficient for detecting leaks, and more versatile.
enable-crypto-mdebug is stricter as it will also
insist that all static variables be freed. This is useful for debugging
library init/deinit; however, it also means that test_main() must free
everything it allocates.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
Normally WPACKETs will use a BUF_MEM which can grow as required. Sometimes
though that may be overkill for what is needed - a static buffer may be
sufficient. This adds that capability.
Reviewed-by: Rich Salz <rsalz@openssl.org>
MD5/SHA1 and MDC-2 have special-case logic beyond the generic DigestInfo
wrapping. Test that each of these works, including hash and length
mismatches (both input and signature). Also add VerifyRecover tests. It
appears 5824cc2981 added support for
VerifyRecover, but forgot to add the test data.
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1474
RFC 3447, section 8.2.2, steps 3 and 4 states that verifiers must encode
the DigestInfo struct and then compare the result against the public key
operation result. This implies that one and only one encoding is legal.
OpenSSL instead parses with crypto/asn1, then checks that the encoding
round-trips, and allows some variations for the parameter. Sufficient
laxness in this area can allow signature forgeries, as described in
https://www.imperialviolet.org/2014/09/26/pkcs1.html
Although there aren't known attacks against OpenSSL's current scheme,
this change makes OpenSSL implement the algorithm as specified. This
avoids the uncertainty and, more importantly, helps grow a healthy
ecosystem. Laxness beyond the spec, particularly in implementations
which enjoy wide use, risks harm to the ecosystem for all. A signature
producer which only tests against OpenSSL may not notice bugs and
accidentally become widely deployed. Thus implementations have a
responsibility to honor the specification as tightly as is practical.
In some cases, the damage is permanent and the spec deviation and
security risk becomes a tax all implementors must forever pay, but not
here. Both BoringSSL and Go successfully implemented and deployed
RSASSA-PKCS1-v1_5 as specified since their respective beginnings, so
this change should be compatible enough to pin down in future OpenSSL
releases.
See also https://tools.ietf.org/html/draft-thomson-postel-was-wrong-00
As a bonus, by not having to deal with sign/verify differences, this
version is also somewhat clearer. It also more consistently enforces
digest lengths in the verify_recover codepath. The NID_md5_sha1 codepath
wasn't quite doing this right.
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #1474
1) Remove some unnecessary fixtures
2) Add EXECUTE_TEST_NO_TEARDOWN shorthand when a fixture exists but has
no teardown.
3) Fix return values in ct_test.c (introduced by an earlier refactoring,
oops)
Note that for parameterized tests, the index (test vector) usually holds all the
customization, and there should be no need for a separate test
fixture. The CTS test is an exception: it demonstrates how to combine
customization with parameterization.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Prior to TLS1.3 we check that the received record version number is correct.
In TLS1.3 we need to ignore the record version number. This adds a test to
make sure we do it correctly.
Reviewed-by: Rich Salz <rsalz@openssl.org>
The sources for internal tests were sometimes badly formed, assuming
perl variables such as $target{cpuid_asm_src} contains only one file
name. This change correctly massages all file names in such a
variable.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1850)
The test fixtures are (meant to be) useful for sharing common
setup. Don't bother when we don't have any setup/teardown.
This only addresses simple tests. Parameterized tests (ADD_ALL_TESTS)
will be made more user-friendly in a follow-up.
Reviewed-by: Rich Salz <rsalz@openssl.org>
The updated shim has the ability to skip tests using unimplemented flags.
This should reduce the number of test failures.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Added the file README.external which describes how to build and run OpenSSL
to use the BoringSSL test suite. Also updated INSTALL to point to it.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Use the newly added "executable" function rather than "system". Also filter
the output to add a prefix to every line so that the "ok" doesn't confuse
Test::More
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
This just disables all tests that fail at the moment. Over time we will
want to go over these and figure out why they are failing (and fix them if
appropriate)
Reviewed-by: Richard Levitte <levitte@openssl.org>
This adds a test to the OpenSSL test suite to invoke the BoringSSL test
suite.
It assumes you have already compiled the ossl_shim (see previous commit).
It also assumes that you have an environment variable BORING_RUNNER_DIR
set up to point to the ssl/test/runner directory of a checkout of BoringSSL.
This has only been tested with a very old version of BoringSSL (from commit
f277add6c) - since that was the last known checkout where the shim compiles
successfully. Even with that version of BoringSSL this test will fail. There
are lots of Boring tests that are failing for various reasons. Some might
be due to bugs in OpenSSL, some might be due to features that BoringSSL has
that OpenSSL doesn't, some are due to assumptions about the way BoringSSL
behaves that are not true for OpenSSL etc.
To get the verbose BoringSSL test output, run like this:
VERBOSE=1 BORING_RUNNER_DIR=/path/to/boring/ssl/test/runner make \
TESTS="test_external" test
Reviewed-by: Richard Levitte <levitte@openssl.org>
The BoringSSL test suite contains numerous tests which OpenSSL does not.
The BoringSSL test runner uses a shim to launch the library and execute the
tests. This is a version of the BoringSSL shim converted to compile against
OpenSSL instead.
This is primarily based on the work of David Benjamin from the BoringSSL
project who did most of the necessary conversion. It also includes a few
other tweaks for opacity changes etc.
This is based on a *very* old version of BoringSSL from commit f277add6c.
That was the last commit known to work with this patched shim. Later
versions may also work but lots of merge conflicts occur when trying to
bring it up to date.
At the moment this has not been integrated into the build system. There is
a very simple standalone makefile in the ossl_shim directory which should
be executed directly before tyring to use the shim.
Reviewed-by: Richard Levitte <levitte@openssl.org>
test/shlibloadtest.c assumes all Unix style platforms use .so as
shared library extension. This is not the case for Mac OS X, which
uses .dylib. Instead of this, have the test recipe find out the
extension from configuration data.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1844)
- Make sure to initialise SHLIB variables
- Make sure to make local variables static
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1838)
This should demonstrate that the atexit() handling is working properly (or
at least not crashing) on process exit.
Reviewed-by: Tim Hudson <tjh@openssl.org>
Also we disable TLS1.3 by default (use enable-tls1_3 to re-enable). This is
because this is a WIP and will not be interoperable with any other TLS1.3
implementation.
Finally, we fix some tests that started failing when TLS1.3 was disabled by
default.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Includes addition of the various options to s_server/s_client. Also adds
one of the new TLS1.3 ciphersuites.
This isn't "real" TLS1.3!! It's identical to TLS1.2 apart from the protocol
and the ciphersuite...and the ciphersuite is just a renamed TLS1.2 one (not
a "real" TLS1.3 ciphersuite).
Reviewed-by: Rich Salz <rsalz@openssl.org>
A BIO_read() 0 return indicates that a failure occurred that may be
retryable. An SSL_read() 0 return indicates a non-retryable failure. Check
that if BIO_read() returns 0, SSL_read() returns <0. Same for SSL_write().
The asyncio test filter BIO already returns 0 on a retryable failure so we
build on that.
Reviewed-by: Richard Levitte <levitte@openssl.org>
So far, apps and test programs, were a bit rigidely accessible as
executables or perl scripts. But what about scripts in some other
language? Or what about running entirely external programs? The
answer is certainly not to add new functions to access scripts for
each language or wrapping all the external program calls in our magic!
Instead, this adds a new functions, cmd(), which is useful to access
executables and scripts in a more generalised manner. app(), test(),
fuzz(), perlapp() and perltest() are rewritten in terms of cmd(), and
serve as examples how to do something similar for other scripting
languages, or constrain the programs to certain directories.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1686)
The prevailing style seems to not have trailing whitespace, but a few
lines do. This is mostly in the perlasm files, but a few C files got
them after the reformat. This is the result of:
find . -name '*.pl' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'
find . -name '*.c' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'
find . -name '*.h' | xargs sed -E -i '' -e 's/( |'$'\t'')*$//'
Then bn_prime.h was excluded since this is a generated file.
Note mkerr.pl has some changes in a heredoc for some help output, but
other lines there lack trailing whitespace too.
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Add update for testing renegotiation. Also change info on CTLOG_FILE
environment variable - which always seems to be required.
Reviewed-by: Rich Salz <rsalz@openssl.org>
The TLSProxy::Record->new call hard-codes a version, like
70-test_sslrecords.t.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
This is a regression test for
https://github.com/openssl/openssl/pull/1431. It tests a
maximally-padded record with each possible invalid offset.
This required fixing a bug in Message.pm where the client sending a
fatal alert followed by close_notify was still treated as success.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
A mem leak could occur on an error path. Also the mempacket BIO_METHOD
needs to be cleaned up, because of the newly added DTLS test.
Also fixed a double semi-colon in ssltestlib.c
Reviewed-by: Rich Salz <rsalz@openssl.org>
There are cases when argc is more trustable than proper argv termination.
Since we trust argc in all other test programs, we might as well treat it
the same way in this program.
Reviewed-by: Matt Caswell <matt@openssl.org>
All the other functions that take an argument for the number of bytes
use convenience macros for this purpose. We should do the same with
WPACKET_put_bytes().
Reviewed-by: Rich Salz <rsalz@openssl.org>
Updated the construction code to use the new function. Also added some
convenience macros for WPACKET_sub_memcpy().
Reviewed-by: Rich Salz <rsalz@openssl.org>
A few style tweaks here and there. The main change is that curr and
packet_len are now offsets into the buffer to account for the fact that
the pointers can change if the buffer grows. Also dropped support for the
WPACKET_set_packet_len() function. I thought that was going to be needed
but so far it hasn't been. It doesn't really work any more due to the
offsets change.
Reviewed-by: Rich Salz <rsalz@openssl.org>
The tests will only work in no-shared builds because WPACKET is an
internal only API that does not get exported by the shared library.
Reviewed-by: Rich Salz <rsalz@openssl.org>
So far, the test runner (test/run_tests.pl) could get a list of tests
to run, and if non were given, it assumes all available tests should
be performed.
However, that makes skipping just one or two tests a bit of a pain.
This change makes the possibilities more versatile, run_checker.pl
takes these arguments and will process them in the given order,
starting with an empty set of tests to perform:
alltests The current set becomes the whole set of
available tests.
test_xxx Adds 'test_xxx' to the current set.
-test_xxx Removes 'test_xxx' from the current set. If
nothing has been added to the set before this
argument, the current set is first initialised
to the whole set of available tests, then
'test_xxx' is removed from the current set.
list Display all available tests, then stop.
If no arguments are given, 'alltests' is assumed.
Reviewed-by: Emilia Käsper <emilia@openssl.org>
These tests take a very long time on some platforms, and arent't
always strictly necessary. This makes it possible to turn them
off. The necessary binaries are still built, though, in case
someone still wants to do a manual run.
Reviewed-by: Andy Polyakov <appro@openssl.org>
The previous commit revealed a long standing problem where CertStatus
processing was broken in DTLS. This would have been revealed by better
testing - so add some!
Reviewed-by: Rich Salz <rsalz@openssl.org>
User can make Windows openssl.exe to treat command-line arguments
and console input as UTF-8 By setting OPENSSL_WIN32_UTF8 environment
variable (to any value). This is likely to be required for data
interchangeability with other OSes and PKCS#12 containers generated
with Windows CryptoAPI.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Test doesn't work on Windows with non-Greek locale, because of
Win32 perl[!] limitation, not OpenSSL. For example it passes on
Cygwin and MSYS...
Reviewed-by: Matt Caswell <matt@openssl.org>
There was a block of code at the start that used the Camellia cipher. The
original idea behind this was to fill the buffer with non-zero data so that
oversteps can be detected. However this block failed when using no-camellia.
This has been replaced with a RAND_bytes() call.
I also updated the the CTR test section, since it seems to be using a CBC
cipher instead of a CTR cipher.
Reviewed-by: Andy Polyakov <appro@openssl.org>
Also, re-organize RSA check to use goto err.
Add a test case.
Try all checks, not just stopping at first (via Richard Levitte)
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
The variable 'buffer', allocated by EC_POINT_point2buf(), isn't
free'd on the success path.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
In mempacket_test_read(), we've already fetched the top value of the
stack, so when we shift the stack, we don't care for the value. The
compiler needs to be told, or it will complain harshly when we tell it
to be picky.
Reviewed-by: Matt Caswell <matt@openssl.org>
Originally PKCS#12 subroutines treated password strings as ASCII.
It worked as long as they were pure ASCII, but if there were some
none-ASCII characters result was non-interoperable. But fixing it
poses problem accessing data protected with broken password. In
order to make asscess to old data possible add retry with old-style
password.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Clang was complaining about some unused functions. Moving the stack
declaration to the header seems to sort it. Also the certstatus variable
in dtlstest needed to be declared static.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Injects a record from epoch 1 during epoch 0 handshake, with a record
sequence number in the future, to test that the record replay protection
feature works as expected. This is described more fully in the next commit.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Add a test to inject a record from the next epoch during the handshake and
make sure it doesn't get processed immediately.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Split the create_ssl_connection() helper function into two steps: one to
create the SSL objects, and one to actually create the connection. This
provides the ability to make changes to the SSL object before the
connection is actually made.
Reviewed-by: Richard Levitte <levitte@openssl.org>
This adds a BIO similar to a normal mem BIO but with datagram awareness.
It also has the capability to inject additional packets at arbitrary
locations into the BIO, for testing purposes.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Dump out the records passed over the BIO. Only works for DTLS at the
moment but could easily be extended to TLS.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Make maximum fragment length configurable and add various fragmentation
tests, in addition to the existing multi-buffer tests.
Reviewed-by: Rich Salz <rsalz@openssl.org>
In practice, CT isn't really functional without EC anyway, as most logs
use EC keys. So, skip loading the log list with no-ec, and skip CT tests
completely in that conf.
Reviewed-by: Rich Salz <rsalz@openssl.org>
This commit only ports existing tests, and adds some coverage for
resumption. We don't appear to have any handshake tests that cover SCT
validation success, and this commit doesn't change that.
Reviewed-by: Rich Salz <rsalz@openssl.org>
In NPN and ALPN, the protocol is renegotiated upon resumption. Test that
resumption picks up changes to the extension.
Reviewed-by: Rich Salz <rsalz@openssl.org>
