Commit Graph

12 Commits

Author SHA1 Message Date
Richard Levitte
90aeaf6bce Add a more versatile test chooser
So far, the test runner (test/run_tests.pl) could get a list of tests
to run, and if non were given, it assumes all available tests should
be performed.

However, that makes skipping just one or two tests a bit of a pain.
This change makes the possibilities more versatile, run_checker.pl
takes these arguments and will process them in the given order,
starting with an empty set of tests to perform:

    alltests            The current set becomes the whole set of
                        available tests.
    test_xxx            Adds 'test_xxx' to the current set.
    -test_xxx           Removes 'test_xxx' from the current set.  If
                        nothing has been added to the set before this
                        argument, the current set is first initialised
                        to the whole set of available tests, then
                        'test_xxx' is removed from the current set.
    list                Display all available tests, then stop.

If no arguments are given, 'alltests' is assumed.

Reviewed-by: Emilia Käsper <emilia@openssl.org>
2016-09-01 20:58:40 +02:00
Richard Levitte
23049aa52e perl: use the 'if' module to conditionally load File::Glob
Trying to use normal perl conditions to conditionally 'use' a perl
module didn't quite work.  Using the 'if' module to do so does work.

Reviewed-by: Andy Polyakov <appro@openssl.org>
2016-05-30 11:55:46 +02:00
Andy Polyakov
9785555616 Configure,test/recipes: "pin" glob to File::Glob::glob.
As it turns out default glob's behaviour for quoted argument varies
from version to version, making it impossible to Configure or run
tests in some cases. The reason for quoting globs was to accommodate
source path with spaces in its name, which was treated by default glob
as multiple paths. File::Glob::glob on the other hand doesn't consider
spaces as delimiters and therefore works with unquoted patterns.

[Unfortunaltely File::Glob::glob, being too csh-ly, doesn't work
on VMS, hence the "pinning" is conditional.]

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-05-29 14:12:35 +02:00
Richard Levitte
04b7805a86 perl glob: make sure to put quotes around the pattern, in case of spaces
RT#4486

Reviewed-by: Matt Caswell <matt@openssl.org>
2016-05-27 17:41:32 +02:00
Emilia Kasper
5a22cf96a0 Replace cipherlist test
The old cipherlist test in ssltest.c only tests the internal order of
the cipher table, which is pretty useless.

Replace this test with a test that catches inadvertent changes to the
default cipherlist.

Fix run_tests.pl to correctly filter tests that have "list" in their name.

(Also includes a small drive-by fix in .gitignore.)

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-05-11 18:59:46 +02:00
Dr. Stephen Henson
be6bdab6f8 Recognise VERBOSE and V as well as HARNESS_VERBOSE
PR#4462

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-05-09 14:08:25 +01:00
Rich Salz
e0a651945c Copyright consolidation: perl files
Add copyright to most .pl files
This does NOT cover any .pl file that has other copyright in it.
Most of those are Andy's but some are public domain.
Fix typo's in some existing files.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-04-20 09:45:40 -04:00
Richard Levitte
42e0ccdfe8 unified build scheme: adjust test framework for out of source build tree
To be able to run tests when we've built in a directory other than
the source tree, the testing framework needs a few adjustments.

test/testlib/OpenSSL/Test.pm needs to know where it can find
shlib_wrap.sh, and a number of other tests need to be told a different
place to find engines than what they may be able to figure out on
their own.  Relying to $TOP is not enough, $SRCTOP and $BLDTOP can be
used as an alternative.

As part of this change, top_file and top_dir are removed and
srctop_file, bldtop_file, srctop_dir and bldtop_dir take their place.

Reviewed-by: Ben Laurie <ben@openssl.org>
2016-02-09 11:43:20 +01:00
FdaSilvaYY
df7421ccb4 few typo fixes
Signed-off-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
2016-01-22 11:47:24 -05:00
Richard Levitte
1780e6d916 Add a method to list available tests
Reviewed-by: Emilia Käsper <emilia@openssl.org>
2015-09-15 14:07:10 +02:00
Richard Levitte
25be5f44b8 Adapt the libssl test harness testing scripts to new testing framework
This involves adding $TOP/util as perl library in test/run_tests.pl.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
aec27d4d52 Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.

The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.

The main runner of the show is run_tests.pl.  As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:

	cd test
	OPENSSL_SRCDIR=.. perl run_tests.pl

The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t.  recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names.  recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.

Individual recipes can be run as well, of course, as they are perl
scripts in themselves.  For example, you can run only
recipes/00-check_testexes.t like so:

	cd test
	OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t

To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:

	use lib 'testlib';
	use OpenSSL::Test;

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00