Commit Graph

33 Commits

Author SHA1 Message Date
Sebastian Andrzej Siewior
9968c77539 Rename x86-32 assembly files from .s to .S.
Rename x86-32 assembly files from .s to .S. While processing the .S file
gcc will use the pre-processor whic will evaluate macros and ifdef. This
is turn will be used to enable the endbr32 opcode based on the __CET__
define.

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18353)
2022-05-24 13:16:06 +10:00
Henry Brausen
999376dcf3 Add clmul-based gmult for riscv64 with Zbb, Zbc
ghash-riscv64.pl implements 128-bit galois field multiplication for
use in the GCM mode using RISC-V carryless multiplication primitives.

The clmul-accelerated routine can be selected by setting the Zbb and
Zbc bits of the OPENSSL_riscvcap environment variable at runtime.

Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu>
Signed-off-by: Henry Brausen <henry.brausen@vrull.eu>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17640)
2022-05-19 16:32:49 +10:00
Andrey Matyukov
63b996e752 AES-GCM enabled with AVX512 vAES and vPCLMULQDQ.
Vectorized 'stitched' encrypt + ghash implementation of AES-GCM enabled
with AVX512 vAES and vPCLMULQDQ instructions (available starting Intel's
IceLake micro-architecture).

The performance details for representative IceLake Server and Client
platforms are shown below

Performance data:
OpenSSL Speed KBs/Sec
Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz (1Core/1Thread)
Payload in Bytes       16          64        256         1024        8192      16384
AES-128-GCM
  Baseline      478708.27   1118296.96  2428092.52  3518199.4   4172355.99  4235762.07
  Patched       534613.95   2009345.55  3775588.15  5059517.64  8476794.88  8941541.79
  Speedup            1.12         1.80        1.55        1.44        2.03        2.11

AES-256-GCM
  Baseline      399237.27   961699.9    2136377.65  2979889.15  3554823.37  3617757.5
  Patched       475948.13   1720128.51  3462407.12  4696832.2   7532013.16  7924953.91
  Speedup            1.19        1.79         1.62        1.58        2.12        2.19
Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz (1Core/1Thread)
Payload in Bytes       16          64        256         1024        8192      16384
AES-128-GCM
  Baseline      259128.54   570756.43   1362554.16  1990654.57  2359128.88  2401671.58
  Patched       292139.47   1079320.95  2001974.63  2829007.46  4510318.59  4705314.41
  Speedup            1.13        1.89         1.47        1.42        1.91        1.96
AES-256-GCM
  Baseline      236000.34   550506.76   1234638.08  1716734.57  2011255.6   2028099.99
  Patched       247256.32   919731.34   1773270.43  2553239.55  3953115.14  4111227.29
  Speedup            1.05        1.67         1.44        1.49        1.97        2.03

Reviewed-by: TJ O'Dwyer, Marcel Cornu, Pablo de Lara
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17239)
2022-02-10 15:10:12 +01:00
XiaokangQian
954f45ba4c Optimize AES-GCM for uarchs with unroll and new instructions
Increase the block numbers to 8 for every iteration.  Increase the hash
table capacity.  Make use of EOR3 instruction to improve the performance.

This can improve performance 25-40% on out-of-order microarchitectures
with a large number of fast execution units, such as Neoverse V1.  We also
see 20-30% performance improvements on other architectures such as the M1.

Assembly code reviewd by Tom Cosgrove (ARM).

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15916)
2022-01-25 14:30:00 +11:00
Danny Tsen
44a563dde1 AES-GCM performance optimzation with stitched method for p9+ ppc64le
Assembly code reviewed by Shricharan Srivatsan <ssrivat@us.ibm.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16854)
2022-01-24 11:25:53 +11:00
Richard Levitte
848af5e8fe Drop libimplementations.a
libimplementations.a was a nice idea, but had a few flaws:

1.  The idea to have common code in libimplementations.a and FIPS
    sensitive helper functions in libfips.a / libnonfips.a didn't
    catch on, and we saw full implementation ending up in them instead
    and not appearing in libimplementations.a at all.

2.  Because more or less ALL algorithm implementations were included
    in libimplementations.a (the idea being that the appropriate
    objects from it would be selected automatically by the linker when
    building the shared libraries), it's very hard to find only the
    implementation source that should go into the FIPS module, with
    the result that the FIPS checksum mechanism include source files
    that it shouldn't

To mitigate, we drop libimplementations.a, but retain the idea of
collecting implementations in static libraries.  With that, we not
have:

libfips.a

    Includes all implementations that should become part of the FIPS
    provider.

liblegacy.a

    Includes all implementations that should become part of the legacy
    provider.

libdefault.a

    Includes all implementations that should become part of the
    default and base providers.

With this, libnonfips.a becomes irrelevant and is dropped.
libcommon.a is retained to include common provider code that can be
used uniformly by all providers.

Fixes #15157

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15171)
2021-05-07 10:17:23 +02:00
Fangming.Fang
31b59078c8 Optimize AES-GCM implementation on aarch64
Comparing to current implementation, this change can get more
performance improved by tunning the loop-unrolling factor in
interleave implementation as well as by enabling high level parallelism.

Performance(A72)

new
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes   16384 bytes
aes-128-gcm     113065.51k   375743.00k   848359.51k  1517865.98k  1964040.19k  1986663.77k
aes-192-gcm     110679.32k   364470.63k   799322.88k  1428084.05k  1826917.03k  1848967.17k
aes-256-gcm     104919.86k   352939.29k   759477.76k  1330683.56k  1663175.34k  1670430.72k

old
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes   16384 bytes
aes-128-gcm     115595.32k   382348.65k   855891.29k  1236452.35k  1425670.14k  1429793.45k
aes-192-gcm     112227.02k   369543.47k   810046.55k  1147948.37k  1286288.73k  1296941.06k
aes-256-gcm     111543.90k   361902.36k   769543.59k  1070693.03k  1208576.68k  1207511.72k

Change-Id: I28a2dca85c001a63a2a942e80c7c64f7a4fdfcf7

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9818)
2019-12-19 12:36:07 +10:00
Shane Lontis
64fd90fbe9 Fix missing Assembler defines
Implementations are now spread across several libraries, so the assembler
related defines need to be applied to all affected libraries and modules.

AES_ASM define was missing from libimplementations.a which disabled AESNI
aarch64 changes were made by xkqian.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10180)
2019-10-16 16:10:39 +10:00
Richard Levitte
dec95d7589 Rework how our providers are built
We put almost everything in these internal static libraries:

libcommon               Block building code that can be used by all
                        our implementations, legacy and non-legacy
                        alike.
libimplementations      All non-legacy algorithm implementations and
                        only them.  All the code that ends up here is
                        agnostic to the definitions of FIPS_MODE.
liblegacy               All legacy implementations.

libnonfips              Support code for the algorithm implementations.
                        Built with FIPS_MODE undefined.  Any code that
                        checks that FIPS_MODE isn't defined must end
                        up in this library.
libfips                 Support code for the algorithm implementations.
                        Built with FIPS_MODE defined.  Any code that
                        checks that FIPS_MODE is defined must end up
                        in this library.

The FIPS provider module is built from providers/fips/*.c and linked
with libimplementations, libcommon and libfips.

The Legacy provider module is built from providers/legacy/*.c and
linked with liblegacy, libcommon and libcrypto.
If module building is disabled, the object files from liblegacy and
libcommon are added to libcrypto and the Legacy provider becomes a
built-in provider.

The Default provider module is built-in, so it ends up being linked
with libimplementations, libcommon and libnonfips.  For libcrypto in
form of static library, the object files from those other libraries
are simply being added to libcrypto.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10088)
2019-10-10 14:12:15 +02:00
Shane Lontis
ca392b2943 Add aes_wrap cipher to providers
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9406)
2019-09-20 12:33:02 +10:00
Richard Levitte
a1c8befd66 build.info: For all assembler generators, remove all arguments
Since the arguments are now generated in the build file templates,
they should be removed from the build.info files.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9884)
2019-09-16 16:29:57 +02:00
Shane Lontis
3a9f26f330 Add aes_xts cipher to providers
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9327)
2019-09-14 09:27:49 +10:00
Shane Lontis
3bfe9005e5 Add aes_ccm to provider
Add Cleanups for gcm - based on the changes to ccm.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/9280)
2019-08-20 08:54:41 +10:00
Shane Lontis
a672a02a64 Add gcm ciphers (aes and aria) to providers.
The code has been modularized so that it can be shared by algorithms.

A fixed size IV is now used instead of being allocated.
The IV is not set into the low level struct now until the update (it uses an
iv_state for this purpose).

Hardware specific methods have been added to a PROV_GCM_HW object.

The S390 code has been changed to just contain methods that can be accessed in
a modular way. There are equivalent generic methods also for the other
platforms.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/9231)
2019-07-31 21:55:16 +10:00
Richard Levitte
7f02a0932c crypto/modes/build.conf: Fix MODES asm mistakes
The old rule in Configure was that if the asm source had a file name
with 'ghash-' as part of the name, GHASH_ASM should be defined.  Since
none of the aarch64 asm files has such a name, that macro shouldn't
have been defined.

Fixes #9173

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9178)
2019-06-18 16:29:54 +02:00
Richard Levitte
a9bb7e9270 Move modes_asm_src file information to build.info files
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9166)
2019-06-17 16:08:53 +02:00
Richard Levitte
07c244f0cd Use variables in build.info files where it's worth the while
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9144)
2019-06-15 00:34:02 +02:00
Matt Caswell
66ad63e801 Make basic AES ciphers available from within the FIPS providers
These ciphers were already provider aware, and were available from the
default provider. We move them into the FIPS provider too.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9038)
2019-06-03 12:56:53 +01:00
Richard Levitte
77adb75e16 Build: Remove BEGINRAW / ENDRAW / OVERRIDE
It was an ugly hack to avoid certain problems that are no more.

Also added GENERATE lines for perlasm scripts that didn't have that
explicitly.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8125)
2019-01-31 16:19:49 +01:00
Todd Short
b1ceb439f2 Add RFC5297 AES-SIV support
Based originally on github.com/dfoxfranke/libaes_siv

This creates an SIV128 mode that uses EVP interfaces for the CBC, CTR
and CMAC code to reduce complexity at the cost of perfomance. The
expected use is for short inputs, not TLS-sized records.

Add multiple AAD input capacity in the EVP tests.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/3540)
2018-12-12 08:16:10 +10:00
Richard Levitte
722c9762f2 Harmonize the make variables across all known platforms families
The make variables LIB_CFLAGS, DSO_CFLAGS and so on were used in
addition to CFLAGS and so on.  This works without problem on Unix and
Windows, where options with different purposes (such as -D and -I) can
appear anywhere on the command line and get accumulated as they come.
This is not necessarely so on VMS.  For example, macros must all be
collected and given through one /DEFINE, and the same goes for
inclusion directories (/INCLUDE).

So, to harmonize all platforms, we repurpose make variables starting
with LIB_, DSO_ and BIN_ to be all encompassing variables that
collects the corresponding values from CFLAGS, CPPFLAGS, DEFINES,
INCLUDES and so on together with possible config target values
specific for libraries DSOs and programs, and use them instead of the
general ones everywhere.

This will, for example, allow VMS to use the exact same generators for
generated files that go through cpp as all other platforms, something
that has been impossible to do safely before now.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5357)
2018-02-14 17:13:53 +01:00
Richard Levitte
8c3bc594e0 Processing GNU-style "make variables" - separate CPP flags from C flags
C preprocessor flags get separated from C flags, which has the
advantage that we don't get loads of macro definitions and inclusion
directory specs when linking shared libraries, DSOs and programs.

This is a step to add support for "make variables" when configuring.

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5177)
2018-01-28 07:26:10 +01:00
Patrick Steuer
bc4e831ccd s390x assembly pack: extend s390x capability vector.
Extend the s390x capability vector to store the longer facility list
available from z13 onwards. The bits indicating the vector extensions
are set to zero, if the kernel does not enable the vector facility.

Also add capability bits returned by the crypto instructions' query
functions.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4542)
2017-10-30 14:31:32 +01:00
Andy Polyakov
c6cb8e3ca4 Alpha assembly pack: make it work on Linux.
Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-05-04 08:51:08 +02:00
Richard Levitte
f425f9dcff Add $(LIB_CFLAGS) for any build.info generator that uses $(CFLAGS)
The reason to do so is that some of the generators detect PIC flags
like -fPIC and -KPIC, and those are normally delivered in LD_CFLAGS.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-03-13 00:02:55 +01:00
Andy Polyakov
ee619197db crypto/*/build.info: make it work on ARM platforms.
Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-03-11 15:30:57 +01:00
Richard Levitte
f0667b1430 Add include directory options for assembler files that include from crypto/
Closes RT#4406

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-03-10 20:30:47 +01:00
Richard Levitte
dca5c70121 Add extra include directory for includers of ppc_arch.h
crypto/evp/e_aes.c and crypto/modes/gcm128.c include ppc_arch.h, which
is located in crypto/, so add that as extra include directory for them.

Issue reported by Jeffrey Walton <noloader@gmail.com>

Reviewed-by: Andy Polyakov <appro@openssl.org>
2016-03-09 12:07:15 +01:00
Richard Levitte
4f0d5f1849 Unified - adapt the generation of modes assembler to use GENERATE
This gets rid of the BEGINRAW..ENDRAW sections in crypto/modes/build.info.

This also moves the assembler generating perl scripts to take the
output file name as last command line argument, where necessary.

Reviewed-by: Andy Polyakov <appro@openssl.org>
2016-03-09 11:09:26 +01:00
Andy Polyakov
eb77e8886d SPARCv9 assembly pack: unify build rules and argument handling.
Make all scripts produce .S, make interpretation of $(CFLAGS)
pre-processor's responsibility, start accepting $(PERLASM_SCHEME).
[$(PERLASM_SCHEME) is redundant in this case, because there are
no deviataions between Solaris and Linux assemblers. This is
purely to unify .pl->.S handling across all targets.]

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-03-08 15:51:06 +01:00
Richard Levitte
de72be2e57 Pass $(CC) to perlasm scripts via the environment
It seems that on some platforms, the perlasm scripts call the C
compiler for certain checks.  These scripts need the environment
variable CC to have the C compiler command.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-02-13 19:21:36 +01:00
Richard Levitte
567a9e6fe0 unified build scheme: add a "unified" template for Unix Makefile
This also adds all the raw sections needed for some files.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-02-10 14:36:04 +01:00
Richard Levitte
777a288270 unified build scheme: add build.info files
Now that we have the foundation for the "unified" build scheme in
place, we add build.info files.  They have been generated from the
Makefiles in the same directories.  Things that are platform specific
will appear in later commits.

Reviewed-by: Andy Polyakov <appro@openssl.org>
2016-02-01 12:46:58 +01:00