Increase the block numbers to 8 for every iteration. Increase the hash
table capacity. Make use of EOR3 instruction to improve the performance.
This can improve performance 25-40% on out-of-order microarchitectures
with a large number of fast execution units, such as Neoverse V1. We also
see 20-30% performance improvements on other architectures such as the M1.
Assembly code reviewd by Tom Cosgrove (ARM).
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15916)
Assembly code reviewed by Shricharan Srivatsan <ssrivat@us.ibm.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16854)
GCC's __ARMEL__ and __ARMEB__ defines denote little- and big-endian arm,
respectively. They are not defined on aarch64, which instead use
__AARCH64EL__ and __AARCH64EB__.
However, OpenSSL's assembly originally used the 32-bit defines on both
platforms and even define __ARMEL__ and __ARMEB__ in arm_arch.h. This is
less portable and can even interfere with other headers, which use
__ARMEL__ to detect little-endian arm.
Over time, the aarch64 assembly has switched to the correct defines,
such as in 32bbb62ea6. This commit
finishes the job: poly1305-armv8.pl needed a fix and the dual-arch
armx.pl files get one more transform to convert from 32-bit to 64-bit.
(There is an even more official endianness detector, __ARM_BIG_ENDIAN in
the Arm C Language Extensions. But I've stuck with the GCC ones here as
that would be a larger change.)
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/17373)
This change adds optional support for
- Armv8.3-A Pointer Authentication (PAuth) and
- Armv8.5-A Branch Target Identification (BTI)
features to the perl scripts.
Both features can be enabled with additional compiler flags.
Unless any of these are enabled explicitly there is no code change at
all.
The extensions are briefly described below. Please read the appropriate
chapters of the Arm Architecture Reference Manual for the complete
specification.
Scope
-----
This change only affects generated assembly code.
Armv8.3-A Pointer Authentication
--------------------------------
Pointer Authentication extension supports the authentication of the
contents of registers before they are used for indirect branching
or load.
PAuth provides a probabilistic method to detect corruption of register
values. PAuth signing instructions generate a Pointer Authentication
Code (PAC) based on the value of a register, a seed and a key.
The generated PAC is inserted into the original value in the register.
A PAuth authentication instruction recomputes the PAC, and if it matches
the PAC in the register, restores its original value. In case of a
mismatch, an architecturally unmapped address is generated instead.
With PAuth, mitigation against ROP (Return-oriented Programming) attacks
can be implemented. This is achieved by signing the contents of the
link-register (LR) before it is pushed to stack. Once LR is popped,
it is authenticated. This way a stack corruption which overwrites the
LR on the stack is detectable.
The PAuth extension adds several new instructions, some of which are not
recognized by older hardware. To support a single codebase for both pre
Armv8.3-A targets and newer ones, only NOP-space instructions are added
by this patch. These instructions are treated as NOPs on hardware
which does not support Armv8.3-A. Furthermore, this patch only considers
cases where LR is saved to the stack and then restored before branching
to its content. There are cases in the code where LR is pushed to stack
but it is not used later. We do not address these cases as they are not
affected by PAuth.
There are two keys available to sign an instruction address: A and B.
PACIASP and PACIBSP only differ in the used keys: A and B, respectively.
The keys are typically managed by the operating system.
To enable generating code for PAuth compile with
-mbranch-protection=<mode>:
- standard or pac-ret: add PACIASP and AUTIASP, also enables BTI
(read below)
- pac-ret+b-key: add PACIBSP and AUTIBSP
Armv8.5-A Branch Target Identification
--------------------------------------
Branch Target Identification features some new instructions which
protect the execution of instructions on guarded pages which are not
intended branch targets.
If Armv8.5-A is supported by the hardware, execution of an instruction
changes the value of PSTATE.BTYPE field. If an indirect branch
lands on a guarded page the target instruction must be one of the
BTI <jc> flavors, or in case of a direct call or jump it can be any
other instruction. If the target instruction is not compatible with the
value of PSTATE.BTYPE a Branch Target Exception is generated.
In short, indirect jumps are compatible with BTI <j> and <jc> while
indirect calls are compatible with BTI <c> and <jc>. Please refer to the
specification for the details.
Armv8.3-A PACIASP and PACIBSP are implicit branch target
identification instructions which are equivalent with BTI c or BTI jc
depending on system register configuration.
BTI is used to mitigate JOP (Jump-oriented Programming) attacks by
limiting the set of instructions which can be jumped to.
BTI requires active linker support to mark the pages with BTI-enabled
code as guarded. For ELF64 files BTI compatibility is recorded in the
.note.gnu.property section. For a shared object or static binary it is
required that all linked units support BTI. This means that even a
single assembly file without the required note section turns-off BTI
for the whole binary or shared object.
The new BTI instructions are treated as NOPs on hardware which does
not support Armv8.5-A or on pages which are not guarded.
To insert this new and optional instruction compile with
-mbranch-protection=standard (also enables PAuth) or +bti.
When targeting a guarded page from a non-guarded page, weaker
compatibility restrictions apply to maintain compatibility between
legacy and new code. For detailed rules please refer to the Arm ARM.
Compiler support
----------------
Compiler support requires understanding '-mbranch-protection=<mode>'
and emitting the appropriate feature macros (__ARM_FEATURE_BTI_DEFAULT
and __ARM_FEATURE_PAC_DEFAULT). The current state is the following:
-------------------------------------------------------
| Compiler | -mbranch-protection | Feature macros |
+----------+---------------------+--------------------+
| clang | 9.0.0 | 11.0.0 |
+----------+---------------------+--------------------+
| gcc | 9 | expected in 10.1+ |
-------------------------------------------------------
Available Platforms
------------------
Arm Fast Model and QEMU support both extensions.
https://developer.arm.com/tools-and-software/simulation-models/fast-modelshttps://www.qemu.org/
Implementation Notes
--------------------
This change adds BTI landing pads even to assembly functions which are
likely to be directly called only. In these cases, landing pads might
be superfluous depending on what code the linker generates.
Code size and performance impact for these cases would be negligible.
Interaction with C code
-----------------------
Pointer Authentication is a per-frame protection while Branch Target
Identification can be turned on and off only for all code pages of a
whole shared object or static binary. Because of these properties if
C/C++ code is compiled without any of the above features but assembly
files support any of them unconditionally there is no incompatibility
between the two.
Useful Links
------------
To fully understand the details of both PAuth and BTI it is advised to
read the related chapters of the Arm Architecture Reference Manual
(Arm ARM):
https://developer.arm.com/documentation/ddi0487/latest/
Additional materials:
"Providing protection for complex software"
https://developer.arm.com/architectures/learn-the-architecture/providing-protection-for-complex-software
Arm Compiler Reference Guide Version 6.14: -mbranch-protection
https://developer.arm.com/documentation/101754/0614/armclang-Reference/armclang-Command-line-Options/-mbranch-protection?lang=en
Arm C Language Extensions (ACLE)
https://developer.arm.com/docs/101028/latest
Addional Notes
--------------
This patch is a copy of the work done by Tamas Petz in boringssl. It
contains the changes from the following commits:
aarch64: support BTI and pointer authentication in assembly
Change-Id: I4335f92e2ccc8e209c7d68a0a79f1acdf3aeb791
URL: https://boringssl-review.googlesource.com/c/boringssl/+/42084
aarch64: Improve conditional compilation
Change-Id: I14902a64e5f403c2b6a117bc9f5fb1a4f4611ebf
URL: https://boringssl-review.googlesource.com/c/boringssl/+/43524
aarch64: Fix name of gnu property note section
Change-Id: I6c432d1c852129e9c273f6469a8b60e3983671ec
URL: https://boringssl-review.googlesource.com/c/boringssl/+/44024
Change-Id: I2d95ebc5e4aeb5610d3b226f9754ee80cf74a9af
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16674)
For functions that exist in 1.1.1 provide a simple aliases via #define.
Fixes#15236
Functions with OSSL_DECODER_, OSSL_ENCODER_, OSSL_STORE_LOADER_,
EVP_KEYEXCH_, EVP_KEM_, EVP_ASYM_CIPHER_, EVP_SIGNATURE_,
EVP_KEYMGMT_, EVP_RAND_, EVP_MAC_, EVP_KDF_, EVP_PKEY_,
EVP_MD_, and EVP_CIPHER_ prefixes are renamed.
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15405)
libimplementations.a was a nice idea, but had a few flaws:
1. The idea to have common code in libimplementations.a and FIPS
sensitive helper functions in libfips.a / libnonfips.a didn't
catch on, and we saw full implementation ending up in them instead
and not appearing in libimplementations.a at all.
2. Because more or less ALL algorithm implementations were included
in libimplementations.a (the idea being that the appropriate
objects from it would be selected automatically by the linker when
building the shared libraries), it's very hard to find only the
implementation source that should go into the FIPS module, with
the result that the FIPS checksum mechanism include source files
that it shouldn't
To mitigate, we drop libimplementations.a, but retain the idea of
collecting implementations in static libraries. With that, we not
have:
libfips.a
Includes all implementations that should become part of the FIPS
provider.
liblegacy.a
Includes all implementations that should become part of the legacy
provider.
libdefault.a
Includes all implementations that should become part of the
default and base providers.
With this, libnonfips.a becomes irrelevant and is dropped.
libcommon.a is retained to include common provider code that can be
used uniformly by all providers.
Fixes#15157
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15171)
The incorrect code is in #ifdef branch that is normally
not compiled in.
Signed-off-by: Zhang Jinde <zjd5536@163.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12968)
This includes error reporting for libcrypto sub-libraries in surprising
places.
This was done using util/err-to-raise
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13318)
Many of the new types introduced by OpenSSL 3.0 have an OSSL_ prefix,
e.g., OSSL_CALLBACK, OSSL_PARAM, OSSL_ALGORITHM, OSSL_SERIALIZER.
The OPENSSL_CTX type stands out a little by using a different prefix.
For consistency reasons, this type is renamed to OSSL_LIB_CTX.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12621)
For example, FreeBSD prepends "FreeBSD" to version string, e.g.,
FreeBSD clang version 11.0.0 (git@github.com:llvm/llvm-project.git llvmorg-11.0.0-rc2-0-g414f32a9e86)
Target: x86_64-unknown-freebsd13.0
Thread model: posix
InstalledDir: /usr/bin
This prevented us from properly detecting AVX support, etc.
CLA: trivial
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/12725)
Fixes#12405Fixes#12377
Calling Init()/Update() and then Init()/Update() again gave a different result when using the same key and iv.
Cipher modes that were using ctx->num were not resetting this value, this includes OFB, CFB & CTR.
The fix is to reset this value during the ciphers einit() and dinit() methods.
Most ciphers go thru a generic method so one line fixes most cases.
Add test for calling EVP_EncryptInit()/EVP_EncryptUpdate() multiple times for all ciphers.
Ciphers should return the same value for both updates.
DES3-WRAP does not since it uses a random in the update.
CCM modes currently also fail on the second update (This also happens in 1_1_1).
Fix memory leak in AES_OCB cipher if EVP_EncryptInit is called multiple times.
Fix AES_SIV cipher dup_ctx and init.
Calling EVP_CIPHER_init multiple times resulted in a memory leak in the siv.
Fixing this leak also showed that the dup ctx was not working for siv mode.
Note: aes_siv_cleanup() can not be used by aes_siv_dupctx() as it clears data
that is required for the decrypt (e.g the tag).
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12413)
The commit claimed to make things more consistent. In fact it makes it
less so. Revert back to the previous namig convention.
This reverts commit d9c2fd51e2.
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12186)
This moves test/ossl_test_endian.h to include/internal/endian.h and
thereby makes the macros in there our standard way to check endianness
in run-time.
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/12390)
functions are now EVP_MAC functions, usually with ctx in their names.
Before 3.0 is released, the names are mutable and this prevents more
inconsistencies being introduced.
There are no functional or code changes.
Just the renaming and a little reformatting.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11997)
A small number of files contain references to the "OpenSSL license"
which has been deprecated and replaced by the "Apache License 2.0".
Amend the occurences.
Fixes#11649
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11663)
In https://github.com/openssl/openssl/pull/10883, I'd meant to exclude
the perlasm drivers since they aren't opening pipes and do not
particularly need it, but I only noticed x86_64-xlate.pl, so
arm-xlate.pl and ppc-xlate.pl got the change.
That seems to have been fine, so be consistent and also apply the change
to x86_64-xlate.pl. Checking for errors is generally a good idea.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: David Benjamin <davidben@google.com>
(Merged from https://github.com/openssl/openssl/pull/10930)
To support Intel CET, all indirect branch targets must start with
endbranch. Here is a patch to add endbranch to function entries
in x86_64 assembly codes which are indirect branch targets as
discovered by running openssl testsuite on Intel CET machine and
visual inspection.
Verified with
$ CC="gcc -Wl,-z,cet-report=error" ./Configure shared linux-x86_64 -fcf-protection
$ make
$ make test
and
$ CC="gcc -mx32 -Wl,-z,cet-report=error" ./Configure shared linux-x32 -fcf-protection
$ make
$ make test # <<< passed with https://github.com/openssl/openssl/pull/10988
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10982)
If one of the perlasm xlate drivers crashes, OpenSSL's build will
currently swallow the error and silently truncate the output to however
far the driver got. This will hopefully fail to build, but better to
check such things.
Handle this by checking for errors when closing STDOUT (which is a pipe
to the xlate driver).
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10883)
Implementations are now spread across several libraries, so the assembler
related defines need to be applied to all affected libraries and modules.
AES_ASM define was missing from libimplementations.a which disabled AESNI
aarch64 changes were made by xkqian.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10180)
We put almost everything in these internal static libraries:
libcommon Block building code that can be used by all
our implementations, legacy and non-legacy
alike.
libimplementations All non-legacy algorithm implementations and
only them. All the code that ends up here is
agnostic to the definitions of FIPS_MODE.
liblegacy All legacy implementations.
libnonfips Support code for the algorithm implementations.
Built with FIPS_MODE undefined. Any code that
checks that FIPS_MODE isn't defined must end
up in this library.
libfips Support code for the algorithm implementations.
Built with FIPS_MODE defined. Any code that
checks that FIPS_MODE is defined must end up
in this library.
The FIPS provider module is built from providers/fips/*.c and linked
with libimplementations, libcommon and libfips.
The Legacy provider module is built from providers/legacy/*.c and
linked with liblegacy, libcommon and libcrypto.
If module building is disabled, the object files from liblegacy and
libcommon are added to libcrypto and the Legacy provider becomes a
built-in provider.
The Default provider module is built-in, so it ends up being linked
with libimplementations, libcommon and libnonfips. For libcrypto in
form of static library, the object files from those other libraries
are simply being added to libcrypto.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10088)
Currently, there are two different directories which contain internal
header files of libcrypto which are meant to be shared internally:
While header files in 'include/internal' are intended to be shared
between libcrypto and libssl, the files in 'crypto/include/internal'
are intended to be shared inside libcrypto only.
To make things complicated, the include search path is set up in such
a way that the directive #include "internal/file.h" could refer to
a file in either of these two directoroes. This makes it necessary
in some cases to add a '_int.h' suffix to some files to resolve this
ambiguity:
#include "internal/file.h" # located in 'include/internal'
#include "internal/file_int.h" # located in 'crypto/include/internal'
This commit moves the private crypto headers from
'crypto/include/internal' to 'include/crypto'
As a result, the include directives become unambiguous
#include "internal/file.h" # located in 'include/internal'
#include "crypto/file.h" # located in 'include/crypto'
hence the superfluous '_int.h' suffixes can be stripped.
The files 'store_int.h' and 'store.h' need to be treated specially;
they are joined into a single file.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9333)
They now generally conform to the following argument sequence:
script.pl "$(PERLASM_SCHEME)" [ C preprocessor arguments ... ] \
$(PROCESSOR) <output file>
However, in the spirit of being able to use these scripts manually,
they also allow for no argument, or for only the flavour, or for only
the output file. This is done by only using the last argument as
output file if it's a file (it has an extension), and only using the
first argument as flavour if it isn't a file (it doesn't have an
extension).
While we're at it, we make all $xlate calls the same, i.e. the $output
argument is always quoted, and we always die on error when trying to
start $xlate.
There's a perl lesson in this, regarding operator priority...
This will always succeed, even when it fails:
open FOO, "something" || die "ERR: $!";
The reason is that '||' has higher priority than list operators (a
function is essentially a list operator and gobbles up everything
following it that isn't lower priority), and since a non-empty string
is always true, so that ends up being exactly the same as:
open FOO, "something";
This, however, will fail if "something" can't be opened:
open FOO, "something" or die "ERR: $!";
The reason is that 'or' has lower priority that list operators,
i.e. it's performed after the 'open' call.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9884)
Since the arguments are now generated in the build file templates,
they should be removed from the build.info files.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9884)
If the passed string length is zero, the function computes the string length
from the passed string.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9760)
Use the header file internal/cryptlib.h instead.
Remove checks for OPENSSL_NO_ASM and I386_ONLY
in cryptlib.c, to match the checks in other
places where OPENSSL_ia32cap_P is used and
assumed to be initialized.
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/9688)
The EVP_PKEY MAC implementations had a diversity of controls that were
really the same thing. We did reproduce that for the provider based
MACs, but are changing our minds on this. Instead of that, we now use
one parameter name for passing the name of the underlying ciphers or
digests to a MAC implementation, "cipher" and "digest", and one
parameter name for passing the output size of the MAC, "size".
Then we leave it to the EVP_PKEY->EVP_MAC bridge to translate "md"
to "digest", and "digestsize" to "size".
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9667)
Add Cleanups for gcm - based on the changes to ccm.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/9280)
The macros are defined in include/openssl/core_names.h and follow the
naming standard OSSL_{OPNAME}_NAME_{ALGONAME}, where {OPNAME} is the
name of the operation (such as MAC) and {ALGONAME} is the name of the
algorithm. Example: OSSL_MAC_NAME_HMAC
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9635)
CRMF, SSKDF, TLS1_PRF and SIV are affected by this.
This also forces the need to check MAC names, which leads to storing
the names in the created methods, which affects all EVP APIs, not just
EVP_MAC. We will want that kind of information anyway (for example
for 'openssl list')... Consequently, EVP_MAC_name() is re-implemented.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8877)
The code has been modularized so that it can be shared by algorithms.
A fixed size IV is now used instead of being allocated.
The IV is not set into the low level struct now until the update (it uses an
iv_state for this purpose).
Hardware specific methods have been added to a PROV_GCM_HW object.
The S390 code has been changed to just contain methods that can be accessed in
a modular way. There are equivalent generic methods also for the other
platforms.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
(Merged from https://github.com/openssl/openssl/pull/9231)
