Aes-xts mode can be optimized by interleaving cipher operation on
several blocks and loop unrolling. Interleaving needs one ideal
unrolling factor, here we adopt the same factor with aes-cbc,
which is described as below:
If blocks number > 5, select 5 blocks as one iteration,every
loop, decrease the blocks number by 5.
If left blocks < 5, treat them as tail blocks.
Detailed implementation has a little adjustment for squeezing
code space.
With this way, for small size such as 16 bytes, the performance is
similar as before, but for big size such as 16k bytes, the performance
improves a lot, even reaches to 2x uplift, for some arches such as A57,
the improvement even reaches more than 2x uplift. We collect many
performance datas on different micro-archs such as thunderx2,
ampere-emag, a72, a75, a57, a53 and N1, all of which reach 0.5-2x uplift.
The following table lists the encryption performance data on aarch64,
take a72, a75, a57, a53 and N1 as examples. Performance value takes the
unit of cycles per byte, takes the format as comparision of values.
List them as below:
A72:
Before optimization After optimization Improve
evp-aes-128-xts@16 8.899913518 5.949087263 49.60%
evp-aes-128-xts@64 4.525512668 3.389141845 33.53%
evp-aes-128-xts@256 3.502906908 1.633573479 114.43%
evp-aes-128-xts@1024 3.174210419 1.155952639 174.60%
evp-aes-128-xts@8192 3.053019303 1.028134888 196.95%
evp-aes-128-xts@16384 3.025292462 1.02021169 196.54%
evp-aes-256-xts@16 9.971105023 6.754233758 47.63%
evp-aes-256-xts@64 4.931479093 3.786527393 30.24%
evp-aes-256-xts@256 3.746788153 1.943975947 92.74%
evp-aes-256-xts@1024 3.401743802 1.477394648 130.25%
evp-aes-256-xts@8192 3.278769327 1.32950421 146.62%
evp-aes-256-xts@16384 3.27093296 1.325276257 146.81%
A75:
Before optimization After optimization Improve
evp-aes-128-xts@16 8.397965173 5.126839098 63.80%
evp-aes-128-xts@64 4.176860631 2.59817764 60.76%
evp-aes-128-xts@256 3.069126585 1.284561028 138.92%
evp-aes-128-xts@1024 2.805962699 0.932754655 200.83%
evp-aes-128-xts@8192 2.725820131 0.829820397 228.48%
evp-aes-128-xts@16384 2.71521905 0.823251591 229.82%
evp-aes-256-xts@16 11.24790935 7.383914448 52.33%
evp-aes-256-xts@64 5.294128847 3.048641998 73.66%
evp-aes-256-xts@256 3.861649617 1.570359905 145.91%
evp-aes-256-xts@1024 3.537646797 1.200493533 194.68%
evp-aes-256-xts@8192 3.435353012 1.085345319 216.52%
evp-aes-256-xts@16384 3.437952563 1.097963822 213.12%
A57:
Before optimization After optimization Improve
evp-aes-128-xts@16 10.57455446 7.165438012 47.58%
evp-aes-128-xts@64 5.418185447 3.721241202 45.60%
evp-aes-128-xts@256 3.855184592 1.747145379 120.66%
evp-aes-128-xts@1024 3.477199757 1.253049735 177.50%
evp-aes-128-xts@8192 3.36768104 1.091943159 208.41%
evp-aes-128-xts@16384 3.360373443 1.088942789 208.59%
evp-aes-256-xts@16 12.54559459 8.745489036 43.45%
evp-aes-256-xts@64 6.542808937 4.326387568 51.23%
evp-aes-256-xts@256 4.62668822 2.119908754 118.25%
evp-aes-256-xts@1024 4.161716505 1.557335554 167.23%
evp-aes-256-xts@8192 4.032462227 1.377749511 192.68%
evp-aes-256-xts@16384 4.023293877 1.371558933 193.34%
A53:
Before optimization After optimization Improve
evp-aes-128-xts@16 18.07842135 13.96980808 29.40%
evp-aes-128-xts@64 7.933818397 6.07159276 30.70%
evp-aes-128-xts@256 5.264604704 2.611155744 101.60%
evp-aes-128-xts@1024 4.606660117 1.722713454 167.40%
evp-aes-128-xts@8192 4.405160115 1.454379201 202.90%
evp-aes-128-xts@16384 4.401592028 1.442279392 205.20%
evp-aes-256-xts@16 20.07084054 16.00803726 25.40%
evp-aes-256-xts@64 9.192647294 6.883876732 33.50%
evp-aes-256-xts@256 6.336143161 3.108140452 103.90%
evp-aes-256-xts@1024 5.62502952 2.097960651 168.10%
evp-aes-256-xts@8192 5.412085608 1.807294191 199.50%
evp-aes-256-xts@16384 5.403062591 1.790135764 201.80%
N1:
Before optimization After optimization Improve
evp-aes-128-xts@16 6.48147613 4.209415473 53.98%
evp-aes-128-xts@64 2.847744115 1.950757468 45.98%
evp-aes-128-xts@256 2.085711968 1.061903238 96.41%
evp-aes-128-xts@1024 1.842014669 0.798486302 130.69%
evp-aes-128-xts@8192 1.760449052 0.713853939 146.61%
evp-aes-128-xts@16384 1.760763546 0.707702009 148.80%
evp-aes-256-xts@16 7.264142817 5.265970454 37.94%
evp-aes-256-xts@64 3.251356212 2.41176323 34.81%
evp-aes-256-xts@256 2.380488469 1.342095742 77.37%
evp-aes-256-xts@1024 2.08853022 1.041718215 100.49%
evp-aes-256-xts@8192 2.027432668 0.944571334 114.64%
evp-aes-256-xts@16384 2.00740782 0.941991415 113.10%
Add more XTS test cases to cover the cipher stealing mode and cases of different
number of blocks.
CustomizedGitHooks: yes
Change-Id: I93ee31b2575e1413764e27b599af62994deb4c96
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/11399)
The keydata argument of OSSL_FUNC_keymgmt_validate() should be read-only.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13201)
The following internal functions are affected:
ossl_do_blob_header
ossl_do_PVK_header
ossl_b2i
ossl_b2i_bio
This is reflected by moving include/internal/pem.h to include/crypto/pem.h
engines/e_loader_attic gets the source code added to it to have
continued access to those functions.
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/13195)
The DH private key length, which is an optional parameter, wasn't
properly imported / exported between legacy and provider side
implementations.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13166)
This change makes the naming more consistent, because three different terms
were used for the same thing. (The term libctx was used by far most often.)
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12621)
Many of the new types introduced by OpenSSL 3.0 have an OSSL_ prefix,
e.g., OSSL_CALLBACK, OSSL_PARAM, OSSL_ALGORITHM, OSSL_SERIALIZER.
The OPENSSL_CTX type stands out a little by using a different prefix.
For consistency reasons, this type is renamed to OSSL_LIB_CTX.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12621)
These were previously added as an internal API. But since the CMS code
needs them, other code might do too.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13088)
There is a large amount of CMS sepcific code in the algorithms. This is in
the wrong place and breaks layering. This code should be in the CMS layer.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13088)
There is quite a large amount of algorithm specific CMS code sitting in
the algorithm directories. However, this seems to break layering.
Algorithms really have no business knowing anything about CMS. Really it
should be the other way around. Where there is algorithm specific CMS code
it is the CMS layer that should know how to handle different algorithms.
Therefore we move this code into the CMS layer.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13088)
There is some data that is very difficult to guess. For example, DSA
parameters and X9.42 DH parameters look exactly the same, a SEQUENCE
of 3 INTEGER. Therefore, callers may need the possibility to select
the exact keytype that they expect to get.
This will also allow use to translate d2i_TYPEPrivateKey(),
d2i_TYPEPublicKey() and d2i_TYPEParams() into OSSL_DECODER terms much
more smoothly.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/13061)
We've had explicit checks for when to fall back to legacy code for
operations that use an EVP_PKEY. Unfortunately, the checks were
radically different in different spots, so we refactor that into a
macro that gets used everywhere.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13043)
Automatically rename all instances of _with_libctx() to _ex() as per
our coding style.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12970)
Fixes#12635
As discussed in the issue, supporting the set0-like semantics long-term is not necessarily desirable, although necessary for short-term compatibility concerns. So I've deprecated the original method and added an equivalent that is explicitly labelled as set1.
I tried to audit existing usages of the (now-deprecated) API and update them to use set1 if that appeared to align with their expectations.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12917)
ECX_KEY was not meant for public consumption, it was only to be
accessed indirectly via EVP routines. However, we still need internal
access for our decoders.
This partially reverts 7c664b1f1bFixes#12880
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12956)
rsa_pss_params_30_fromdata() now uses the OSSL_PKEY_PARAM_RSA_DIGEST_PROPS parameter also.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12944)
This permits the default trio of DRBGs to have their type and parameters set
using configuration.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12931)
OSSL_ENCODER was developed before OSSL_DECODER, so the idea of
chaining and the resulting API came later. This series of changes
brings the same sort of API and functionality back to OSSL_ENCODER,
making the two APIs more consistent with each other.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12873)
This was written before the ec key contained a library context,
now that it contains a libctx it can be passed correctly to the callback.
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12877)
SP800-56Br2 requires support for the RSA primitives for RSASVE generate and recover.
As these are simple KEM operations another operation type has been added that can support future extensions.
Added public functions EVP_PKEY_encapsulate_init(), EVP_PKEY_encapsulate(), EVP_PKEY_decapsulate_init() and EVP_PKEY_decapsulate()
Added EVP_KEM_* functions.
Added OSSL_FUNC_kem_* dispatch functions
Added EVP_PKEY_CTX_set_kem_op() so that different types of KEM can be added in the future. This value must currently be set to
"RSASVE" after EVP_PKEY_encapsulate_init() & EVP_PKEY_decapsulate_init() as there is no default value.
This allows the existing RSA key types, keymanagers, and encoders to be used with the encapsulation operations.
The design of the public API's resulted from contributions from @romen & @levitte.
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12750)
This is purely to allow exporting without having to repeatedly specify
the keymgmt and keydata from the EVP_PKEY.
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12853)
add various checks for malformedness to static check_chain_extensions() in x509_vfc.c
improve error reporting of X509v3_cache_extensions() in v3_purp.c
add error reporting to x509_init_sig_info() in x509_set.c
improve static setup_dp() and related functions in v3_purp.c and v3_crld.c
add test case for non-conforming cert from https://tools.ietf.org/html/rfc8410#section-10.2
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12478)
If we initialise an EVP_MD_CTX with a legacy MD, and then reuse the same
EVP_MD_CTX with a provided MD then we end up leaking the md_data.
We need to ensure we free the md_data if we change to a provided MD.
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12779)
PEM_write_bio_PrivateKey_traditional() didn't handle provider-native
keys very well. Originally, it would simply use the corresponding
encoder, which is likely to output modern PEM (not "traditional").
PEM_write_bio_PrivateKey_traditional() is now changed to try and get a
legacy copy of the input EVP_PKEY, and use that copy for traditional
output, if it has such support.
Internally, evp_pkey_copy_downgraded() is added, to be used when
evp_pkey_downgrade() is too intrusive for what it's needed for.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12738)
Add the AuthEnvelopedData as defined in RFC 5083 with AES-GCM
parameter as defined in RFC 5084.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/8024)
They get called "delayed parameters" because they may make it to the
implementation at a later time than when they're given.
This currently only covers the distinguished ID, as that's the only
EVP_PKEY operation parameter so far that has been possible to give
before the operation has been initialized.
This includes a re-implementation of EVP_PKEY_CTX_set1_id(),
EVP_PKEY_CTX_get1_id(), and EVP_PKEY_CTX_get1_id_len().
Also, the more rigorous controls of keytype and optype are restored.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12789)
As long as there are internal legacy keys for EVP_PKEY, we need to preserve
the EVP_PKEY numeric identity when generating a key, and when creating the
EVP_PKEY_CTX.
For added consistency, the EVP_PKEY_CTX contructor tries a little
harder to find a EVP_PKEY_METHOD. Otherwise, we may run into
situations where the EVP_PKEY_CTX ends up having no associated methods
at all.
Reviewed-by: Paul Yang <kaishen.yy@antfin.com>
(Merged from https://github.com/openssl/openssl/pull/12785)
This replaces the older 'file:' loader that is now an engine.
It's still possible to use the older 'file:' loader by explicitly
using the engine, and tests will remain for it as long as ENGINEs are
still supported (even through deprecated).
To support this storemgmt implementation, a few internal OSSL_DECODER
modifications are needed:
- An internal function that implements most of
OSSL_DECODER_CTX_new_by_EVP_PKEY(), but operates on an already
existing OSSL_DECODER_CTX instead of allocating a new one.
- Allow direct creation of a OSSL_DECODER from an OSSL_ALGORITHM.
It isn't attached to any provider, and is only used internally, to
simply catch any DER encoded object to be passed back to the
object callback with no further checking. This implementation
becomes the last resort decoder, when all "normal"
decodation attempts (i.e. those that are supposed to result
in an OpenSSL object of some sort) have failed.
Because file_store_attach() uses BIO_tell(), we must also support
BIO_ctrl() as a libcrypto upcall.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12587)
From this point on, this engine must be specifically specified.
To replace the internal EMBEDDED hack with something unique for the
new module, functions to create application specific OSSL_STORE_INFO
types were added.
Furthermore, the following function had to be exported:
ossl_do_blob_header()
ossl_do_PVK_header()
asn1_d2i_read_bio()
Finally, evp_pkcs82pkey_int() has become public under a new name,
EVP_PKCS82PKEY_with_libctx()
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12587)
Now that the all the legacy PKEY MAC bridge code has been moved to the
providers we no longer need the old bridge and it can be removed.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12637)
This adds the needed code to make the OSSL_STORE API functions handle
provided STORE implementations.
This also modifies OSSL_STORE_attach() for have the URI, the
library context and the properties in the same order as
OSSL_STORE_open_with_libctx().
The most notable change, though, is how this creates a division of
labor between libcrypto and any storemgmt implementation that wants to
pass X.509, X.509 CRL, etc structures back to libcrypto. Since those
structures aren't directly supported in the libcrypto <-> provider
interface (asymmetric keys being the only exception so far), we resort
to a libcrypto object callback that can handle passed data in DER form
and does its part of figuring out what the DER content actually is.
This also adds the internal x509_crl_set0_libctx(), which works just
like x509_set0_libctx(), but for X509_CRL.
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12512)
This was added for backward compatability.
Added EC_GROUP_new_from_params() that supports explicit curve parameters.
This fixes the 15-test_genec.t TODO.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/12604)
dsa_algorithmidentifier_encoding(), ecdsa_algorithmidentifier_encoding(),
rsa_algorithmidentifier_encoding() have been replaced with DER writer
functions, so they aren't useful any more.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12693)
Just like d2i_PrivateKey() / d2i_PrivateKey_ex(), there's a need to
associate an EVP_PKEY extracted from a PUBKEY to a library context and
a property query string. Without it, a provider-native EVP_PKEY can
only fetch necessary internal algorithms from the default library
context, even though an application specific context should be used.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12671)
A config file can change the global default properties. Therefore we
must ensure that the config file is loaded before reading or amending
them.
Fixes#12565
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/12567)
Inline the pre-13273237a65d46186b6bea0b51aec90670d4598a versions
of EVP_CIPHER_CTX_iv(), EVP_CIPHER_CTX_original_iv(), and
EVP_CIPHER_CTX_iv_noconst() in evp.h.
These macros are internal-only, used to implement legacy libcrypto
EVP ciphers, with no real provider involvement. Accordingly, just use the
EVP_CIPHER_CTX storage directly and don't try to reach into a provider-side
context.
This does necessitate including evp_local.h in several more files.
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/12233)
