This patch optimizes SM4 for ARM processor using ASIMD instruction
It will improve performance if both of following conditions are met:
1) Input data equal to or more than 4 blocks
2) Cipher mode allows parallelism, including ECB,CTR,GCM or CBC decryption
This patch implements SM4 SBOX lookup in vector registers, with the
benefit of constant processing time over existing C implementation.
It is only enabled for micro-architecture N1/V1. In the ideal scenario,
performance can reach up to 2.7X
When either of above two conditions is not met, e.g. single block input
or CFB/OFB mode, CBC encryption, performance could drop about 50%.
The assembly code has been reviewed internally by ARM engineer
Fangming.Fang@arm.com
Signed-off-by: Daniel Hu <Daniel.Hu@arm.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17951)
Check the return value of EVP_KDF_fetch to avoid a potential
null pointer dereference.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18062)
This refactors OSSL_LIB_CTX to avoid using CRYPTO_EX_DATA. The assorted
objects to be managed by OSSL_LIB_CTX are hardcoded and are initialized
eagerly rather than lazily, which avoids the need for locking on access
in most cases.
Fixes#17116.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17881)
The sizes are rounded via the expression: (cmpl + 7) / 8 which overflows if
cmpl is near to the type's maximum. Instead we use the safe_math function to
computer this without any possibility of error.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17884)
Fixes#17869.
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17870)
Fixes#17876
CLA: trivial
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17877)
The IV length cache value was being invalidated excessively, causing IV
length caching to be ineffective.
Related to #17064.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17862)
Partial fix for #17064. Avoid excessive writes to the cache line
containing the refcount for an EVP_MD object to avoid extreme
cache contention when using a single EVP_MD at high frequency on
multiple threads. This changes performance in 3.0 from being double
that of 1.1 to only slightly higher than that of 1.1.
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17857)
Instead of doing a heavy params based query every time a context is
asked for its key length, this value is cached in the context and only
queried if it could have been modified.
Fixes#17064
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17543)
Instead of doing a heavy params based query every time a context is asked for
its IV length, this value is cached in the context and only queried if it could
have been modified.
Fixes#17064
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17543)
Including e_os.h with a path from a header file doesn't work well on
certain exotic platform. It simply fails to build.
Since we don't seem to be able to stop ourselves, the better move is
to move e_os.h to an include directory that's part of the inclusion
path given to the compiler.
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17641)
Add copyright to files that were missing it.
Update license from OpenSSL to Apache as needed.
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17606)
This patch implements the SM4 optimization for ARM processor,
using SM4 HW instruction, which is an optional feature of
crypto extension for aarch64 V8.
Tested on some modern ARM micro-architectures with SM4 support, the
performance uplift can be observed around 8X~40X over existing
C implementation in openssl. Algorithms that can be parallelized
(like CTR, ECB, CBC decryption) are on higher end, with algorithm
like CBC encryption on lower end (due to inter-block dependency)
Perf data on Yitian-710 2.75GHz hardware, before and after optimization:
Before:
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
SM4-CTR 105787.80k 107837.87k 108380.84k 108462.08k 108549.46k 108554.92k
SM4-ECB 111924.58k 118173.76k 119776.00k 120093.70k 120264.02k 120274.94k
SM4-CBC 106428.09k 109190.98k 109674.33k 109774.51k 109827.41k 109827.41k
After (7.4x - 36.6x faster):
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
SM4-CTR 781979.02k 2432994.28k 3437753.86k 3834177.88k 3963715.58k 3974556.33k
SM4-ECB 937590.69k 2941689.02k 3945751.81k 4328655.87k 4459181.40k 4468692.31k
SM4-CBC 890639.88k 1027746.58k 1050621.78k 1056696.66k 1058613.93k 1058701.31k
Signed-off-by: Daniel Hu <Daniel.Hu@arm.com>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17455)
PR #17255 fixed a bug in EVP_DigestInit_ex(). While backporting the PR
to 1.1.1 (see #17472) I spotted an error in the original patch. This fixes
it.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17473)
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17471)
If an EVP_MD_CTX is reused then memory allocated and stored in md_data
can be leaked unless the EVP_MD's cleanup function is called.
Fixes#17149
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/17255)
MDs created via EVP_MD_meth_new() are inherently legacy and therefore
need to go down the legacy route when they are used.
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/17255)
EVP_PKEY_eq() assumed that an EVP_PKEY always has the public key
component if it has a private key component. However, this assumption
no longer strictly holds true, at least for provider backed keys.
EVP_PKEY_eq() therefore needs to be modified to specify that the
private key should be checked too (at the discretion of what's
reasonable for the implementation doing the actual comparison).
Fixes#16267
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16765)
When an integer value was specified, it was not being passed back via
the orig_p2 weirdness.
Regression test included.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17136)
Removed the three checks of type against NULL since type cannot be NULL for
any of them.
Moved a check of ->engine inside a CPP guard for engines.
Didn't address the teardown and rebuild of the provider context.
Partially fixes#16947
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16952)
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16943)
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16918)
This mostly entails passing around a provider pointer, and handling
queries that includes a pointer to a provider, where NULL means "any".
This also means that there's a need to pass the provider pointer, not
just down to the cache functions, but also be able to get it from
ossl_method_store_fetch(). To this end, that function's OSSL_PROVIDER
pointer argument is modified to be a pointer reference, so the
function can answer back what provider the method comes from.
Test added.
Fixes#16614
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)
Functions like EVP_PKEY_sign_init() do an implicit fetch of the
operation implementation (EVP_SIGNATURE in this case), then get the
KEYMGMT from the same provider, and tries to export the key there if
necessary.
If an export of the key isn't possible (because the provider that
holds the key is an HSM and therefore can't export), we would simply
fail without looking any further.
This change modifies the behaviour a bit by trying a second fetch of
the operation implementation, but specifically from the provider of
the EVP_PKEY that's being used. This is done with the same properties
that were used with the initial operation implementation fetch, and
should therefore be safe, allowing only what those properties allow.
Fixes#16614
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)
Added functions:
evp_signature_fetch_from_prov(), evp_asym_cipher_fetch_from_prov(),
evp_keyexch_fetch_from_prov(), evp_kem_fetch_from_prov()
These are all like the public conterparts, except they all take a
provider instead of a library context as first argument.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)
In all initializing functions for functionality that use an EVP_PKEY, the
coded logic was to find an KEYMGMT implementation first, and then try to
find the operation method (for example, SIGNATURE implementation) in the
same provider.
This implies that in providers where there is a KEYMGMT implementation,
there must also be a SIGNATURE implementation, along with a KEYEXCH,
ASYM_CIPHER, etc implementation.
The intended design was, however, the opposite implication, i.e. that
where there is a SIGNATURE implementation, there must also be KEYMGMT.
This change reverses the logic of the code to be closer to the intended
design.
There is a consequence; we now use the query_operation_name function from
the KEYMGMT of the EVP_PKEY given by the EVP_PKEY_CTX (ultimately given by
the application). Previously, we used the query_operation_name function
from the KEYMGMT found alongside the SIGNATURE implementation.
Another minor consequence is that the |keymgmt| field in EVP_PKEY_CTX
is now always a reference to the KEYMGMT of the |pkey| field if that
one is given (|pkey| isn't NULL) and is provided (|pkey->keymgmt|
isn't NULL).
Fixes#16614
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)
This is an internal function to fetch a keymgmt method from a specific
provider.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)
This function leverages the generic possibility to fetch EVP methods
from a specific provider.
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)
This makes it possible to limit the search of methods to that
particular provider. This uses already available possibilities in
ossl_algorithm_do_all().
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16725)
