Commit Graph

123 Commits

Author SHA1 Message Date
Min Zhou
9a41a3c6a4 LoongArch64 assembly pack: add ChaCha20 modules
This assembly implementation for ChaCha20 includes three code paths:
scalar path, 128-bit LSX path and 256-bit LASX path. We prefer the
LASX path or LSX path if the hardware and system support these
extensions.

There are 32 vector registers avaialable in the LSX and LASX
extensions. So, we can load the 16 initial states and the 16
intermediate states of ChaCha into the 32 vector registers for
calculating in the implementation. The test results on the 3A5000
and 3A6000 show that this assembly implementation significantly
improves the performance of ChaCha20 on LoongArch based machines.
The detailed test results are as following.

Test with:
$ openssl speed -evp chacha20

3A5000
type               16 bytes     64 bytes    256 bytes    1024 bytes    8192 bytes   16384 bytes
C code           178484.53k   282789.93k   311793.70k    322234.99k    324405.93k    324659.88k
assembly code    223152.28k   407863.65k   989520.55k   2049192.96k   2127248.70k   2131749.55k
                   +25%         +44%         +217%        +536%         +556%         +557%

3A6000
type               16 bytes     64 bytes     256 bytes    1024 bytes    8192 bytes   16384 bytes
C code           214945.33k   310041.75k    340724.22k    349949.27k    352925.01k    353140.74k
assembly code    299151.34k   492766.34k   2070166.02k   4300909.91k   4473978.88k   4499084.63k
                   +39%         +59%         +508%         +1129%        +1168%        +1174%

Signed-off-by: Min Zhou <zhoumin@loongson.cn>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21998)
2023-09-11 08:49:09 +10:00
Matt Caswell
da1c088f59 Copyright year updates
Reviewed-by: Richard Levitte <levitte@openssl.org>
Release: yes
2023-09-07 09:59:15 +01:00
Bernd Edlinger
6d38ccedb2 Fix ChaCha assembly code on 32-bit HPUX itanium systems
This fixes the reported crashes 32-bit HPUX systems due to
raw out and inp pointer values, and adds one nop instruction
on 64-bit systems, like it is done in other assembly modules
for those systems.

The fix was tested by @johnkohl-hcl see:
https://github.com/openssl/openssl/issues/17067#issuecomment-1668468033

Fixes #17067

Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/21681)
2023-08-09 12:22:20 +02:00
Tom Cosgrove
7b508cd1e1 Ensure there's only one copy of OPENSSL_armcap_P in libcrypto.a
Change-Id: Ia94e528a2d55934435de6a2949784c52eb38d82f

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/20621)
2023-03-29 12:21:31 +02:00
Evan Miller
175645a1a6 Do not build P10-specific AES-GCM assembler on macOS
Reviewed-by: Tom Cosgrove <tom.cosgrove@arm.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/20543)
2023-03-22 14:26:26 +01:00
fangming.fang
cd7a8e96bb Fix big-endian issue in chacha20 SVE implementation on aarch64
Fixes: #19902

Reviewed-by: Todd Short <todd.short@me.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/20028)
2023-01-16 17:03:34 +01:00
Tomas Mraz
abfc152126 Do not build P10-specific Chacha20 assembler on AIX
Fixes #18145

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/19865)

(cherry picked from commit cdcc439aa0)
2022-12-14 12:53:00 +01:00
Daniel Hu
3f42f41ad1 Improve chacha20 perfomance on aarch64 by interleaving scalar with SVE/SVE2
The patch will process one extra block by scalar in addition to
blocks by SVE/SVE2 in parallel. This is esp. helpful in the
scenario where we only have 128-bit vector length.

The actual uplift to performance is complicated, depending on the
vector length and input data size. SVE/SVE2 implementation don't
always perform better than  Neon, but it should prevail in most
cases

On a CPU with 256-bit SVE/SVE2, interleaved processing can
handle 9 blocks in parallel (8 blocks by SVE and 1 by Scalar).
on 128-bit SVE/SVE2 it is 5 blocks. Input size that is a multiple
of 9/5 blocks on respective CPU can be typically handled at
maximum speed.

Here are test data for 256-bit and 128-bit SVE/SVE2 by running
"openssl speed -evp chacha20 -bytes 576" (and other size)

----------------------------------+---------------------------------
                256-bit SVE       |        128-bit SVE2
----------------------------------|---------------------------------
Input  576 bytes     512 bytes    |  320 bytes        256 bytes
----------------------------------|---------------------------------
SVE    1716361.91k   1556699.18k  |  1615789.06k      1302864.40k
----------------------------------|---------------------------------
Neon   1262643.44k   1509044.05k  |  680075.67k       1060532.31k
----------------------------------+---------------------------------

If the input size gets very large, the advantage of SVE/SVE2 over
Neon will fade out.

Signed-off-by: Daniel Hu <Daniel.Hu@arm.com>
Change-Id: Ieedfcb767b9c08280d7c8c9a8648919c69728fab

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18901)
2022-09-01 18:01:19 +10:00
Hongren (Zenithal) Zheng
ca6286c382 Add ROTATE inline RISC-V zbb/zbkb asm for chacha
Reviewed-by: Hugo Landau <hlandau@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18289)
2022-07-13 18:15:12 +01:00
Daniel Hu
bcb52bcc9f Optimize chacha20 on aarch64 by SVE2
This patch improves existing chacha20 SVE patch by using SVE2,
which is an optional architecture feature of aarch64, with XAR
instruction that can improve the performance of chacha20.

Signed-off-by: Daniel Hu <Daniel.Hu@arm.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18522)
2022-06-22 17:07:17 +10:00
Tomas Mraz
db24ed5430 Generate the preprocessed .s files for chacha and poly 1305 on ia64
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18263)
2022-05-27 08:10:49 +02:00
Tomas Mraz
debd921006 Revert "Use .s extension for ia64 assembler"
This reverts commit 6009997abd.

The .s extension is incorrect as the assembler files contain
preprocessor directives.

Fixes #18259

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18263)
2022-05-27 08:10:49 +02:00
Sebastian Andrzej Siewior
9968c77539 Rename x86-32 assembly files from .s to .S.
Rename x86-32 assembly files from .s to .S. While processing the .S file
gcc will use the pre-processor whic will evaluate macros and ifdef. This
is turn will be used to enable the endbr32 opcode based on the __CET__
define.

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18353)
2022-05-24 13:16:06 +10:00
Matt Caswell
fecb3aae22 Update copyright year
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Release: yes
2022-05-03 13:34:51 +01:00
Daniel Hu
b1b2146ded Acceleration of chacha20 on aarch64 by SVE
This patch accelerates chacha20 on aarch64 when Scalable Vector Extension
(SVE) is supported by CPU. Tested on modern micro-architecture with
256-bit SVE, it has the potential to improve performance up to 20%

The solution takes a hybrid approach. SVE will handle multi-blocks that fit
the SVE vector length, with Neon/Scalar to process any tail data

Test result:
With SVE
type            1024 bytes   8192 bytes  16384 bytes
ChaCha20        1596208.13k  1650010.79k  1653151.06k

Without SVE (by Neon/Scalar)
type            1024 bytes   8192 bytes  16384 bytes
chacha20        1355487.91k  1372678.83k  1372662.44k

The assembly code has been reviewed internally by
ARM engineer Fangming.Fang@arm.com

Signed-off-by: Daniel Hu <Daniel.Hu@arm.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17916)
2022-05-03 14:37:46 +10:00
Jon Spillett
6009997abd Use .s extension for ia64 assembler
Reviewed-by: Shane Lontis <shane.lontis@oracle.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/18136)
2022-04-25 14:04:57 +02:00
Danny Tsen
7e1f3ffcc5 Fixed conditional statement testing 64 and 256 bytes
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/17760)
2022-03-14 09:36:24 +01:00
Deepankar Bhattacharjee
f596bbe4da chacha20 performance optimizations for ppc64le with 8x lanes,
Performance increase around 50%.

Co-authored-by: Madhusudhanan Duraisamy <madurais@in.ibm.com>

Co-authored-by: Nilamjyoti Goswami <nilamgoswami@in.ibm.com>

Co-authored-by: Siva Sundar Anbareeswaran <srisivasundar@in.ibm.com>

Reviewed-by: Danny Tsen <dtsen@us.ibm.com>
Tested-by: Danny Tsen <dtsen@us.ibm.com>
Signed-off-by: Danny <dtsen@us.ibm.com>

Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16637)
2022-02-22 16:58:55 +11:00
Russ Butler
19e277dd19 aarch64: support BTI and pointer authentication in assembly
This change adds optional support for
- Armv8.3-A Pointer Authentication (PAuth) and
- Armv8.5-A Branch Target Identification (BTI)
features to the perl scripts.

Both features can be enabled with additional compiler flags.
Unless any of these are enabled explicitly there is no code change at
all.

The extensions are briefly described below. Please read the appropriate
chapters of the Arm Architecture Reference Manual for the complete
specification.

Scope
-----

This change only affects generated assembly code.

Armv8.3-A Pointer Authentication
--------------------------------

Pointer Authentication extension supports the authentication of the
contents of registers before they are used for indirect branching
or load.

PAuth provides a probabilistic method to detect corruption of register
values. PAuth signing instructions generate a Pointer Authentication
Code (PAC) based on the value of a register, a seed and a key.
The generated PAC is inserted into the original value in the register.
A PAuth authentication instruction recomputes the PAC, and if it matches
the PAC in the register, restores its original value. In case of a
mismatch, an architecturally unmapped address is generated instead.

With PAuth, mitigation against ROP (Return-oriented Programming) attacks
can be implemented. This is achieved by signing the contents of the
link-register (LR) before it is pushed to stack. Once LR is popped,
it is authenticated. This way a stack corruption which overwrites the
LR on the stack is detectable.

The PAuth extension adds several new instructions, some of which are not
recognized by older hardware. To support a single codebase for both pre
Armv8.3-A targets and newer ones, only NOP-space instructions are added
by this patch. These instructions are treated as NOPs on hardware
which does not support Armv8.3-A. Furthermore, this patch only considers
cases where LR is saved to the stack and then restored before branching
to its content. There are cases in the code where LR is pushed to stack
but it is not used later. We do not address these cases as they are not
affected by PAuth.

There are two keys available to sign an instruction address: A and B.
PACIASP and PACIBSP only differ in the used keys: A and B, respectively.
The keys are typically managed by the operating system.

To enable generating code for PAuth compile with
-mbranch-protection=<mode>:

- standard or pac-ret: add PACIASP and AUTIASP, also enables BTI
  (read below)
- pac-ret+b-key: add PACIBSP and AUTIBSP

Armv8.5-A Branch Target Identification
--------------------------------------

Branch Target Identification features some new instructions which
protect the execution of instructions on guarded pages which are not
intended branch targets.

If Armv8.5-A is supported by the hardware, execution of an instruction
changes the value of PSTATE.BTYPE field. If an indirect branch
lands on a guarded page the target instruction must be one of the
BTI <jc> flavors, or in case of a direct call or jump it can be any
other instruction. If the target instruction is not compatible with the
value of PSTATE.BTYPE a Branch Target Exception is generated.

In short, indirect jumps are compatible with BTI <j> and <jc> while
indirect calls are compatible with BTI <c> and <jc>. Please refer to the
specification for the details.

Armv8.3-A PACIASP and PACIBSP are implicit branch target
identification instructions which are equivalent with BTI c or BTI jc
depending on system register configuration.

BTI is used to mitigate JOP (Jump-oriented Programming) attacks by
limiting the set of instructions which can be jumped to.

BTI requires active linker support to mark the pages with BTI-enabled
code as guarded. For ELF64 files BTI compatibility is recorded in the
.note.gnu.property section. For a shared object or static binary it is
required that all linked units support BTI. This means that even a
single assembly file without the required note section turns-off BTI
for the whole binary or shared object.

The new BTI instructions are treated as NOPs on hardware which does
not support Armv8.5-A or on pages which are not guarded.

To insert this new and optional instruction compile with
-mbranch-protection=standard (also enables PAuth) or +bti.

When targeting a guarded page from a non-guarded page, weaker
compatibility restrictions apply to maintain compatibility between
legacy and new code. For detailed rules please refer to the Arm ARM.

Compiler support
----------------

Compiler support requires understanding '-mbranch-protection=<mode>'
and emitting the appropriate feature macros (__ARM_FEATURE_BTI_DEFAULT
and __ARM_FEATURE_PAC_DEFAULT). The current state is the following:

-------------------------------------------------------
| Compiler | -mbranch-protection | Feature macros     |
+----------+---------------------+--------------------+
| clang    | 9.0.0               | 11.0.0             |
+----------+---------------------+--------------------+
| gcc      | 9                   | expected in 10.1+  |
-------------------------------------------------------

Available Platforms
------------------

Arm Fast Model and QEMU support both extensions.

https://developer.arm.com/tools-and-software/simulation-models/fast-models
https://www.qemu.org/

Implementation Notes
--------------------

This change adds BTI landing pads even to assembly functions which are
likely to be directly called only. In these cases, landing pads might
be superfluous depending on what code the linker generates.
Code size and performance impact for these cases would be negligible.

Interaction with C code
-----------------------

Pointer Authentication is a per-frame protection while Branch Target
Identification can be turned on and off only for all code pages of a
whole shared object or static binary. Because of these properties if
C/C++ code is compiled without any of the above features but assembly
files support any of them unconditionally there is no incompatibility
between the two.

Useful Links
------------

To fully understand the details of both PAuth and BTI it is advised to
read the related chapters of the Arm Architecture Reference Manual
(Arm ARM):
https://developer.arm.com/documentation/ddi0487/latest/

Additional materials:

"Providing protection for complex software"
https://developer.arm.com/architectures/learn-the-architecture/providing-protection-for-complex-software

Arm Compiler Reference Guide Version 6.14: -mbranch-protection
https://developer.arm.com/documentation/101754/0614/armclang-Reference/armclang-Command-line-Options/-mbranch-protection?lang=en

Arm C Language Extensions (ACLE)
https://developer.arm.com/docs/101028/latest

Addional Notes
--------------

This patch is a copy of the work done by Tamas Petz in boringssl. It
contains the changes from the following commits:

aarch64: support BTI and pointer authentication in assembly
    Change-Id: I4335f92e2ccc8e209c7d68a0a79f1acdf3aeb791
    URL: https://boringssl-review.googlesource.com/c/boringssl/+/42084
aarch64: Improve conditional compilation
    Change-Id: I14902a64e5f403c2b6a117bc9f5fb1a4f4611ebf
    URL: https://boringssl-review.googlesource.com/c/boringssl/+/43524
aarch64: Fix name of gnu property note section
    Change-Id: I6c432d1c852129e9c273f6469a8b60e3983671ec
    URL: https://boringssl-review.googlesource.com/c/boringssl/+/44024

Change-Id: I2d95ebc5e4aeb5610d3b226f9754ee80cf74a9af

Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16674)
2021-10-01 09:35:38 +02:00
Tomas Mraz
3d178db73b ppccap.c: Split out algorithm-specific functions
Fixes #13336

Reviewed-by: Paul Dale <pauli@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15828)
2021-06-25 08:49:45 +01:00
Matt Caswell
38fc02a708 Update copyright year
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15801)
2021-06-17 13:24:59 +01:00
Jean-Philippe Boivin
6d3f798cba Properly restore XMM registers in ChaCha20's AVX-512(VL) assembly
Reviewed-by: Paul Dale <pauli@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/15315)
2021-05-21 13:05:18 +02:00
Romain Geissler
a135948dda Fix aarch64 static linking into shared libraries (see issue #10842 and pull request #11464)
This tries to fix the following link errors on aarch64 when using OpenSSL
3.0.0 alpha 6, compiling it with "no-shared" and -fPIC in CFLAGS, then
trying to use the resulting OpenSSL static libraries in the build of
elfutils, which embed libcrypto.a into libdebuginfo.so, which hides all
symbols (except the libdebuginfod ones) by default:

/opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha1-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
/workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha1-armv8.o): in function `sha1_block_data_order':
(.text+0x0): dangerous relocation: unsupported relocation
/opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-chacha-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
/workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-chacha-armv8.o): in function `ChaCha20_ctr32':
(.text+0x6c): dangerous relocation: unsupported relocation
/opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha256-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha256-armv8.o): in function `sha256_block_data_order':
(.text+0x0): dangerous relocation: unsupported relocation
/opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha512-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-sha512-armv8.o): in function `sha512_block_data_order':
(.text+0x0): dangerous relocation: unsupported relocation
/opt/1A/toolchain/aarch64-v4.0.86/lib/gcc/aarch64-1a-linux-gnu/8.4.1/../../../../aarch64-1a-linux-gnu/bin/ld: /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-poly1305-armv8.o): relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `OPENSSL_armcap_P' which may bind externally can not be used when making a shared object; recompile with -fPIC
/workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-poly1305-armv8.o): in function `poly1305_init':
(.text+0x14): dangerous relocation: unsupported relocation
/workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-poly1305-armv8.o): in function `poly1305_emit_neon':
(.text+0x8e4): relocation truncated to fit: R_AARCH64_CONDBR19 against symbol `poly1305_emit' defined in .text section in /workdir/build/build-pack/build-pack-temporary-static-dependencies/install/lib/libcrypto.a(libcrypto-lib-poly1305-armv8.o)

In poly1305-armv8.pl, hide symbols the same way they are hidden in poly1305-x86_64.pl.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/13056)
2020-10-22 12:16:49 +10:00
Jung-uk Kim
cd84d8832d Ignore vendor name in Clang version number.
For example, FreeBSD prepends "FreeBSD" to version string, e.g.,

FreeBSD clang version 11.0.0 (git@github.com:llvm/llvm-project.git llvmorg-11.0.0-rc2-0-g414f32a9e86)
Target: x86_64-unknown-freebsd13.0
Thread model: posix
InstalledDir: /usr/bin

This prevented us from properly detecting AVX support, etc.

CLA: trivial

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Ben Kaduk <kaduk@mit.edu>
(Merged from https://github.com/openssl/openssl/pull/12725)
2020-08-27 20:27:26 -07:00
Richard Levitte
e39e295e20 Update copyright year
Reviewed-by: Nicola Tuveri <nic.tuv@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/12463)
2020-07-16 14:47:04 +02:00
Richard Levitte
e23d850ff3 Add and use internal header that implements endianness check
This moves test/ossl_test_endian.h to include/internal/endian.h and
thereby makes the macros in there our standard way to check endianness
in run-time.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/12390)
2020-07-11 10:00:33 +02:00
Matt Caswell
33388b44b6 Update copyright year
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/11616)
2020-04-23 13:55:52 +01:00
David Benjamin
a21314dbbc Also check for errors in x86_64-xlate.pl.
In https://github.com/openssl/openssl/pull/10883, I'd meant to exclude
the perlasm drivers since they aren't opening pipes and do not
particularly need it, but I only noticed x86_64-xlate.pl, so
arm-xlate.pl and ppc-xlate.pl got the change.

That seems to have been fine, so be consistent and also apply the change
to x86_64-xlate.pl. Checking for errors is generally a good idea.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: David Benjamin <davidben@google.com>
(Merged from https://github.com/openssl/openssl/pull/10930)
2020-02-17 12:17:53 +10:00
David Benjamin
32be631ca1 Do not silently truncate files on perlasm errors
If one of the perlasm xlate drivers crashes, OpenSSL's build will
currently swallow the error and silently truncate the output to however
far the driver got. This will hopefully fail to build, but better to
check such things.

Handle this by checking for errors when closing STDOUT (which is a pipe
to the xlate driver).

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from https://github.com/openssl/openssl/pull/10883)
2020-01-22 18:11:30 +01:00
Richard Levitte
9bb3e5fd87 For all assembler scripts where it matters, recognise clang > 9.x
Fixes #10853

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/10855)
2020-01-17 08:55:45 +01:00
Ard Biesheuvel
07a470729c chacha/asm/chacha-armv8.pl: preserve FP registers d8 and d9 correctly
Depending on the size of the input, we may take different paths through
the accelerated arm64 ChaCha20 routines, each of which use a different
subset of the FP registers, some of which need to be preserved and
restored, as required by the AArch64 calling convention (AAPCS64)

In some cases, (e.g., when the input size is 640 bytes), we call the 512
byte NEON path followed directly by the scalar path, and in this case,
we preserve and restore d8 and d9, only to clobber them again
immediately before handing over to the scalar path which does not touch
the FP registers at all, and hence does not restore them either.

Fix this by moving the restoration of d8 and d9 to a later stage in the
512 byte routine, either before calling the scalar path, or when exiting
the function.

Fixes #10470
CLA: trivial

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10497)
2019-11-28 16:40:43 +01:00
Joerg Schmidbauer
c31950b964 chacha_enc.c: fix for EBCDIC platforms
Signed-off-by: Joerg Schmidbauer <jschmidb@de.ibm.com>

Reviewed-by: Patrick Steuer <patrick.steuer@de.ibm.com>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10417)
2019-11-13 17:59:48 +01:00
Patrick Steuer
826112295a s390x assembly pack: perlasm module update
- add instructions: clfi, stck, stckf, kdsa
- clfi and clgfi belong to extended-immediate (not long-displacement)
- some cleanup

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/10346)
2019-11-05 10:05:27 +01:00
Dr. Matthias St. Pierre
25f2138b0a Reorganize private crypto header files
Currently, there are two different directories which contain internal
header files of libcrypto which are meant to be shared internally:

While header files in 'include/internal' are intended to be shared
between libcrypto and libssl, the files in 'crypto/include/internal'
are intended to be shared inside libcrypto only.

To make things complicated, the include search path is set up in such
a way that the directive #include "internal/file.h" could refer to
a file in either of these two directoroes. This makes it necessary
in some cases to add a '_int.h' suffix to some files to resolve this
ambiguity:

  #include "internal/file.h"      # located in 'include/internal'
  #include "internal/file_int.h"  # located in 'crypto/include/internal'

This commit moves the private crypto headers from

  'crypto/include/internal'  to  'include/crypto'

As a result, the include directives become unambiguous

  #include "internal/file.h"       # located in 'include/internal'
  #include "crypto/file.h"         # located in 'include/crypto'

hence the superfluous '_int.h' suffixes can be stripped.

The files 'store_int.h' and 'store.h' need to be treated specially;
they are joined into a single file.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9333)
2019-09-28 20:26:34 +02:00
Richard Levitte
1aa89a7a3a Unify all assembler file generators
They now generally conform to the following argument sequence:

    script.pl "$(PERLASM_SCHEME)" [ C preprocessor arguments ... ] \
              $(PROCESSOR) <output file>

However, in the spirit of being able to use these scripts manually,
they also allow for no argument, or for only the flavour, or for only
the output file.  This is done by only using the last argument as
output file if it's a file (it has an extension), and only using the
first argument as flavour if it isn't a file (it doesn't have an
extension).

While we're at it, we make all $xlate calls the same, i.e. the $output
argument is always quoted, and we always die on error when trying to
start $xlate.

There's a perl lesson in this, regarding operator priority...

This will always succeed, even when it fails:

    open FOO, "something" || die "ERR: $!";

The reason is that '||' has higher priority than list operators (a
function is essentially a list operator and gobbles up everything
following it that isn't lower priority), and since a non-empty string
is always true, so that ends up being exactly the same as:

    open FOO, "something";

This, however, will fail if "something" can't be opened:

    open FOO, "something" or die "ERR: $!";

The reason is that 'or' has lower priority that list operators,
i.e. it's performed after the 'open' call.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9884)
2019-09-16 16:29:57 +02:00
Richard Levitte
a1c8befd66 build.info: For all assembler generators, remove all arguments
Since the arguments are now generated in the build file templates,
they should be removed from the build.info files.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9884)
2019-09-16 16:29:57 +02:00
Richard Levitte
bcb7afe18a Move chacha_asm_src file information to build.info files
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9166)
2019-06-17 16:08:53 +02:00
Patrick Steuer
302aa3c26d s390x assembly pack: remove chacha20 dependency on non-base memnonics
Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8181)
2019-04-25 23:07:36 +02:00
Andy Polyakov
86a7ac5e76 chacha/asm/chacha-armv8.pl: replace 3+1 code paths with 4+1.
The change is triggered by ThunderX2 where 3+1 was slower than scalar
code path, but it helps all processors [to handle <512 inputs].

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8776)
2019-04-17 21:31:01 +02:00
Andy Polyakov
6465321e40 ARM64 assembly pack: add ThunderX2 results.
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8776)
2019-04-17 21:08:13 +02:00
Andy Polyakov
291bc802e4 IA64 assembly pack: add {chacha|poly1305}-ia64 modules.
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8540)
2019-03-29 07:33:15 +01:00
Patrick Steuer
d1229190bf s390x assembly pack: import chacha from cryptogams repo
featuring 6x"horizontal" code path which is up to 25%
faster than present 4x"vertical" for larger blocks.

Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8287)
2019-03-11 21:22:28 +01:00
Andy Polyakov
db42bb440e ARM64 assembly pack: make it Windows-friendly.
"Windows friendliness" means a) unified PIC-ification, unified across
all platforms; b) unified commantary delimiter; c) explicit ldur/stur,
as Visual Studio assembler can't automatically encode ldr/str as
ldur/stur when needed.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8256)
2019-02-16 17:01:15 +01:00
Andy Polyakov
3405db97e5 ARM assembly pack: make it Windows-friendly.
"Windows friendliness" means a) flipping .thumb and .text directives,
b) always generate Thumb-2 code when asked(*); c) Windows-specific
references to external OPENSSL_armcap_P.

(*) so far *some* modules were compiled as .code 32 even if Thumb-2
was targeted. It works at hardware level because processor can alternate
between the modes with no overhead. But clang --target=arm-windows's
builtin assembler just refuses to compile .code 32...

Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8252)
2019-02-16 16:59:23 +01:00
Patrick Steuer
b2b580fe44 s390x assembly pack: fix formal interface bug in chacha module
Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8257)
2019-02-16 12:47:21 +01:00
Andy Polyakov
9a18aae5f2 AArch64 assembly pack: authenticate return addresses.
ARMv8.3 adds pointer authentication extension, which in this case allows
to ensure that, when offloaded to stack, return address is same at return
as at entry to the subroutine. The new instructions are nops on processors
that don't implement the extension, so that the vetification is backward
compatible.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8205)
2019-02-12 19:00:42 +01:00
Richard Levitte
77adb75e16 Build: Remove BEGINRAW / ENDRAW / OVERRIDE
It was an ugly hack to avoid certain problems that are no more.

Also added GENERATE lines for perlasm scripts that didn't have that
explicitly.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8125)
2019-01-31 16:19:49 +01:00
Patrick Steuer
f760137b21 crypto/chacha/asm/chacha-s390x.pl: add vx code path.
Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com>

Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6919)
2019-01-05 09:40:35 +01:00
Richard Levitte
03d770d935 Following the license change, modify the boilerplates in crypto/chacha/
[skip ci]

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7781)
2018-12-06 14:37:22 +01:00
Andy Polyakov
d5487a454c chacha/asm/chacha-x86_64.pl: add dedicated path for 128-byte inputs.
The 128-byte vectors are extensively used in chacha20_poly1305_tls_cipher
and dedicated code path is ~30-50% faster on most platforms.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6626)
2018-07-03 19:02:02 +02:00