Wire asprintf into test infrastructure for formatted printf output
specifiers.
Owing to mtrace logging of lots of memory allocation calls these tests
take a considerable amount of time to complete, except for the character
conversion, taking from 00m20s for 'tst-printf-format-as-s --direct s',
through 01m10s and 03m53s for 'tst-printf-format-as-char --direct i' and
'tst-printf-format-as-double --direct f' respectively, to 19m24s for
'tst-printf-format-as-ldouble --direct f', all in standalone execution
from NFS on a RISC-V FU740@1.2GHz system and with output redirected over
100Mbps network via SSH. It is with the skeleton's stub implementation
of dladdr(3); execution times with regular dladdr(3) are up to over
twice longer.
Set timeouts for the tests accordingly then, with a global default for
all the asprintf tests, and then individual higher settings for double
and long double tests each.
Reviewed-by: DJ Delorie <dj@redhat.com>
This is a collection of tests for formatted printf output specifiers
covering the d, i, o, u, x, and X integer conversions, the e, E, f, F,
g, and G floating-point conversions, the c character conversion, and the
s string conversion. Also the hh, h, l, and ll length modifiers are
covered with the integer conversions as is the L length modifier with
the floating-point conversions.
The -, +, space, #, and 0 flags are iterated over, as permitted by the
conversion handled, in tuples of 1..5, including tuples with repetitions
of 2, and combined with field width and/or precision, again as permitted
by the conversion. The resulting format string is then used to produce
output from respective sets of input data corresponding to the specific
conversion under test. POSIX extensions beyond ISO C are not used.
Output is produced in the form of records which include both the format
string (and width and/or precision where given in the form of separate
arguments) and the conversion result, and is verified with GNU AWK using
the format obtained from each such record against the reference value
also supplied, relying on the fact that GNU AWK has its own independent
implementation of format processing, striving to be ISO C compatible.
In the course of implementation I have determined that in the non-bignum
mode GNU AWK uses system sprintf(3) for the floating-point conversions,
defeating the objective of doing the verification against an independent
implementation. Additionally the bignum mode (using MPFR) is required
to correctly output wider integer and floating-point data. Therefore
for the conversions affected the relevant shell scripts sanity-check AWK
and terminate with unsupported status if the bignum mode is unavailable
for floating-point data or where data is output incorrectly.
The f and F floating-point conversions are build-time options for GNU
AWK, depending on the environment, so they are probed for before being
used. Similarly the a and A floating-point conversions, however they
are currently not used, see below. Also GNU AWK does not handle the b
or B integer conversions at all at the moment, as at 5.3.0. Support for
the a, A, b, and B conversions can however be easily added following the
approach taken for the f and F conversions.
Output produced by gawk for the a and A floating-point conversions does
not match one produced by us: insufficient precision is used where one
hasn't been explicitly given, e.g. for the negated maximum finite IEEE
754 64-bit value of -1.79769313486231570814527423731704357e+308 and "%a"
format we produce -0x1.fffffffffffffp+1023 vs gawk's -0x1.000000p+1024
and a different exponent is chosen otherwise, such as with "%.a" where
we output -0x2p+1023 vs gawk's -0x1p+1024 for the same value, or "%.20a"
where -0x1.fffffffffffff0000000p+1023 is our output, but gawk produces
-0xf.ffffffffffff80000000p+1020 instead. Consequently I chose not to
include a and A conversions in testing at this time.
And last but not least there are numerous corner cases that GNU AWK does
not handle correctly, which are worked around by explicit handling in
the AWK script. These are in particular:
- extraneous leading 0 produced for the alternative form with the o
conversion, e.g. { printf "%#.2o", 1 } produces "001" rather than
"01",
- unexpected 0 produced where no characters are expected for the input
of 0 and the alternative form with the precision of 0 and the integer
hexadecimal conversions, e.g. { printf "%#.x", 0 } produces "0" rather
than "",
- missing + character in the non-bignum mode only for the input of 0
with the + flag, precision of 0 and the signed integer conversions,
e.g. { printf "%+.i", 0 } produces "" rather than "+",
- missing space character in the non-bignum mode only for the input of 0
with the space flag, precision of 0 and the signed integer
conversions, e.g. { printf "% .i", 0 } produces "" rather than " ",
- for released gawk versions of up to 4.2.1 missing - character for the
input of -NaN with the floating-point conversions, e.g. { printf "%e",
"-nan" }' produces "nan" rather than "-nan",
- for released gawk versions from 5.0.0 onwards + character output for
the input of -NaN with the floating-point conversions, e.g. { printf
"%e", "-nan" }' produces "+nan" rather than "-nan",
- for released gawk versions from 5.0.0 onwards + character output for
the input of Inf or NaN in the absence of the + or space flags with
the floating-point conversions, e.g. { printf "%e", "inf" }' produces
"+inf" rather than "inf",
- for released gawk versions of up to 4.2.1 missing + character for the
input of Inf or NaN with the + flag and the floating-point
conversions, e.g. { printf "%+e", "inf" }' produces "inf" rather than
"+inf",
- for released gawk versions of up to 4.2.1 missing space character for
the input of Inf or NaN with the space flag and the floating-point
conversions, e.g. { printf "% e", "nan" }' produces "nan" rather than
" nan",
- for released gawk versions from 5.0.0 onwards + character output for
the input of Inf or NaN with the space flag and the floating-point
conversions, e.g. { printf "% e", "inf" }' produces "+inf" rather than
" inf",
- for released gawk versions from 5.0.0 onwards the field width is
ignored for the input of Inf or NaN and the floating-point
conversions, e.g. { printf "%20e", "-inf" }' produces "-inf" rather
than " -inf",
NB for released gawk versions of up to 4.2.1 floating-point conversion
issues apply to the bignum mode only, as in the non-bignum mode system
sprintf(3) is used. As from version 5.0.0 specialized handling has been
added for [-]Inf and [-]NaN inputs and the issues listed apply to both
modes. The '--posix' flag makes gawk versions from 5.0.0 onwards avoid
the issue with field width and the + character unconditionally output
for the input of Inf or NaN, however not the remaining issues and then
the 'gensub' function is not supported in the POSIX mode, so to go this
path I deemed not worth it.
Each test completes within single seconds except for the long double
one. There the F/f formats produce a large number of digits, which
appears to be computationally intensive and CPU-bound. Standalone
execution time for 'tst-printf-format-p-ldouble --direct f' is in the
range of 00m36s for POWER9@2.166GHz and 09m52s for FU740@1.2GHz and
output redirected locally to /dev/null, and 10m11s for FU740 and output
redirected over 100Mbps network via SSH to /dev/null, so the throughput
of the network adds very little (~3.2% in this case) to the processing
time. This is with IEEE 754 quad.
So I have scaled the timeout for 'tst-printf-format-skeleton-ldouble'
accordingly. Regardless, following recent practice the test has been
added to the standard rather than extended set. However, unlike most
of the remaining tests it has been split by the conversion specifier,
so as to allow better parallelization of this long-running test. As
a side effect this lets the test report the unsupported status for the
F/f conversions where applicable, so 'tst-printf-format-p-double' has
been split for consistency as well.
Only printf itself is handled at the moment, but the infrastructure
provides for all the printf family functions to be verified, changes
for which to be supplied separately. The complication around having
some tests iterating over all the relevant conversion specifiers and
other verifying conversion specifiers individually combined with
iterating over printf family functions has hit a peculiarity in GNU
make where the use of multiple targets with a pattern rule is handled
differently from such use with an ordinary rule. Consequently it
seems impossible to bulk-define a pattern rule using '$(foreach ...)',
where each target would simply trigger the recipe according to the
pattern and matching dependencies individually (such a rule does work,
but implies all targets to be updated with a single recipe execution).
Therefore as a compromise a single single-target pattern rule has been
defined that has listed all the conversion-specific scripts and all the
test executables as dependencies. Consequently tests will be rerun in
the absence of changes to their actual sources or scripts whenever an
unrelated file has changed that has been listed. Also all the formatted
printf output tests will always be built whenever any single one is to
be run. This only affects test development and not test runs in the
field, though it does change the order of execution of the individual
steps and also acts as a Makefile barrier in parallel runs. As the
execution time dominates the compilation time for these tests it is not
seen as a serious shortcoming.
As pointed out by Florian Weimer <fweimer@redhat.com> the malloc tracing
facility can take a substantial amount of time in calling dladdr(3) to
determine the caller's location. This is not needed by the verification
made with these tests, so I chose to interpose the symbol with a stub
implementation that always fails in the shared skeleton. We have total
control over the test environment, so I think it is a safe and minimal
impact approach. If there's ever anything else added to the tests that
would actually rely on dladdr(3) returning usable results, only then we
can think of a different approach.
Reviewed-by: DJ Delorie <dj@redhat.com>
On GCC 11 (x86-64), the previous code produced test failures like
this one:
Failure: Test: exp10m1_towardzero (-0x1.1p+4)
Result:
is: -1.00000000e+00 -0x1.000000p+0
should be: -9.99999940e-01 -0x1.fffffep-1
difference: 5.96046447e-08 0x1.000000p-24
ulp : 1.0000
max.ulp : 0.0000
Apply a similar fix to exp2m1f.
Co-authored-by: Paul Zimmermann <Paul.Zimmermann@inria.fr>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Change name of the access_rights argument to access_restrictions
of the following functions:
- pkey_alloc()
- pkey_set()
as this argument refers to access restrictions rather than access
rights and previous name might have been misleading.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Update the name of the argument in several pkey_*() functions that refers
to access restrictions rather than access rights: change access "rights"
to access "restrictions".
Specify that the result of the pkey_get() should be checked using bitwise
operations rather than plain equals comparison.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Before commit ee1ada1bdb
("elf: Rework exception handling in the dynamic loader
[BZ #25486]"), the previous order called the main calloc
to allocate a shadow GOT/PLT array for auditing support.
This happened before libc.so.6 ELF constructors were run, so
a user malloc could run without libc.so.6 having been
initialized fully. One observable effect was that
environ was NULL at this point.
It does not seem to be possible at present to trigger such
an allocation, but it seems more robust to delay switching
to main malloc after ld.so self-relocation is complete.
The elf/tst-rtld-no-malloc-audit test case fails with a
2.34-era glibc that does not have this fix.
Reviewed-by: DJ Delorie <dj@redhat.com>
For a long time, libc.so.6 has dependend on ld.so, which
means that there is a reference to ld.so in all processes,
and rtld_multiple_ref is always true. In fact, if
rtld_multiple_ref were false, some of the ld.so setup code
would not run.
Reviewed-by: DJ Delorie <dj@redhat.com>
Linux 3.15 and 6.2 added HWCAP2_* values for Arm. These bits have
already been added to dl-procinfo.{c,h} in commits 9aea0cb842 and
8ebe9c0b38. Also add them to <bits/hwcap.h> so that they can be used
in user code. For example, for checking bits in the value returned by
getauxval(AT_HWCAP2).
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Reviewed-by: Yury Khrustalev <yury.khrustalev@arm.com>
This patch starts preparation for C2Y support in glibc headers by
adding a feature test macro _ISOC2Y_SOURCE and corresponding
__GLIBC_USE (ISOC2Y). (I mostly copied the work of Joseph Myers
for C2X). As with other such macros, C2Y features are also
enabled by compiling for a standard newer than C23, or by using
_GNU_SOURCE.
This patch does not itself enable anything new in the headers for C2Y;
that is to be done in followup patches. (For example an implementation
of WG14 N3349.)
Once C2Y becomes an actual standard we'll presumably move to using the
actual year in the feature test macro and __GLIBC_USE, with some
period when both macro spellings are accepted, as was done with
_ISOC2X_SOURCE.
Tested for x86_64.
Signed-off-by: Lenard Mollenkopf <glibc@lenardmollenkopf.de>
By using a combination of mask-and-add instead of the shift-based
index calculation the routines can share the same table as other
variants with no performance degradation.
The tables change name because of other changes in downstream AOR.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
The CORE-MATH exp2m1f implementation showed slight worse latency
when using x86_64 baseline ABI. This patch adds a ifunc variant
with similar performance for x86_64-v3.
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Reviewed-by: DJ Delorie <dj@redhat.com>
The CORE-MATH exp10m1f implementation showed slight worse latency
when using x86_64 baseline ABI. This patch adds a ifunc variant
with similar performance for x86_64-v3.
Reviewed-by: Noah Goldstein <goldstein.w.n@gmail.com>
Reviewed-by: DJ Delorie <dj@redhat.com>
The CORE-MATH implementation is correctly rounded (for any rounding mode)
and shows better performance compared to the generic exp2m1f.
The code was adapted to glibc style and to use the definition of
math_config.h (to handle errno, overflow, and underflow). The
only change is to handle FLT_MAX_EXP for FE_DOWNWARD or FE_TOWARDZERO.
The benchmark inputs are based on exp2f ones.
Benchtest on x64_64 (Ryzen 9 5900X, gcc 14.2.1), aarch64 (Neoverse-N1,
gcc 13.3.1), and powerpc (POWER10, gcc 13.2.1):
Latency master patched improvement
x86_64 40.6042 48.7104 -19.96%
x86_64v2 40.7506 35.9032 11.90%
x86_64v3 35.2301 31.7956 9.75%
i686 102.094 94.6657 7.28%
aarch64 18.2704 15.1387 17.14%
power10 11.9444 8.2402 31.01%
reciprocal-throughput master patched improvement
x86_64 20.8683 16.1428 22.64%
x86_64v2 19.5076 10.4474 46.44%
x86_64v3 19.2106 10.4014 45.86%
i686 56.4054 59.3004 -5.13%
aarch64 12.0781 7.3953 38.77%
power10 6.5306 5.9388 9.06%
The generic implementation calls __ieee754_exp2f and x86_64 provides
an optimized ifunc version (built with -mfma -mavx2, not correctly
rounded). This explains the performance difference for x86_64.
Same for i686, where the ABI provides an optimized __ieee754_exp2f
version built with '-msse2 -mfpmath=sse'. When built wth same
flags, the new algorithm shows a better performance:
master patched improvement
latency 102.094 91.2823 10.59%
reciprocal-throughput 56.4054 52.7984 6.39%
Signed-off-by: Alexei Sibidanov <sibid@uvic.ca>
Signed-off-by: Paul Zimmermann <Paul.Zimmermann@inria.fr>
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Reviewed-by: DJ Delorie <dj@redhat.com>
The CORE-MATH implementation is correctly rounded (for any rounding mode)
and shows better performance compared to the generic exp10m1f.
The code was adapted to glibc style and to use the definition of
math_config.h (to handle errno, overflow, and underflow). I mostly
fixed some small issues in corner cases (sNaN handling, -INFINITY,
a specific overflow check).
Benchtest on x64_64 (Ryzen 9 5900X, gcc 14.2.1), aarch64 (Neoverse-N1,
gcc 13.3.1), and powerpc (POWER10, gcc 13.2.1):
Latency master patched improvement
x86_64 45.4690 49.5845 -9.05%
x86_64v2 46.1604 36.2665 21.43%
x86_64v3 37.8442 31.0359 17.99%
i686 121.367 93.0079 23.37%
aarch64 21.1126 15.0165 28.87%
power10 12.7426 8.4929 33.35%
reciprocal-throughput master patched improvement
x86_64 19.6005 17.4005 11.22%
x86_64v2 19.6008 11.1977 42.87%
x86_64v3 17.5427 10.2898 41.34%
i686 59.4215 60.9675 -2.60%
aarch64 13.9814 7.9173 43.37%
power10 6.7814 6.4258 5.24%
The generic implementation calls __ieee754_exp10f which has an
optimized version, although it is not correctly rounded, which is
the main culprit of the the latency difference for x86_64 and
throughp for i686.
Signed-off-by: Alexei Sibidanov <sibid@uvic.ca>
Signed-off-by: Paul Zimmermann <Paul.Zimmermann@inria.fr>
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Reviewed-by: DJ Delorie <dj@redhat.com>
Also remove the use of builtins in favor of standard names, compiler
already inline them (if supported) with current compiler options.
It also fixes and issue where __builtin_roundeven is not support on
gcc older than version 10.
Checked on x86_64-linux-gnu and i686-linux_gnu.
Signed-off-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Reviewed-by: DJ Delorie <dj@redhat.com>
This will be required by the rseq extensible ABI implementation on all
Linux architectures exposing the '__rseq_size' and '__rseq_offset'
symbols to set the initial value of the 'cpu_id' field which can be used
by applications to test if rseq is available and registered. As long as
the symbols are exposed it is valid for an application to perform this
test even if rseq is not yet implemented in libc for this architecture.
Both code paths are compile tested with build-many-glibcs.py but I don't
have access to any hardware to run the tests.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Arjun Shankar <arjun@redhat.com>
Save lr in a non-volatile register before scv in clone/clone3.
For clone, the non-volatile register was unused and already
saved/restored. Remove the dead code from clone.
Signed-off-by: Sachin Monga <smonga@linux.ibm.com>
Reviewed-by: Peter Bergner <bergner@linux.ibm.com>
There are no tests specifically focused on the functions time,
gettimeofday and clock_gettime, although there are some incidental
uses in tests of other functions. Add tests specifically for these
three functions.
Tested for x86_64 and x86.
There are various existing tests that call pthread_attr_init and then
verify properties of the resulting initial values retrieved with
pthread_attr_get* functions. However, those are missing coverage of
the initial values retrieved with pthread_attr_getschedparam and
pthread_attr_getstacksize. Add testing for initial values from those
functions as well.
(tst-attr2 covers pthread_attr_getdetachstate,
pthread_attr_getguardsize, pthread_attr_getinheritsched,
pthread_attr_getschedpolicy, pthread_attr_getscope. tst-attr3 covers
some of those together with pthread_attr_getaffinity_np.
tst-pthread-attr-sigmask covers pthread_attr_getsigmask_np.
pthread_attr_getstack has unspecified results if called before the
relevant attributes have been set, while pthread_attr_getstackaddr is
deprecated.)
Tested for x86_64.
The gilbc manual has some documentation in llio.texi of requirements
for moving between I/O on FILE * streams and file descriptors on the
same open file description.
The documentation of what must be done on a FILE * stream to move from
it to either a file descriptor or another FILE * for the same open
file description seems to match POSIX. However, there is an
additional requirement in POSIX on the *second* of the two handles
being moved between, which is not mentioned in the glibc manual: "If
any previous active handle has been used by a function that explicitly
changed the file offset, except as required above for the first
handle, the application shall perform an lseek() or fseek() (as
appropriate to the type of handle) to an appropriate location.".
Document this requirement on seeking in the glibc manual, limited to
the case that seems relevant to glibc (the new channel is a previously
active stream, on which the seeking previously occurred). Note that
I'm not sure what the "except as required above for the first handle"
is meant to be about, so I haven't documented anything for it. As far
as I can tell, nothing specified for moving from the first handle
actually list calling a seek function as one of the steps to be done.
(Current POSIX doesn't seem to have any relevant rationale for this
section. The rationale in the 1996 edition says "In requiring the
seek to an appropriate location for the new handle, the application is
required to know what it is doing if it is passing streams with seeks
involved. If the required seek is not done, the results are undefined
(and in fact the program probably will not work on many common
implementations)." - which also doesn't help in understanding the
purpose of "except as required above for the first handle".)
Tested with "make info" and "make pdf".
In both routines, reduce register pressure such that GCC 14 emits no
spills for erf and fewer spills for erfc. Also use more efficient
comparison for the special-case in erf.
Benchtests show erf improves by 6.4%, erfc by 1.0%.
In commit c628c22963 (elf: Remove
ldconfig kernel version check), the layout of auxcache entries
changed because the osversion field was removed from
struct aux_cache_file_entry. However, AUX_CACHEMAGIC was not
changed, so existing files are still used, potentially leading
to unintended ldconfig behavior. This commit changes AUX_CACHEMAGIC,
so that the file is regenerated.
Reported-by: DJ Delorie <dj@redhat.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This fixes a buffer overflow in wide character string output, reproducing
when output fails, such as if the output fd is closed or is redirected
to a full device.
Wide character output data attempts to maintain the invariant that
`_IO_buf_base <= _IO_write_base <= _IO_write_end <= _IO_buf_end` (that is,
that the write region is a sub-region of `_IO_buf`). Prior to this commit,
this invariant is violated by the `_IO_wfile_overflow` function as so:
1. `_IO_wsetg` is called, assigning `_IO_write_base` to `_IO_buf_base`
2. `_IO_doallocbuf` is called, which jumps to `_IO_wfile_doallocate` via
the _IO_wfile_jumps vtable. This function then assigns the wide data
`_IO_buf_base` and `_IO_buf_end` to a malloc'd buffer.
Thus the invariant is violated. The fix is simply to reverse the order:
malloc the `_IO_buf` first and then assign `_IO_write_base` to it.
We also take this opportunity to defensively guard the initialization of
the number of unwritten characters via pointer arithmetic. We now check
that the buffer end is not before the buffer beginning; this matches a
similar defensive check in the narrow analogue `fileops.c`.
Add a test which fails without the fix.
Signed-off-by: Peter Ammon <corydoras@ridiculousfish.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The scanf family of functions like sscanf and fscanf currently
ignore nan() and nan(n-char-sequence). This happens because
__vfscanf_internal only checks for 'nan'.
This commit adds support for all valid nan types i.e. nan, nan()
and nan(n-char-sequence), where n-char-sequence can be
[a-zA-Z0-9_]+, thus fixing the bug 30647. Any other representation
of NaN should result in conversion error.
New tests are also added to verify the correct parsing of NaN types for
float, double and long double formats.
Signed-off-by: Avinal Kumar <avinal.xlvii@gmail.com>
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
The refactoring did not take the change of variable into account.
Fixes commit 43db5e2c06
("elf: Signal RT_CONSISTENT after relocation processing in dlopen
(bug 31986)").
Previously, a la_activity audit event was generated before
relocation processing completed. This does did not match what
happened during initial startup in elf/rtld.c (towards the end
of dl_main). It also caused various problems if an auditor
tried to open the same shared object again using dlmopen:
If it was the directly loaded object, it had a search scope
associated with it, so the early exit in dl_open_worker_begin
was taken even though the object was unrelocated. This caused
the r_state == RT_CONSISTENT assert to fail. Avoidance of the
assert also depends on reversing the order of r_state update
and auditor event (already implemented in a previous commit).
At the later point, args->map can be NULL due to failure,
so use the assigned namespace ID instead if that is available.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Auditors can call into the dynamic loader again if
LA_ACT_CONSISTENT, and those recursive calls could observe
r_state != RT_CONSISTENT.
We should consider failing dlopen/dlmopen/dlclose if
r_state != RT_CONSISTENT. The dynamic linker is probably not
in a state in which it can handle reentrant calls. This
needs further investigation.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
This is conceptually similar to the reported bug, but does not
depend on auditing. The fix is simple: just complete execution
of the constructors. This exposed the fact that the link map
for statically linked executables does not have l_init_called
set, even though constructors have run.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
