Commit Graph

3759 Commits

Author SHA1 Message Date
jhendersonHDF
b929dbd312
Fix use of possibly uninitialized memory in h5repack (#4354) 2024-04-08 15:47:32 -07:00
Dana Robinson
4dbaec7991
Remove HD from last non-win32 C89 functions (#4348)
Removes the HD prefix from the last C89 functions with no special
Windows equilvalent:

* rename
* rewind
* setbuf
* signal
* tmpfile
2024-04-08 07:58:13 -07:00
Dana Robinson
2f4372104a
Remove #undef ERR from H5private.h (#4346)
This is an old work-around that is no longer necessary.

The only place where we defined ERR is in h5import, and that code
has been updated (in this PR) to rename ERR to INVALID_TOKEN.
2024-04-08 07:57:42 -07:00
Dana Robinson
a99d620172
Clean up random number generator code (#4338)
* Clean up random number generator code

Depending on the platform, we use a mix of random, rand, and rand_r
to generate pseudo-random numbers, along with a messy set of ifdefs
in H5private.h. We are not a cryptographic library, only use random
numbers in our test code, and have no need for anything more than the
C standard's (s)rand(). There's no point dithering about using rand()
vs random() when we're also doing bad things like using mod to
restrict the range, which introduces bias.

Also removes CMake/configure checks for rand_r and random

* Remove random/rand_r checks from build system

* Fix missed HDrandom after GitHub merge
2024-04-07 15:15:25 -07:00
Dana Robinson
0de523bc42
Remove HD macros for C time functions (#4337)
* HDasctime
* HDclock
* HDctime
* HDdifftime
* HDgmtime
* HDlocaltime
* HDmktime
* HDtime
* HDtzset

HDgettimeofday will be done later
2024-04-07 14:09:23 -07:00
jhendersonHDF
ce53bc020d
Fixes for file format security issues (#4283) 2024-03-31 22:44:18 -05:00
Dana Robinson
80fc8558b0
Fix off_t straggers (#4291)
Convert off_t to HDoff_t (mainly for Windows):

* h5jam
* h5unjam
* chunk_info test
2024-03-31 12:12:26 -07:00
Dana Robinson
136739b37a
File format security issues (#4234) 2024-03-27 13:12:32 -07:00
jhendersonHDF
638c4d901f
Fix an issue with type size assumptions in h5dumpgentest (#4222) 2024-03-24 15:13:46 -05:00
jhendersonHDF
86ca181bd8
Fix 'make check-vfd' target for Autotools (#4211)
Changes Autotools testing to use HDF5_TEST_DRIVER environment
variable to avoid running tests that don't work well with several
VFDs

Restores old h5_get_vfd_fapl() testing function to setup a FAPL
with a particular VFD

Adds a macro for the default VFD name
2024-03-22 21:31:16 -05:00
Dana Robinson
15608c00b5
Call memset before stat calls (#4202)
The buffers passed to stat-like calls are only partially filled in by
the call, leaving ununitialized memory areas when the stat buffers are
created on the stack.

This change memsets the buffers to 0 before the stat calls, quieting
the -fsanitze=memory complaints.
2024-03-21 05:31:30 -07:00
Allen Byrne
977f4aab14
h5diff compares attribute data like dataset data (#4191)
Updates tools docs to indicate that dataset and attribute data are compared in the same way
2024-03-20 10:54:44 -07:00
jhendersonHDF
330b80a266
Add support for _Float16 16-bit floating point type (#4065)
Fixed some conversion issues with Clang due to problematic undefined
behavior when casting a negative floating-point value to an integer

Fixed a bug in the library's software integer to floating-point
conversion function where a user's conversion exception function
returning H5T_CONV_UNHANDLED in the case of overflows would result in
incorrect data after conversion

Added configure checks for functions and macros related to _Float16
usage since some compilers expose the datatype but not the functions or
macros

Fixed a dt_arith test failure when H5_WANT_DCONV_EXCEPTION isn't defined

Fixed a few warnings from not explicitly casting some _Float16 variables
upwards
2024-03-18 21:36:46 -07:00
Allen Byrne
50a6522322
Change compression tests reference files to use masking for compression ratios (#4083)
Rework TEST_FILTER tests to handle slightly different compression ratios
2024-03-10 16:43:37 -07:00
Dana Robinson
f0ba8baa0d
Minimize use of abort() (#4110)
The abort() call is used at several places where it probably shouldn't.
2024-03-10 12:42:16 -07:00
jhendersonHDF
ef401a5f5e
Refactor datatype conversion code to use pointers rather than IDs (#4104)
The datatype conversion code previously used IDs for the source and
destination datatypes rather than pointers to the internal structures
for those datatypes. This was mostly due to the need for an ID for these
datatypes that can be passed to an application-registered datatype
conversion function or datatype conversion exception function. However,
using IDs internally caused a lot of unnecessary ID lookups and hurt
performance of datatype conversions in general. This was especially
problematic for compound datatype conversions, where the ID lookups were
occuring on every member of every compound element of a dataset. The
code has now been refactored to use pointers internally and only create
IDs for datatypes when necessary.

Fixed a test issue in dt_arith where a library datatype conversion
function was being cast to an application conversion function. Since the
two have different prototypes, this started failing after the parameters
for a library conversion function changed from hid_t to H5T_t * and an
extra parameter was added. This appears to have worked coincidentally in
the past since the only different between a library conversion function
and application conversion function was an extra DXPL parameter at the
end of an application conversion function

Fixed an issue where memory wasn't being freed in the h5fc_chk_idx test
program. Even though the program exits quickly after allocating the
memory, it still causes failures when testing with -fsanitize=address
2024-03-09 23:47:31 -08:00
Dana Robinson
0ea1d7280b
Clean up off_t usage (#4095)
* Add comments to C++ and Fortran API calls that use off_t
* Remove noise casts for small integers
2024-03-09 09:06:10 -08:00
H. Joe Lee
1a05563183
Add subfiling for h5dump filedriver option help message (#3878) 2024-03-08 13:39:33 -08:00
Allen Byrne
0be892585a
Rename incorrectly named option (#4067)
* Rename incorrectly named option

* Restore the correct uses of USING_MEMCHECKER

* Update release note
2024-03-05 09:07:16 -08:00
Larry Knox
968cf9e707
Improve error messages when tools attempt to use non-enabled S3 and HDFS VFDs. (#4047) 2024-03-04 06:54:24 -08:00
Allen Byrne
7ce8c6f81a
Add options to allow tools type selection and naming (#4046) 2024-03-04 06:53:52 -08:00
Allen Byrne
02a57328f7
Grep for error messages instead of comparing files (#4024) 2024-02-16 09:14:22 -08:00
jhendersonHDF
be956ea302
Fix some output issues with ph5diff (#4008) 2024-02-13 09:06:33 -06:00
Sean McBride
f309fcebbd
Issue #1824: Replaced most remaining sprintf with safer snprint (#4003) 2024-02-13 08:22:17 -06:00
H. Joe Lee
0843a8c1be
Add h5dump tests that require subfiling VFD (#3879) 2024-02-01 09:02:47 -08:00
H. Joe Lee
6a5dd86315
Remove check for MPI initialization in tools library when using parallel VFDs
The tools library no longer checks for whether MPI is initialized when attempting to use a parallel VFD, such as the MPI I/O VFD or Subfiling VFD. This requires that parallel VFDs check for this situation and initialize MPI on their own if need be, but also allows parallel VFDs to be used with non-parallel HDF5 tools, such as h5dump.
2024-01-09 10:38:35 -06:00
Quincey Koziol
3a21ee0877
Add 'warning density' computation to the warnhist script (#3910)
* Add 'warning density' computation to the warnhist script, along with several
cleanups to it.   Add "--enable-show-all-warnings" configure (and CMake)
option to disable compiler diagnostic suppression (and therefore show all the
otherwise suppressed compiler diagnostics), disabled by default.  Clean up
a buncn of misc. warnings.

Signed-off-by: Quincey Koziol <qkoziol@amazon.com>
2023-12-29 08:42:35 -06:00
Allen Byrne
6f2d22d86e
Add variable option syncing for examples (#3885) 2023-12-14 12:50:30 -06:00
Allen Byrne
4f01fb39d8
Correct CMake command and example packaging (#3888) 2023-12-08 07:45:54 -06:00
H. Joe Lee
0fa3029c88
Fix h5dump segmentation fault when --vfd-value option is used (#3873) 2023-11-28 15:31:35 -06:00
H. Joe Lee
ccc1ebd6fe
Complete the if command line option sentence. (#3868) 2023-11-24 06:19:27 -06:00
Allen Byrne
3ea21ccb3e
Add HDF5_DISABLE_TESTS_REGEX option to skip tests (#3859) 2023-11-22 07:51:30 -06:00
H. Joe Lee
37c75e057b
Fix typo: look -> loop (#3866) 2023-11-21 08:53:17 -08:00
H. Joe Lee
2512bdd7a9
Add extra space in comments for consistency (#3852)
* Add extra space in comments for consistency

* uncomment tfloatsattrs test
2023-11-14 07:58:20 -08:00
jhendersonHDF
562c53c44a
Fix misc. warnings from GCC when compiling with -fsanitize=undefined (#3787) 2023-11-01 08:16:33 -05:00
Robert Adam
87b3e7c821
Don't build util tests when HDF5_EXTERNALLY_CONFIGURED=ON (#3781)
Fixes #3780
2023-10-30 12:47:06 -05:00
jhendersonHDF
556add35b7
Don't install h5tools_test_utils test program on system (#3793) 2023-10-30 11:12:10 -05:00
bmribler
744a20fc21
Add helpful text to h5clear (#3754)
Added text to the usage of h5clear to explain that this tool is not
for fixing corrupted files but simply for helping in the inspection
of the damage..

Fixed expected output for testing the modified usage.
2023-10-23 12:46:56 -07:00
Dana Robinson
7843db55ff
Clean up onion VFD files in tools make clean (#3739)
Cleans up h5dump and h5diff *.onion files in the Autotools when
runing `make clean`.
2023-10-21 11:44:03 -07:00
Dana Robinson
98b5779ea5
Add tools/libtest to Autotools builds (#3735)
This was only added to CMake many years ago and tests the tools
library.
2023-10-20 15:50:57 -05:00
Allen Byrne
b5c63fb3fe
Test scripts now execute in-source with creation of tmp dir (#3723)
Fixes a few issues created in #3580:

* Fixes a problem where committed tools test files were deleted when cleaning after an in-source build
* Fixes issues with test file paths in Autotools tools test scripts
2023-10-20 10:57:09 -07:00
jhendersonHDF
c4a146efc4
Fix issue with unmatched messages in ph5diff (#3719) 2023-10-19 08:14:20 -05:00
jhendersonHDF
9557256819
Sync changes that are only in 1.14 branch (#3704) 2023-10-17 21:54:40 -07:00
jhendersonHDF
34fcb9c5a4
Fix several spelling/grammar issues (#3621) 2023-10-03 09:01:28 -07:00
Dana Robinson
a03506b089
Remove unnecessary assignment in test generator (#3603)
Fixes what looks like a copy/paste/modify error in the format convert
test file generator, where an array element is assigned one value
and them immediately overwritten by another value.

Fixes Coverity issue 1542285
2023-09-27 09:57:02 -05:00
Dana Robinson
d8fbc56f78
Fix potential uninitialized variable (#3602)
Moves a union initialization up a bit so it's performed before code
that can jump to the cleanup target, where file descriptors could
be checked without being initialized.

This could only happen in test code and only in an out-of-memory
situation.

Fixes Coverity 1542254
2023-09-27 09:56:25 -05:00
Allen Byrne
1529ec0ab3
Develop tools move (#3580)
Reorganizes the tools files to support the VOL tests
2023-09-26 13:46:55 -07:00
Scot Breitenfeld
a87ca572fe
Replaces HDgetenv with getenv (#3599) 2023-09-26 13:11:22 -07:00
Dana Robinson
58f3d6664e
Use HDoff_t with lseek consistently (#3600)
lseek on Windows uses __int64 for both the offset and return type
instead of off_t like most POSIX systems. This changes ensures we
use HDoff_t (which is typdef'd correctly on Windows) w/ lseek.
2023-09-26 11:38:07 -07:00
Dana Robinson
44a00ef876
Strip HD prefix from string/char C API calls (#3540)
* Strip HD prefix from string/char C API calls

* HD(f)(put|get)(s|c)
* HDstr*
* HDv*printf
* HD(s)(print|scan)f
* HDperror

But NOT:

* HDstrcase*
* HDvasprintf
* HDstrtok_r
* HDstrndup

As those are not C99 and have portability work-around
implementations. They will be handled later.

* Fix th5_system.c screwup
2023-09-15 15:13:18 -07:00