Commit Graph

1044 Commits

Author SHA1 Message Date
Dana Robinson
4dbaec7991
Remove HD from last non-win32 C89 functions (#4348)
Removes the HD prefix from the last C89 functions with no special
Windows equilvalent:

* rename
* rewind
* setbuf
* signal
* tmpfile
2024-04-08 07:58:13 -07:00
Dana Robinson
a99d620172
Clean up random number generator code (#4338)
* Clean up random number generator code

Depending on the platform, we use a mix of random, rand, and rand_r
to generate pseudo-random numbers, along with a messy set of ifdefs
in H5private.h. We are not a cryptographic library, only use random
numbers in our test code, and have no need for anything more than the
C standard's (s)rand(). There's no point dithering about using rand()
vs random() when we're also doing bad things like using mod to
restrict the range, which introduces bias.

Also removes CMake/configure checks for rand_r and random

* Remove random/rand_r checks from build system

* Fix missed HDrandom after GitHub merge
2024-04-07 15:15:25 -07:00
Dana Robinson
0de523bc42
Remove HD macros for C time functions (#4337)
* HDasctime
* HDclock
* HDctime
* HDdifftime
* HDgmtime
* HDlocaltime
* HDmktime
* HDtime
* HDtzset

HDgettimeofday will be done later
2024-04-07 14:09:23 -07:00
jhendersonHDF
86ca181bd8
Fix 'make check-vfd' target for Autotools (#4211)
Changes Autotools testing to use HDF5_TEST_DRIVER environment
variable to avoid running tests that don't work well with several
VFDs

Restores old h5_get_vfd_fapl() testing function to setup a FAPL
with a particular VFD

Adds a macro for the default VFD name
2024-03-22 21:31:16 -05:00
jhendersonHDF
c4cc33abe1
Fix issue with Subfiling VFD and multiple opens of same file (#4194)
* Fix issue with Subfiling VFD and multiple opens of same file

* Update H5_subfile_fid_to_context to return error value instead of ID

* Add helper routine to initialize open file mapping
2024-03-21 17:39:13 -07:00
Dana Robinson
15608c00b5
Call memset before stat calls (#4202)
The buffers passed to stat-like calls are only partially filled in by
the call, leaving ununitialized memory areas when the stat buffers are
created on the stack.

This change memsets the buffers to 0 before the stat calls, quieting
the -fsanitze=memory complaints.
2024-03-21 05:31:30 -07:00
Glenn Song
a286066123
Handle certain empty subfiling environment variables (#4038) 2024-03-20 10:52:27 -07:00
Dana Robinson
0ea1d7280b
Clean up off_t usage (#4095)
* Add comments to C++ and Fortran API calls that use off_t
* Remove noise casts for small integers
2024-03-09 09:06:10 -08:00
Allen Byrne
0be892585a
Rename incorrectly named option (#4067)
* Rename incorrectly named option

* Restore the correct uses of USING_MEMCHECKER

* Update release note
2024-03-05 09:07:16 -08:00
Sean McBride
f309fcebbd
Issue #1824: Replaced most remaining sprintf with safer snprint (#4003) 2024-02-13 08:22:17 -06:00
Dana Robinson
ab11e076bf
Replace off_t with HDoff_t internally (#3944)
off_t is a 32-bit signed value on Windows, so we should use HDoff_t
(which is __int64 on Windows) internally instead.

Also defines HDftell on Windows to be _ftelli64().
2024-01-18 09:03:59 -08:00
Allen Byrne
5f5f5f030e
Remove incorrect include of sz header (#3939) 2024-01-10 16:32:31 -06:00
jhendersonHDF
6ffc55c6d9
Implement optimized support for vector I/O in Subfiling VFD (#3896)
Vector I/O requests are now processed within a single
set of I/O call batches, rather than each I/O vector
entry (tuple constructed from the types, addrs, sizes
and bufs arrays) being processed individually. This allows I/O to be
more efficiently parallelized among the I/O concentrator processes
during large I/O requests.

* Fixed some calculations and add test cases for issues spotted from review

* Removed a variable that was compensating for previous miscalculations
2023-12-27 16:28:41 -06:00
Allen Byrne
3ea21ccb3e
Add HDF5_DISABLE_TESTS_REGEX option to skip tests (#3859) 2023-11-22 07:51:30 -06:00
vchoi-hdfgroup
ed31aaca79
Implement selection vector I/O with collective chunk filling (#3826)
* Changes for ECP-344: Implement selection vector I/O with collective chunk filling.
Also fix a bug in H5FD__mpio_write_vector() to account for fixed size optimization
when computing max address.

* Fixes based on PR review comments:
For H5Dchunk.c: fix H5MM_xfree()
For H5FDmpio.c:
1) Revert the fix to H5FD__mpio_write_vector()
2) Apply the patch from Neil on the proper length of s_sizes reported by H5FD__mpio_vector_build_types()

* Put back the logic of dividing up the work among all the mpi ranks similar to the
original H5D__chunk_collective_fill() routine.

* Add a test to verify the fix for the illegal reference problem in H5FD__mpio_write_vector().
2023-11-16 10:12:45 -06:00
jhendersonHDF
c779464bfe
Add GitHub actions for testing VOL connectors (#3849)
* Fix issue with HDF5_VOL_ALLOW_EXTERNAL CMake variable

* Add initial API test workflow

* Initialize parallel testing with MPI_THREAD_MULTIPLE when testing API

* Add CMake variable to allow specifying a VOL connector's package name

* Remove call to MPI_Init in serial API tests

While previously necessary, it now interferes with VOL connectors that
may need to be initialized with MPI_THREAD_MULTIPLE
2023-11-15 08:20:50 -06:00
jhendersonHDF
28d2b6771f
HDF5 API test updates (#3835)
* HDF5 API test updates

Removed test duplication from bringing API tests
back into the library from external VOL tests
repo

Synced changes between API tests and library's
tests

Updated API tests CMake code to directly use and
install testhdf5, testphdf5, etc. instead of
creating duplicate binaries

Added new h5_using_native_vol() test function to
determine whether the VOL connector being used
is (or the VOL connector stack being used resolves
to) the native VOL connector

* Remove duplicate variable
2023-11-13 13:49:38 -06:00
Scot Breitenfeld
ec695cf34f
Renamed h5fuse.sh to h5fuse (#3834)
* provide an alternative to mapfile for older bash
2023-11-07 09:50:03 -06:00
Glenn Song
3c07147971
Create test for H5Pget_dxpl_mpio (#3825)
* Create test and add to testphdf5
2023-11-06 09:27:51 -06:00
jhendersonHDF
ea3b6fd12b
Fix usage of h5_clean_files in t_pflush2.c (#3807) 2023-10-31 06:05:25 -07:00
jhendersonHDF
8df0008f07
Fix memory corruption in 'MPI I/O FAPL preserve' test (#3806) 2023-10-31 06:05:12 -07:00
jhendersonHDF
1bcef50b86
Print some messages in parallel tests on MPI rank 0 only (#3785)
Avoids overly verbose output from all processes emitting progress, etc. info.
2023-10-27 21:18:24 -07:00
Scot Breitenfeld
1d8513f034
Add compression tests for subfiling (#3769) 2023-10-25 13:49:30 -05:00
vchoi-hdfgroup
6578c452b5
Preserve MPI-I/O file hints when fapl is closed (#3755)
* Fix for issue #3025: Save the MPI info in the file struct so H5Fget_access_plist() can retrieve it from there.
2023-10-24 21:48:28 -05:00
Glenn Song
ea1714b303
Fix H5Pset_evict_on_close failing regardless of actual parallel use (#3761)
Allow H5Pset_evict_on_close to be called regardless of whether a parallel build of HDF5 is being used

Fail during file opens if H5Pset_evict_on_close has been set to true on the given File Access Property List and the size of the MPI communicator being used is greater than 1
2023-10-24 12:51:55 -05:00
Neil Fortner
1900cc63eb
Fix failure in t_select_io_dset when run with more than 10 ranks (#3758) 2023-10-23 19:08:48 -07:00
jhendersonHDF
af49eb5b86
Fix hangs during collective I/O with independent metadata writes (#3693) 2023-10-23 19:06:44 -07:00
Dana Robinson
e4fb67c84a
Clean Autotools files in parallel tests (#3744)
Adds missing files to `make clean` for parallel, including Fortran.
2023-10-22 18:45:22 -07:00
Neil Fortner
630d6e27c9
Add new API function H5Pget_actual_select_io_mode() (#2974)
This function allows the user to determine if the library performed selection I/O, vector I/O, or scalar (legacy) I/O during the last HDF5 operation performed with the provided DXPL. Expanded existing tests to check this functionality.
2023-10-20 10:32:17 -07:00
jhendersonHDF
af56339d3b
Fixes and adjustments to t_filters_parallel (#3714)
Broadcast number of datasets to create in multi-dataset I/O cases
so that interference with random number generation doesn't cause
mismatches between ranks

Set fill time for datasets to "never" by default and adjust on a
per-test basis to avoid writing fill values to chunks when it's
unnecessary

Reduce number of loops run in some tests when performing multi-dataset
I/O

Fix an issue in the "fill time never" test where data verification
could fill if file space reuse causes application buffers to be
filled with chosen fill value when reading from datasets with
uninitialized storage

Skip multi-chunk I/O test configurations for multi-dataset I/O
configurations when the TestExpress level is > 1 since those
tests can be more stressful on the file system

Disable use of persistent file free space management for now
since it occasionally runs into an infinite loop in the library's
free space management code
2023-10-19 15:23:59 -07:00
Dana Robinson
e2d40ef16e
Suppress format string warnings in subfiling test (#3699) 2023-10-17 11:05:20 -07:00
jhendersonHDF
45c4729799
Add test info output to t_filters_parallel (#3696) 2023-10-17 10:40:42 -07:00
Dana Robinson
ccb1a917b6
Fix printf warnings in t_mpi (#3679)
* Fix printf warnings in t_mpi

The type of MPI_Offset varies with implementation. In MPICH, it's long,
which raises warnings when we attempt to use long long format
specifiers. Casting to long long fixes the warnings.
2023-10-16 12:19:15 -05:00
Dana Robinson
ef1edba2ce
Fix uninitialized subfiling test variable (#3675)
Picked up by gcc 10 on skybridge. Probably spurious, but no harm in
initializing it to a "bad" value.
2023-10-14 22:34:29 -05:00
jhendersonHDF
3ec119b558
Add expedited testing support to t_filters_parallel (#3665) 2023-10-12 16:12:57 -05:00
H. Joe Lee
72f94bc1f5
Simplify. (#3659)
* Address @jhendersonHDF review
2023-10-12 11:37:56 -05:00
jhendersonHDF
bfbfaf72e1
Update parallel compression feature to support multi-dataset I/O (#3591) 2023-10-10 10:11:22 -05:00
jhendersonHDF
34fcb9c5a4
Fix several spelling/grammar issues (#3621) 2023-10-03 09:01:28 -07:00
H. Joe Lee
e7360391fd
Fix typos (#3609) 2023-09-27 10:39:17 -07:00
H. Joe Lee
4233ae69c0
Remove sh to run bash script. (#3590) 2023-09-26 14:52:31 -07:00
Scot Breitenfeld
a87ca572fe
Replaces HDgetenv with getenv (#3599) 2023-09-26 13:11:22 -07:00
H. Joe Lee
f994aa5956
Fix typos and grammar in t_pread. (#3565) 2023-09-22 08:44:53 -05:00
H. Joe Lee
16ac8f74d4
Check return values from HDF5 API calls. (#3556) 2023-09-22 08:40:56 -05:00
Dana Robinson
44a00ef876
Strip HD prefix from string/char C API calls (#3540)
* Strip HD prefix from string/char C API calls

* HD(f)(put|get)(s|c)
* HDstr*
* HDv*printf
* HD(s)(print|scan)f
* HDperror

But NOT:

* HDstrcase*
* HDvasprintf
* HDstrtok_r
* HDstrndup

As those are not C99 and have portability work-around
implementations. They will be handled later.

* Fix th5_system.c screwup
2023-09-15 15:13:18 -07:00
Dana Robinson
3479b51dc1
Remove HD prefix from HDlog10 calls (#3539)
Was missed in a previous commit and causes building subfiling to
fail.
2023-09-15 09:36:14 -05:00
Scot Breitenfeld
157dda4887
quiet warning on sunspot (gcc 11.2.0) (#3534) 2023-09-13 07:34:07 -05:00
Scot Breitenfeld
117d579c32
Consistent initialization of hid_t in the tests (#3521) 2023-09-08 18:06:23 -05:00
Dana Robinson
d24f5d5223
Convert hbool_t --> bool in testpar (#3495) 2023-09-05 11:49:37 -07:00
Scot Breitenfeld
896270cc6b
Removed all the ranks printing out testing information (#3457) 2023-08-31 21:34:57 -07:00
H. Joe Lee
df161ac293
chore: match function call and VRFY() output (#3428) 2023-08-25 08:13:32 -05:00