* Fix an oversize allocation in t_bigio on 32-bit systems
A test in t_bigio.c attempts to allocate more than 4 GiB of memory,
which will overflow (and wrap) the size_t type on 32-bit systems,
creating a very small allocation instead of a very large allocation.
The test then segfaults when it accesses memory outside of the small
buffer.
The test has been fixed by limiting the buffer to 2 GiB on 32-bit
systems.
* Fix error in H5Ddebug(). Add h5ls test cases.
* Committing clang-format changes
* Fix cmake (hopefully)
* Fix cmake again (hopefully)
* Fix issue with modification times showing up differently on different
systems. Add test case for v1 b-trees.
* Add cmake for new tests
* Add RELEASE.txt entry
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Added infrastructure support for multithreaded concurrency by adding an optional way to switch to using a non-recursive R/W lock for the global API lock. This is enabled with a new 'concurrency' configuration flag for the autotools & CMake builds, which is disabled by default.
When the 'concurrency' build option is chosen, the global API lock will use the R/W lock and all API calls currently will acquire a write lock, ensuring exclusive access by one thread. Over time, the API routines that are converted to support multithreaded concurrency will switch to acquiring a read lock instead.
Reentering the library from application callbacks is managed by the 'disable locking for this thread' (DLFTT) threadsafety protocol. This is internally handled within the H5_API_LOCK / H5_API_UNLOCK macros in H5private.h (as before), which invoke the 'dlftt' routines in H5TSint.c.
To support this change, the threadsafety configuration macros for the library have been updated:
- --enable-threadsafe now defines the H5_HAVE_THREADSAFE macro
- --enable-concurrency defines the H5_HAVE_CONCURRENCY macro
The new H5_HAVE_THREADSAFE_API macro is set if either H5_HAVE_THREADSAFE or H5_HAVE_CONCURRENCY is enabled.
New Github actions are added to include the concurrency configuration in the CI for the develop branch.
To support the new non-recursive R/W locking for API routines, some other changes are necessary:
Added macro wrappers around all callback invocations that could call an
application function, and therefore re-enter the library:
H5_BEFORE_USER_CB* / H5_AFTER_USER_CB*
Added H5_user_cb_prepare / H5_user_cb_restore routines that save the
state of the library when callback leaves the library. Includes error
stack and threadsafe reentry state currently.
There's also some small cleanups to various places in the library:
Moved the H5E_mpi_error_str / H5E_mpi_error_str_len globals to be local for
pushing MPI errors, so that multiple threads can't interfere with each
other.
Added H5TS_rwlock_trywrlock() routine to R/W lock interface.
Emulate R/W locks on MacOS because its implementation of
pthread_rwlock_wrlock() does not conform to the POSIX standard.
Don't acquire the global API lock in H5close, since it's acquired in H5_term_library, which is necessary because H5_term_library is invoked via other code paths that don't hold the global API lock.
Don't call H5Eget_auto2 API routine within H5_term_library.
Switched 'return NULL' in H5allocate_memory to HGOTO_DONE(NULL).
Switched H5Pget_file_space_strategy / H5Pset_file_space_strategy to use
internal routines instead of API routines.
Switched H5Oopen_by_addr & H5Ovisit1 to use internal routines instead of
API routines.
Fixed a few places in src/H5Odeprec.c where a major
error ID was passed as a minor ID.
The hash_size parameter of H5Iregister_type() hasn't been used since 1.8.
It's been removed and the API call has been versioned.
This PR also updates the make_vers script to handle v2.0.0.
Fixes#4344
H5O__visit() uses the object information to be returned to the
application, so when the application did not request for certain
information, they were not available to H5O__visit. This lack of
information caused incorrect behavior down the road.
We now call H5O_get_info again providing H5O_INFO_BASIC for "fields",
so we can obtain correct object information for H5O__visit to use.
Fixes#4941
* Implement support for complex number datatypes
Adds the new datatype class H5T_COMPLEX
Adds the new API function H5Tcomplex_create which creates a complex
number datatype from an ID of a base floating-point datatype
Adds the new feature check macros H5_HAVE_COMPLEX_NUMBERS and
H5_HAVE_C99_COMPLEX_NUMBERS
Adds the new datatype size macros H5_SIZEOF_FLOAT_COMPLEX,
H5_SIZEOF_DOUBLE_COMPLEX and H5_SIZEOF_LONG_DOUBLE_COMPLEX
Adds the new datatype ID macros H5T_NATIVE_FLOAT_COMPLEX,
H5T_NATIVE_DOUBLE_COMPLEX, H5T_NATIVE_LDOUBLE_COMPLEX,
H5T_CPLX_IEEE_F16LE, H5T_CPLX_IEEE_F16BE,
H5T_CPLX_IEEE_F32LE, H5T_CPLX_IEEE_F32BE,
H5T_CPLX_IEEE_F64LE and H5T_CPLX_IEEE_F64BE
Adds hard and soft datatype conversion paths between complex number
datatypes and all the integer and floating-point datatypes, as well as
between other complex number datatypes
Adds a special conversion path between complex number datatypes and
array or compound datatypes where the in-memory layout of data is the
same between the datatypes and data can be converted directly
Adds support for complex number datatypes to the h5dump, h5ls and
h5diff/ph5diff tools. Allows h5dump '-m' option to change floating-point
printing format for float complex and double complex datatypes, as well
as long double complex if it has the same size as double complex
Adds minimal support to the h5watch and h5import tools
Adds support for the predefined complex number datatypes and
H5Tcomplex_create function to the Java wrappers. Also adds initial,
untested support to the JNI for future use with HDFView
Adds support for just the H5T_COMPLEX datatype class to the Fortran
wrappers
Adds support for the predefined complex number datatypes and
H5Tcomplex_create function to the high level library H5LT interface
for use with the H5LTtext_to_dtype and H5LTdtype_to_text functions
Changes some usages of "complex" in the library since it conflicts with
the "complex" keyword from the complex.h header. Also changes various
usages of the word "complex" throughout the library to distinguish
compound datatypes from complex datatypes.
* Split CMake HDFCompileFlags into specific compiler files
* Separate out CXX Flags
* Add Fortran compiler specific files
* Merge in #4816 changes and close#4816
* fix hanging endif
---------
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Since each API context is local to a thread, use the stack to
store the context instead of allocating & releasing it each time.
This improves performance (slightly), reduces alloc/free calls,
and eliminates the H5FL package from the push & pop operations,
which helps simplify threadsafe operation.
One effect of this change is that the H5VLstart_lib_state /
H5VLfinish_lib_state API routines for pass through connector
authors now require a parameter that can be used to store
the library's context. It was probably a mistake to assume
that these two routines would not do this previously, so this
is essentially a bug fix for them.
Some other minor things:
* Added API context push+pop operations to cache tests
(I'm not actually certain why this was working before) and
a few other places
* Cleaned up a bunch of warnings in test code (calloc args, mainly)
* Made header file inclusions more standard in some source files
- Reverts PR#1024, which (unnecessarily) switched from deferred
package initialization to centralized initialization of all packages
and introduced H5FDperform_init() to wrap an internal routine
to initialize VFD plugins.
- Went back to deferred package initialization (primarily), to
eliminate unnecessary resource use. (Performance has been
verified to be the same either way)
- Switched VFD plugins to use “#define (H5OPEN, )” pattern,
with registration of internal VFD plugins at library initialization
time. Eliminates calling API routine (H5FDperform_init) from
within the library, which was deadlocking threadsafe concurrency.
And also eliminates exposing internal library routines in a public
header file.
- Removed copy-and-paste replicas of the H5OPEN macro and put
a (better) single definition in H5public.h
- Separated API and internal routine calls in stdio and multi VFD
plugins into separate source files, so that the library doesn’t
invoke API routines internally (also a deadlock problem for
threadsafe concurrency). Also needed a “private” header for these
plugins.
- Separated registering/unregistering a VFD plugin from initializing
/finalizing the plugin, instead of blurring those ideas together.
Defers the VFD plugin init to when it’s actually used, which reduces
resource usage, especially for the MPI-based plugins like the subfiling,
etc.
- Refactored the copy-and-pasted check for locking into a central
location in the H5FD.c code.
- Fixed a bunch of compiler warnings, especially ones that trigger CI
failures with -Werror
Also cleans up the links test, which had a ton of copy-and-pasted incorrect negative error checks (Calling TEST_ERROR within H5E_BEGIN_TRY / H5E_END_TRY pairs will goto out of the pair, leaving errors suppressed).
This was an unsupported build script to make building the library
with CMake look like building it with the Autotools. It hasn't been
maintained in a long time and we've had it marked as deprecated for
years.
These files are infrequently updated and generating them adds an
annoying dependency on Perl. We're checking them in and will
probably add a GitHub action to check if anything is stale
when creating a PR.
Adds:
* H5Edefin.h
* H5Einit.h
* H5Emajdef.h
* H5Emindef.h
* H5Epubgen.h
* H5Eterm.h
* H5overflow.h
* H5version.h
