7948 Commits

Author SHA1 Message Date
Allen Byrne
73157768f9
Convert external chunking documentation to doxygen (#5131) 2024-11-20 17:55:32 -08:00
bmribler
dc4bdade49
Added note about leaving callback functions properly. (#5127)
Added the alias \callback_note to APIs with a user callback function.  The note is
brief and refers to a more detail note in a central place, i.e., alias cpp_c_api_note.

Fixes issue GH-5089
2024-11-18 10:25:14 -08:00
Dana Robinson
9019113587
Make H5F_LIBVER_LATEST a real enum value (#5083)
It was formerly a macro, but is now an enum value with the same
value as the latest numbered API.
2024-11-12 07:34:25 -06:00
Dana Robinson
faec68f936
Move overflow macros from VFDs to H5FDpkg.h (#5084)
These are identical in all VFDs except the core VFD, so they've
been moved to H5FDpkg.h and renamed:

    * MAXADDR           -->     H5FD_MAXADDR
    * ADDR_OVERFLOW     -->     H5FD_ADDR_OVERFLOW
    * SIZE_OVERFLOW     -->     H5FD_SIZE_OVERFLOW
    * REGION_OVERFLOW   -->     H5FD_REGION_OVERFLOW
2024-11-08 07:22:19 -08:00
jhendersonHDF
c130d5144d
Fix failures in dtypes test on cygwin (#5077) 2024-11-07 08:22:53 -08:00
Scot Breitenfeld
92c411b88f
Renamed H5VLstart_lib_state and H5VLfinish_lib_state (#5070) 2024-11-05 11:05:43 -06:00
bmribler
49935f8bbc
Fixed H5Ovisit2() change of behavior between 1.10.11 and v1.14.4.3 (#5022)
H5O__visit() uses the object information to be returned to the
application, so when the application did not request for certain
information, they were not available to H5O__visit.  This lack of
information caused incorrect behavior down the road.

We now call H5O_get_info again providing H5O_INFO_BASIC for "fields",
so we can obtain correct object information for H5O__visit to use.

Fixes #4941
2024-11-04 08:08:11 -08:00
Dana Robinson
8eb4c0efee
Update stdio API calls (#5064)
The stdio VFD set up ftell, etc. equivalents differently than the
rest of the library. This PR aligns them with H5private.h and
H5win32defs.h and should fix some issues with MinGW. The stdio VFD
now also uses HDoff_t, which will be a public symbol in the next
major release of HDF5.

In the main library, HDftell was set to be ftell and not ftello.
This probably isn't a big deal since the systems where it's used
all have 64-bit longs, but we should match types correctly.
2024-11-04 08:03:01 -08:00
Allen Byrne
ad3ea2633a
Move hl example into HDF5Examples (#5057) 2024-11-04 05:22:27 -08:00
Allen Byrne
3f86b4f228
Add parallel cdash script to daily-build workflow (#5032) 2024-11-01 07:54:33 -05:00
jhendersonHDF
90429f5e7d
Implement support for complex number datatypes (#4630)
* Implement support for complex number datatypes

Adds the new datatype class H5T_COMPLEX

Adds the new API function H5Tcomplex_create which creates a complex
number datatype from an ID of a base floating-point datatype

Adds the new feature check macros H5_HAVE_COMPLEX_NUMBERS and
H5_HAVE_C99_COMPLEX_NUMBERS

Adds the new datatype size macros H5_SIZEOF_FLOAT_COMPLEX,
H5_SIZEOF_DOUBLE_COMPLEX and H5_SIZEOF_LONG_DOUBLE_COMPLEX

Adds the new datatype ID macros H5T_NATIVE_FLOAT_COMPLEX,
H5T_NATIVE_DOUBLE_COMPLEX, H5T_NATIVE_LDOUBLE_COMPLEX,
H5T_CPLX_IEEE_F16LE, H5T_CPLX_IEEE_F16BE,
H5T_CPLX_IEEE_F32LE, H5T_CPLX_IEEE_F32BE,
H5T_CPLX_IEEE_F64LE and H5T_CPLX_IEEE_F64BE

Adds hard and soft datatype conversion paths between complex number
datatypes and all the integer and floating-point datatypes, as well as
between other complex number datatypes

Adds a special conversion path between complex number datatypes and
array or compound datatypes where the in-memory layout of data is the
same between the datatypes and data can be converted directly

Adds support for complex number datatypes to the h5dump, h5ls and
h5diff/ph5diff tools. Allows h5dump '-m' option to change floating-point
printing format for float complex and double complex datatypes, as well
as long double complex if it has the same size as double complex

Adds minimal support to the h5watch and h5import tools

Adds support for the predefined complex number datatypes and
H5Tcomplex_create function to the Java wrappers. Also adds initial,
untested support to the JNI for future use with HDFView

Adds support for just the H5T_COMPLEX datatype class to the Fortran
wrappers

Adds support for the predefined complex number datatypes and
H5Tcomplex_create function to the high level library H5LT interface
for use with the H5LTtext_to_dtype and H5LTdtype_to_text functions

Changes some usages of "complex" in the library since it conflicts with
the "complex" keyword from the complex.h header. Also changes various
usages of the word "complex" throughout the library to distinguish
compound datatypes from complex datatypes.
2024-10-31 13:54:25 -07:00
Dana Robinson
e8257bd2b5
Make H5T__free more aggressive about cleanup (#5054)
oss-fuzz often trips over unfreed datatype memory when parsing
fuzzed files. This changes H5T__free() to use HDONE macros so
cleanup continues on errors.
2024-10-31 09:43:04 -05:00
Dana Robinson
92033dff84
malloc --> calloc in H5B cache entry (#5053)
oss-fuzz occasionally complains about intermittent undefined behavior
when when Boolean variables in H5C_cache_entry_t (like is_protected)
are set to values other than 0 or 1 when parsing fuzzed files. Using
calloc() to iniitialize the H5B_t struct (like other metadata already
does) should fix this.
2024-10-31 09:38:50 -05:00
Dana Robinson
1e977c1db4
Remove comment about multi VFD duplication (#5029) 2024-10-29 10:18:26 -07:00
Dana Robinson
1cd0168e6e
Rework FUNC_ENTER macros (#5024)
* Initial cleanup of "prep" macros
* Cleanup of initialization routines
* Renamed init/push macros
2024-10-29 05:58:30 -07:00
bmribler
5bdd379a64
Fix segfault in h5dump caused by corrupted btree node level (#5002)
Added another argument, expected node level, to H5B__iterate_helper to pass down
to H5B__cache_deserialize for checking the decoded node level.  When this expected
level is not known, the new macro H5_UNKNOWN_NODELEVEL (-1) will be used for not
checking the level.

Fixes GH-4432
2024-10-29 00:53:50 -07:00
Dana Robinson
60dd8526d7
Fix bad private FUNC_ENTER macros (#5020)
The name check in the FUNC_ENTER checks for private FUNC_ENTER macros
(which have NOAPI in the name) only checked for not having the
public form (H5Xfoo), so they could be inappropriately applied to
package-level functions (H5X__foo).

This PR fixes the bug and updates the inappropriate macros.
2024-10-28 07:35:33 -07:00
Dana Robinson
a38d87a631
Clean up comments in FUNC_ENTER macros (#5019)
* Clean up comments in FUNC_ENTER macros
2024-10-28 05:54:39 -07:00
Dana Robinson
369099d843
Fix minor Windows warnings (#5021)
* Cast away a signed/unsigned issue in H5HFhuge.c
* Use our platform-independent POSIX types in external_common.c
  and h5test.c
* Replace a memset() call with a (const) array initializer in h5test.c
* Fix an unused done: target in h5diff
2024-10-28 05:54:16 -07:00
Allen Byrne
5425a571e0
Convert develop to v2.0.0 (#5006)
Switches previous 1.16/17/18 values to 2.0
2024-10-27 21:51:07 -07:00
Dana Robinson
8b2f40f2ff
Remove duplicate sec2 VFD ID variable (#5013) 2024-10-25 15:12:11 -07:00
Allen Byrne
6122828198
Split out compiler flags into specific compiler files (#4974)
* Split CMake HDFCompileFlags into specific compiler files

* Separate out CXX Flags

* Add Fortran compiler specific files

* Merge in #4816 changes and close #4816

* fix hanging endif

---------

Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
2024-10-25 10:03:35 -05:00
Larry Knox
0439876eeb
Switch default for HDF5_GENERATE_HEADERS (CMake) to OFF. (#5005) 2024-10-24 22:31:15 -07:00
Quincey Koziol
97e1ed4fc8
Refactor allocation of API context (#4942)
Since each API context is local to a thread, use the stack to
store the context instead of allocating & releasing it each time.
This improves performance (slightly), reduces alloc/free calls,
and eliminates the H5FL package from the push & pop operations,
which helps simplify threadsafe operation.

One effect of this change is that the H5VLstart_lib_state /
H5VLfinish_lib_state API routines for pass through connector
authors now require a parameter that can be used to store
the library's context. It was probably a mistake to assume
that these two routines would not do this previously, so this
is essentially a bug fix for them.

Some other minor things:

 * Added API context push+pop operations to cache tests
  (I'm not actually certain why this was working before) and
  a few other places
* Cleaned up a bunch of warnings in test code (calloc args, mainly)
* Made header file inclusions more standard in some source files
2024-10-24 10:09:22 -07:00
Quincey Koziol
2c58357b8f
Refactor H5FD and package initialization (#4934)
- Reverts PR#1024, which (unnecessarily) switched from deferred
  package initialization to centralized initialization of all packages
  and introduced H5FDperform_init() to wrap an internal routine
  to initialize VFD plugins.
- Went back to deferred package initialization (primarily), to
  eliminate unnecessary resource use. (Performance has been
  verified to be the same either way)
- Switched VFD plugins to use “#define (H5OPEN, )” pattern,
  with registration of internal VFD plugins at library initialization
  time. Eliminates calling API routine (H5FDperform_init) from
  within the library, which was deadlocking threadsafe concurrency.
  And also eliminates exposing internal library routines in a public
  header file.
- Removed copy-and-paste replicas of the H5OPEN macro and put
  a (better) single definition in H5public.h
- Separated API and internal routine calls in stdio and multi VFD
  plugins into separate source files, so that the library doesn’t
  invoke API routines internally (also a deadlock problem for
  threadsafe concurrency). Also needed a “private” header for these
  plugins.
- Separated registering/unregistering a VFD plugin from initializing
  /finalizing the plugin, instead of blurring those ideas together.
  Defers the VFD plugin init to when it’s actually used, which reduces
  resource usage, especially for the MPI-based plugins like the subfiling,
  etc.
- Refactored the copy-and-pasted check for locking into a central
  location in the H5FD.c code.
- Fixed a bunch of compiler warnings, especially ones that trigger CI
  failures with -Werror
2024-10-24 06:28:40 -07:00
Allen Byrne
26cf1640b2
Add szip documentation to dataset section of user guide (#4997) 2024-10-23 19:37:15 -07:00
Allen Byrne
1b72a1bca0
Add support for native zlib-ng in the library and compression references (#4935)
* Correct compile defs for zip_perf program
2024-10-23 16:20:15 -05:00
Quincey Koziol
ed4419cc52
Remove another couple of H5E_clear_stack calls (#4968)
Also cleans up the links test, which had a ton of copy-and-pasted incorrect negative error checks (Calling TEST_ERROR within H5E_BEGIN_TRY / H5E_END_TRY pairs will goto out of the pair, leaving errors suppressed).
2024-10-23 10:13:57 -07:00
Allen Byrne
baa1e8e292
Prefix remaining CMake options except for CMake BUILD* variables (#4990) 2024-10-22 15:40:54 -07:00
Dana Robinson
680cdd08bb
Check in generated files in src (#4981)
These files are infrequently updated and generating them adds an
annoying dependency on Perl. We're checking them in and will
probably add a GitHub action to check if anything is stale
when creating a PR.

Adds:
    * H5Edefin.h
    * H5Einit.h
    * H5Emajdef.h
    * H5Emindef.h
    * H5Epubgen.h
    * H5Eterm.h
    * H5overflow.h
    * H5version.h
2024-10-20 12:48:46 -07:00
Dana Robinson
245bb2cd23
Remove some Solaris Studio work-arounds (#4979)
Solaris Studio hasn't been updated in almost a decade and the last
version (12.4, circa 2015) doesn't seem to fully support C11.

This PR removes some work-arounds for things like __attribute__()
support.
2024-10-20 12:47:52 -07:00
Dana Robinson
7f1e49206d
Renamed COPYING to LICENSE (#4978)
This is where most people will expect to find license information. The
COPYING_LBNL_HDF5 file has also been renamed to LICENSE_LBNL_HDF5.
The licenses are unchanged.
2024-10-18 21:13:04 -07:00
Dana Robinson
bb5661101a
Remove the high-level GIF tools (#4976)
The high-level GIF tools, h52gif and gif2h5, have unfixed CVE issues
(with no proof-of-concept files). They are not critical tools, are not
well maintained, and are an odd fit for building with the library.
Because of this, they have been removed. We may move them to a separate
repository in the future.
2024-10-18 12:09:42 -07:00
Dana Robinson
478c7d166f
Set the C standard to 11 (#4975)
Ignores some of the older Autotools platform files, since the Autotools
will be dropped in the next major release (and we don't have
compilers on which to test, anyway).

Also drops support for the old, non-compliant MSVC
preprocessor.
2024-10-18 12:04:27 -07:00
Dana Robinson
4a10a06072
Switch HDoff_t from __int64 to int64_t on Windows (#4973)
__int64 raises warnings when building with clang
2024-10-18 09:51:47 -07:00
Dana Robinson
e12f51381d
Move warning suppression macros to H5warnings.h (#4972) 2024-10-18 09:21:43 -07:00
Dana Robinson
bfcb91652a
Moved timer functionality to (new) H5timer.h (#4970) 2024-10-18 07:58:43 -07:00
Dana Robinson
1c23395bd6
Remove TRUE and FALSE from H5private.h (#4969)
* Remove TRUE and FALSE from H5private.h

* Replace hbool_t with bool in test code
2024-10-18 07:57:34 -07:00
Allen Byrne
29c84e0d59
Incorporate remaining parts of the filters.html file (#4966) 2024-10-17 17:10:24 -07:00
Scot Breitenfeld
ad307bf61f
Updated hyperslab documentation (#4965)
* corrected Fortran docs for hyperslab selection

* updated hyperslab op doc
2024-10-17 09:17:20 -07:00
Dana Robinson
fc5b66b1a8
Only clear FE_INVALID when that symbol is present on the system (#4954)
When we initialize the floating-point types at library startup, it's
possible to raise floating-point exceptions when we check which things
are supported. Normally, we clear these floating-point exceptions via
feclearexcept(FE_INVALID), but FE_INVALID may not be present on all
systems. Specifically, this was reported as being a problem when using
Emscripten 3.1.68 to compile HDF5 1.14.5 to WebAssembly.

We've added an #ifdef FE_INVALID block around the exception clearing
code to correct this.

Fixes #4952
2024-10-14 14:30:22 -07:00
vchoi-hdfgroup
9df4c0dad4
Fix for issue #4849 that settings in fapl libver bounds causes unexpe… (#4939)
* Fix for issue #4849 that settings in fapl libver bounds causes unexpected H5Fopen failures.
File with non-SWMR-write access can now be opened without regard for superblock version.
Due to the fix, H5Fstart_swmr_write() also needs to be modified as well as the tests for libver bounds.
The "RFC: Setting Bounds for Object Creation in HDF5 1.10.0" is also updated to reflect the changes.

* Fix c++ libver bound test failure.
2024-10-11 07:38:22 -05:00
Allen Byrne
6e8c7a9597
add analysis and coverage to CDash reports (#4938)
* Add sanitzers and coverage workflows
* add UndefinedBehavior sanitize
2024-10-10 13:16:36 -05:00
Dana Robinson
54f8f12ed9
Update clang-format to 17 (#4931)
Also bump the clang-format GitHub actions to 17
2024-10-04 19:49:35 -07:00
Quincey Koziol
767282f68a
VOL refactor and cleanup (#4856)
Cleanup and prepare for thread-safety changes.

Big ideas:

* Wrap H5VL_class_t with H5VL_connector_t, so use of the class can be refcounted within the H5VL package, instead of relying on storing an ID within the H5VL_t struct and incrementing & decrementing the ID's refcount.
* Register H5VL_connector_t* for VOL connector IDs, instead of the H5VL_class_t*
* Stop other packages from rummaging around inside H5VL_connector_t and H5VL_object_t data structures, so that the H5VL package can change implementation details without coupled changes throughout the library

Small things:

* Simplified the coding for creating links
* Moved some routines into more logical locations
2024-10-03 12:19:33 -07:00
Quincey Koziol
626f639850
Remove H5E_clear_stack() from H5SM code (#4861)
Introduces 'try' flag for H5B2_modify() call
2024-10-02 21:55:37 -05:00
Allen Byrne
f4105bec7c Fix typo 2024-10-02 19:53:38 -05:00
Allen Byrne
07756c87f5 Update develop to 1.18 API 2024-10-02 19:53:38 -05:00
Larry Knox
698ee8e439
Set develop branch version to 1.17.0. (#4912) 2024-10-02 12:28:19 -05:00
Matt L
e2da35371b
Clean up thread-local error stacks in all threads (#4852)
* Clean up error stacks from secondary threads
2024-10-01 08:53:53 -07:00