Removes H5detect and H5make_libsettings from the build and replaces
their functionality with things that don't affect cross-compiling.
H5detect --> floating-point types are now detected on library load
H5make_libsettings --> Moved functionality to a new H5build_settings.c template file
Several options and public symbols that were provided to paper
over non-standard long double conversions between signed/unsigned
long and long long values were removed from the Autotools and
CMake. These were added twenty years ago, when C99 and 64-bit
platforms were less common and are no longer needed.
Autotools:
--enable-dconv-accuracy
CMake:
HDF5_WANT_DATA_ACCURACY
H5pubconf.h symbols:
H5_WANT_DATA_ACCURACY
H5_LDOUBLE_TO_LONG_SPECIAL
H5_LONG_TO_LDOUBLE_SPECIAL
H5_LDOUBLE_TO_LLONG_ACCURATE
H5_LLONG_TO_LDOUBLE_CORRECT
H5_DISABLE_SOME_LDOUBLE_CONV
When copying an empty enum type (including implicitly, as when an enum
is contained in a compound type), the library would allocate 0-size
blocks of memory and attempt to memcpy 0 bytes from NULL pointers, which
are undefined behavior. In debug mode, the library would raise an
assert in H5MM.
The library now avoid undefined memory operations when copying empty
enum types and a test that copies empty enums has been added.
* Remove macros from api tests (#2929)
* Remove macros and undefined callbacks (#2959)
* Remove remaining macros from H5_api_tests_disabled.h (#2968)
* Put some vol capability checks in testpar tests and remove remaining warnings (#2995)
* API tests datatype generation cleanup
* Clean up API tests' random datatype generation and fix bug with enum
datatype generation
* Init parallel API tests with MPI_THREAD_MULTIPLE
* HDF5 API tests - Check VOL connector registration
* Determine whether a VOL connector failed to load before running API
tests
* Cleanup some usages of H5VL_CAP_FLAG_CREATION_ORDER in API tests
* Remove some now-unused macros from H5_api_tests_disabled.h
* Enable HDF5 API tests by default
* Implement CMake option to install HDF5 API tests
* Check for invalid AAPL from H5Acreate
* Enable building of VOL connectors alongside HDF5 in CMake
* Prepend CMake VOL URL option indices with 0s so they come in order
* Don't turn on API tests by default yet
* Document VOL connector FetchContent functionality
* Add release note for API test updates
* Only install testing library if API tests are installed
* Fix grammar
Initial implementation of selection I/O with type conversion. Allows
Parallel collective I/O with type conversion, as long as selection I/O
is enabled.
Malformed object header continuation messages can result in a too-small
buffer being passed to the decode function, which could lead to reading
past the end of the buffer. Additionally, errors in processing these
malformed messages can lead to allocated memory not being cleaned up.
This fix adds bounds checking and cleanup code to the object header
continuation message processing.
Fixes#2604
* Fixed GH-2603, heap-buffer-overflow in H5O__linfo_decode
Verified with valgrind -v --tool=memcheck --leak-check=full h5dump POV-GH-2603
The several invalid reads shown originally are now gone.
* Fix for github issue #2599:
As indicated in the description, memory leak is detected when running "./h5dump pov".
The problem is: when calling H5O__add_cont_msg() from H5O__chunk_deserialize(),
memory is allocated for cont_msg_info->msgs. Eventually, when the library tries to load
the continuation message via H5AC_protect() in H5O_protect(), error is
encountered due to illegal info in the continuation message.
Due to the error, H5O_protect() exits but the memory allocated for cont_msg_info->msgs is not freed.
When we figure out how to handle fuzzed files that we didn't generate,
a test needs to be added to run h5dump with the provided "pov" file.
* Add message to release notes for the fix to github issue #2599.
* Remove duplicate code
Signed-off-by: Egbert Eich <eich@suse.com>
* Add test case for CVE-2021-37501
Bogus sizes in this test case causes the on-disk data size
calculation in H5O__attr_decode() to overflow so that the
calculated size becomes 0. This causes the read to overflow
and h5dump to segfault.
This test case was crafted, the test file was not directly
generated by HDF5.
Test case from:
https://github.com/ST4RF4LL/Something_Found/blob/main/HDF5_v1.13.0_h5dump_heap_overflow.md
* Add fetchcontent for compression libs and fix cmake config
* MSDOS is a reserved define name
* Add release note and update install doc for FetchContent
* Add CI test for FetchContent
* Use LINK_COMP_LIBS instead of STATIC_LIBRARY for depends
* Use general link
* CMake generated pkg-config file is incorrect #2259
* Fix fortran pc template
* hdf5.pc is incorrect for debug builds #1546
* Correct pkg name and lib name
* Fix typo
* Fix missing space
* H5F_LIBVER_LATEST changes for move to 1.15
* Add new default api check
* Format fixes
* Fix default configure
* fix lib version tests
* Fix another version variable
* Add 1.14 doc link
* Fix for HDFFV-11052: h5debug fails on a corrupted file (h5_nrefs_POC) producing a core dump.
When h5debug closes the corrupted file, the library calls H5F__dest() which performs all the
closing operations for the file "f" (H5F_t *) but just keeping note of errors in "ret_value"
all the way till the end of the routine. The user-provided corrupted file has an illegal
file size causing failure when reading the image during the closing process.
At the end of this routine it sets f->shared to NULL and then frees "f".
This is done whether there is error or not in "ret_value".
Due to the failure in reading the file earlier, the routine then returns error.
The error return from H5F__dest() causes the file object "f" not being removed from the
ID node table. When the library finally exits, it will try to close the
file objects in the table. This causes assert failure when H5F_ID_EXISTS(f) or H5F_NREFS(f).
Fix:
a) H5F_dest(): free the f only when there is no error in "ret_value" at the end of the routine.
b) H5VL__native_file_close(): if f->shared is NULL, free "f"; otherwise, perform closing on "f" as before.
c) h5debug.c main(): track error return from H5Fclose().
* Committing clang-format changes
* Add test and release note info for fix to HDFFV-11052 which is merged via PR#2291.
* Committing clang-format changes
* Add the test file to Cmake.
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Compound datatypes may not have members of size 0
A member size of 0 may lead to an FPE later on as reported in
CVE-2021-46244. To avoid this, check for this as soon as the
member is decoded.
This should probably be done in H5O_dtype_decode_helper() already,
however it is not clear whether all sizes are expected to be != 0.
This fixes CVE-2021-46244 / Bug #2242.
Signed-off-by: Egbert Eich <eich@suse.com>
* Rework error recovery code in H5O__dtype_decode_helper() and
H5O__dtype_decode().
* Format changes for src/H5Odtype.c.
Signed-off-by: Egbert Eich <eich@suse.com>
Co-authored-by: Neil Fortner <nfortne2@hdfgroup.org>
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* When evicting driver info block, NULL the corresponding entry
Since H5C_expunge_entry() called (from H5AC_expunge_entry()) sets the flag
H5C__FLUSH_INVALIDATE_FLAG, the driver info block will be freed. NULLing the
pointer in f->shared->drvinfo will prevent use-after-free when it is used in other
functions (like H5F__dest()) - as other places will check whether the pointer is
initialized before using its value.
This fixes CVE-2021-46242 / Bug #2254
Signed-off-by: Egbert Eich <eich@suse.com>
* When evicting the superblock, NULL the corresponding entry
The call to H5AC_expunge_entry() will free the corresonding structure,
to avoid a use-after-free, the corrsponding pointer entry will be NULLed.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.com>
For Data Layout Messages version 1 & 2 the specification state
that the value stored in the data field is 1 greater than the
number of dimensions in the dataspace. For version 3 this is
not explicitly stated but the implementation suggests it to be
the case.
Thus the set value needs to be at least 2. For dimensionality
< 2 an out-of-bounds access occurs as in CVE-2021-45833.
This fixes CVE-2021-45833 / Bug #2240.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.com>
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
The size of the enumeration values is determined by the size of the parent.
Functions accessing the enumeration values use the size of the enumeration
to determine the size of each element and how much data to copy. Thus the
size of the enumeration and its parent need to match.
Check here to avoid unpleasant surprises later.
This fixes CVE-2018-14031 / Bug #2236.
Signed-off-by: Egbert Eich <eich@suse.com>
According to the specification, the information block for external links
contains 1 byte of version/flag information and two 0 terminated strings
for the object linked to and the full path.
Although not very useful, the minimum string length for each (with
terminating 0) would be one byte.
Checking this will help to avoid SEGVs triggered by bogus files.
This fixes CVE-2018-16438 / Bug #2233.
Signed-off-by: Egbert Eich <eich@suse.com>
Initially, the accumulated metadata location is initialized to HADDR_UNDEF
- the highest available address. Bogus input files may provide a location
or size matching this value. Comparing this address against such bogus
values may provide false positives. This make sure, the value has been
initilized or fail the comparison early and let other parts of the
code deal with the bogus address/size.
Note: To avoid unnecessary checks, we have assumed that if the 'dirty'
member in the same structure is true the location is valid.
This fixes CVE-2018-13867 / Bug #2230.
Signed-off-by: Egbert Eich <eich@suse.com>
When decoding a file space info message in H5O__fsinfo_decode() make
sure each element to be decoded is still within the message. Malformed
hdf5 files may have trunkated content which does not match the
expected size. Checking this will prevent attempting to decode
unrelated data and heap overflows. So far, only free space manager
address data was checked before decoding.
This fixes CVE-2021-45830 / Bug #2228.
Signed-off-by: Egbert Eich <eich@suse.com>
Additions
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Malformed hdf5 files may provide more dimensions than the array dim[] in
H5IMget_image_info() is able to hold. Check number of elements first by calling
H5Sget_simple_extent_dims() with NULL for both 'dims' and 'maxdims' arguments.
This will cause the function to return only the number of dimensions.
The fix addresse a stack overflow on write.
This fixes CVE-2018-17439 / HDFFV-10589 / Bug #2226.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.com>
* Removes MPE instrumentation support.
The Autotools will no longer accept --with-mpe= and the logging commands
have been removed from the FUNC_ENTER macros. CMake has never supported
instrumenting for MPE.
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
This has not been used to debug the library in a very long time. Most
developers use valgrind, -fsanitize=address, or some other memory checker
instead of this library.
This removes:
* dmalloc.h include from H5private.h
* --with-dmalloc= Autotools configure option
* HDF5_ENABLE_USING_DMALLOC CMake option
* Update link for Software Changes in RELEASE.txt.
Add RELEASE.txt contents from 1.13.3 release to HISTORY-1_13.txt.
* Reverse order of entries in HISTORY-1_13.txt to match the order in previous HISTORY
files.
* Some Javadoc warning fixes
* Updated javadoc fixes
* # WARNING: head commit changed in the meantime
HDFFV-11229 Fix long double usage in tools and java
Changed h5dump and h5ls to just print 128-bit for long double type.
Added test and file for dataset and attributes with all three float
types.
* Committing clang-format changes
* HDFFV-11229 Add release note
* HDFFV-11229 windows testfile needed
* fix typo
* Remove non supported message text
* HDFFV-11229 - change ldouble test to check both native and general
* HDFFV-11229 add second file compare
* HDFFV-11229 fix reference file
* HDFFV-11229 autotools check two refs
* HDFFV-11229 revert back to removal of NATIVE_LDOUBLE in tools output
* Committing clang-format changes
* Update release note
* Update attribute type of ref file
* Change source of ninja for macs
* try port instead of brew
* Recommended is to use brew.
* Undo non long double changes
* remove unneeded file
* Fix perl and doxygen CMake code
* Add "option" command for clang options
* Rework CMake add_custom to use the BYPRODUCTS argument
* Add stamp files to BYPRODUCTS
* Only one copy of file exists
* Fix custom cmmand depends targets
* Fix fortran custom command DEPENDS
* Add LD_LIBRARY_PATH to tests
* Add custom target for DEPENDS
* Add h5detect conditionaly to generated target DEPENDS
* Correct DEPENDS targets
* Parallel builds need the mpi compiler for pkgconfig scripts.
* install only if MPI build
* Fortran target depends
* Remove incorrect source attribute
* doxygen adjustments
* doxygen build updates
* Correct version
* Correct function version - function has been merged to 1.12
* Correct version string for map functions
* Cleanup warnings for tools debug builds
* TRILAB-227 - fix indexing for h5diff selections
* Correct location of pos to index function call
* TRILAB-227 Enable test
* Quote subset args
* Use MATCHES because of AppleClang
* if blocks needed for build modes
* Update list of DT platforms
* VS2019 correctly displays float values
* revert VS2019 change
* Issue #669 remove version from pkgcfg filename
* remove version from h5cc script
* Java reference functions updated
enabled fortran in cmake gcc action yaml file
java reference test changed to correctly test refs
jni reference functions that create ids changed to use jni id logging
* Correct BYPRODUCTS argument
* Correct more genereated files BYPRODUCTS
* BYPRODUCTS must have unique locations
* Fix typo
* Fix fortran configure checks
* Rework H5_PAC_C_MAX_REAL_PRECISION setting logic
* Add note about fortran configure change
* Adds a quick for for some egregious chunk_info badness (#722)
* Fixes issue with ccmake that prevents building Fortran (#723)
ccmake runs iteratively, and the check_fortran_source_runs
macros were clobbering a single output file that did not
get updated on further configure iterations
* Fix conflicts with merge
* Move MAX_PRECISION back to HDF5UseFortran.cmake
* Use STREQUAL to test macro argument
* Move C language test to ConfigureChecks from HDF5UseFortran
* MAX_PRECISION defines must be defined
* Organize flags and align autotools and cmake
* Fix comment in no-error-general warnings files.
* Flag cleanup and fix typos
* Add comment
* Correct VAR used to find configure time file
* Set the path correctly
* Update missing release note info.
* Update code owners
* Correct JIRA note
* add known problem.
* Use only core library for testing dynamic plugins.
* Committing clang-format changes
* Update main.yml
* Update HISTORY-1_13.txt
Missing release note added for changed location of CMake configuration files.
* Update HISTORY-1_13.txt
Fix typo
* Update main.yml
revert change
* Updated version to 1.13.4-1 after creating hdf5_1_13_3 branch for 1.13.3
release.
Added new version to exceptions in lines 74-5 of src/H5.c
* Cleared RELEASE.txt entries following HDF5 1.13.3 release.
Co-authored-by: Allen Byrne <byrn@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
* Fix for HDFFV-10840: Instead of using fill->buf for datatype conversion
if it is large enough, a buffer is allocated regardless so that the element
in fill->buf can later be reclaimed.
Valgrind is run on test/set_extent.c and there is no memory leak.
* Add information of this fix to release notes.
Co-authored-by: vchoi <vchoi@jelly.ad.hdfgroup.org>
* Fix bug in attribute type conversion where the background buffer would
not be initialized with the destination contents when necessary. Other
minor simplification.
* Committing clang-format changes
* Fix warnings.
* Address review comments.
* Add RELEASE.txt note for PR #2016
* Add GitHub number to release note
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fix test failure
Description:
- Added two H5File constructors to open file with non-default fapl
- Added non-default fapl to test functions
- Commented certain renaming attribute tests because of the unexpected
behavior in renaming an attribute (HDFFV-11327)
Platform tested:
Linux/64 (jelly)
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Rename Subfiling IOC "thread_pool_count" field to "thread_pool_size"
* Add simple HDF5 example for Subfiling VFD
* Subfiling VFD - never cache app topology as it may change
* Subfiling VFD - cleanup unused funtionality and tidy up some TODOs
* Subfiling VFD - tidy up subfiling error handling in H5subfiling_common.c
* Subfiling VFD - show number of failed I/O requests on close
* Subfiling VFD - Update file cmp callback after switching to MPI I/O VFD
* Amend RELEASE.txt with info about h5fuse.sh and Subfiling limitations
* Subfiling VFD - switch to using H5_basename and H5_dirname
* Remove generated file h5fuse.sh
* Link pthreads library when Subfiling VFD is built
* Switch to MPI I/O driver for Subfiling HDF5 stub file
* Rough first implementation for Subfiling file deletion
* Subfiling VFD - get file dirname for file deletion
* Subfiling VFD - set lock callback to NULL for now to avoid performance
issues
* Committing clang-format changes
* Minor tidying up of Subfiling testing
* Fixups for Subfiling VFD support in tools
* Tidy up Subfiling public interface and add Doxygen
* Respect Subfiling configuration settings from application
* Add release note for Subfiling VFD
* Committing clang-format changes
* Committing clang-format changes
* Shorten some Subfiling environment variable names
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Removes the small perf tool
This tool doesn't really do anything special and installed, which
conflicts with gnu's perf tool.
* Adds suggestions from code review
* Rework java vl check in read/write. Handle old refs in h5dump
* Committing clang-format changes
* Java changes allow default VL reads
* Fix JNI utility for old refs
* HDFFV-11310 - implement vlen read/write for atomic types.
* format check fix
* Mostly format fixes
* More format issues
* Two format changes
* Use JNI names for sizeof
* format change
* fix size typo
* Change to older method to initialize list
* remove unused var
* format fix
* switch writeVL functions to use datatype instead of java type
* Add VL option to generic read/write check
* Correct function name typo
* Add JIRA issue
* Correct note to match change in code.
* HDFFV-11318 add VL references as byte arrays
* Add release note and format changes
* Another format update
* Fix unreleased allocations
* Fix format
* format correction
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fixed -Wreserved-id-macro warnings from header include guards
* Removed all __int64 and LL suffix stuff now that C99 is minimum requirement
* Rename `H5FD_CTL__` to `H5FD_CTL_` to fix -Wreserved-id-macro warnings
Double underscore is reserved in C++ and this public header should be C++ compatible.
* Never define __STDC_FORMAT_MACROS anymore
Defining it causes a -Wreserved-id-macro.
Happily, according to the C++11 standard:
"The macros defined by <stdint> are provided unconditionally. In particular, the symbols __STDC_LIMIT_MACROS and __STDC_CONSTANT_MACROS (mentioned in C99 footnotes 219, 220, and 222) play no role in C++."
https://cplusplus.github.io/LWG/issue984
So looks like it's not necessary to define it with reasonably new toolchains.
* Fixed some -Wunused-macros warnings, removed dead code
* Fixed all -Wdouble-promotion warnings in C++ files
* Fixed remaining -Wsuggest-destructor-override warnings
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Committing clang-format changes
* Spelling of preceed was corrected to proceed, but should have been
corrected to precede.
* Correct spelling correction of 'preceed' incorrectly to 'proceed'. It should be 'precede'.
* Update version to 1.13.2-1 after 1.13.1 release.
Add Makefile.in to MANIFEST for addition of utils/tools and h5dwalk.
* Update VERS_RELEASE_EXCEPTIONS with new incompatible release version.
* Add HDF5 1.13.1 RELEASE.txt to HISTORY-1_13.txt contents to
HISTORY-1_13.txt.
Clean entries from RELEASE.txt.
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
When resizing a pinned/protected cache entry, the metadata
cache code previously would wait until after resizing the
entry to attempt to log the newly-dirtied entry. This would
cause H5C_resize_entry to mark the entry as dirty and make
H5AC_resize_entry think that it doesn't need to add the
newly-dirtied entry to the dirty entries skiplist.
Thus, a subsequent H5AC__log_moved_entry would think it
needs to allocate a new entry for insertion into the dirty
entry skip list, since the entry doesn't exist on that list.
This causes an assertion failure, as the code to allocate a
new entry assumes that the entry is not dirty.
* Fix several places where rank 0 can skip past collective MPI operations on failure
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Convert SWMR shell scripts to Windows powershell.
* Use $LastExitCode instead of $? in script
* Prevent execution in a different window
* Github #969 Use stdout instead of file for configure check (#1089)
* Use stdout instead of file for configure check
* Make change requested by #1157
* Change fortran to use stderr for configure
* Correct typo
* remove obsolete file check
* Fortran statement fix
* Don't allow H5Pset(get)_all_coll_metadata_ops for DXPLs (#1201)
* Fixes const warnings in H5ES package (#1211)
* Quiets a 'set but not used' warning in h5diff_array.c (#1210)
* Convert vds swmr test script to powershell
* Enable vdsswmr powershell test
* Add configure vdsswmr statement
* Correct powershell script vars
* Convert SWMR shell scripts to Windows powershell.
* Use $LastExitCode instead of $? in script
* Prevent execution in a different window
* Github #969 Use stdout instead of file for configure check (#1089)
* Use stdout instead of file for configure check
* Make change requested by #1157
* Change fortran to use stderr for configure
* Correct typo
* remove obsolete file check
* Fortran statement fix
* Convert vds swmr test script to powershell
* Enable vdsswmr powershell test
* Add configure vdsswmr statement
* Correct powershell script vars
* Cleanup review issues
* Fix variable assignment
* Change the wait function to pass in the path
* Disable actual test execution until programs fixed
* Adjust copyright text
* Fix spelling
Co-authored-by: jhendersonHDF <jhenderson@hdfgroup.org>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Adds tool h5dwalk and configure options to enable building it.
Co-authored-by: Richard Warren <Richard.Warren@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Richard.Warren <richard.warren@jelly.ad.hdfgroup.org>
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
* Fixes an assert in H5Pget_filter_by_id1/2 w/ out-of-range IDs
Filter IDs < 0 or > H5Z_FILTER_MAX could trip an assert in the
library due to missing ID range checks in H5Pget_filter_by_id1/2.
The library now returns a normal error code when filter IDs are
out of range. Fixes HDFFV-11286.
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Enable usage of new-style references with dimension scale APIs
* Add API to check if an object ID represents a native connector object
* Modified code to use new function H5DSwith_new_ref to determine if new references should be used with
Dimension Scales. The new function return TRUE if non-native connector is used or if H5_DIMENSION_SCALES_WITH_NEW_REF
varible is define at configure time (--enable-dimension-scales-with-new-ref).
Tested on jelly.
ToDo: generate testing file on BE system and enable the test; add flag to CMake; test netCDF-4 with the new references.
* Adding new test files generated on BE system (hedgehog) created by 32 and 64-bit library.
test_ds chokes on test_ds_le_new_ref.h5 on BE system; test passes for test_ds_be_new_ref-32bit.h5
for the 32-bit library and fails for the 64-bit library, and vice versa. I am checking the files for further
investigation; but current implementation of the new references is not portable between LE and BE systems,
and 32 and 64-bit systems.
* Minor fixes for testing issues
* Update test_ds.c
Enabled broken test; tests pass now.
* Update RELEASE.txt
Documented new option to use new references with the HDF5 dimension scales APIs (H5DS*).
* Update MANIFEST for new 32-bit new-style references test file for H5DS APIs
* Update 'dimension scales w/ new-style refs' feature based on review
Co-authored-by: Elena <epourmal@hdfgroup.org>
* Move test utilities to utils/test folder
* Fix makefile assignment
* Add new dir
* add new folder
* Correct copied makefile
* Fix dir typo
* Add missing include dir
* Remove unnecessary lib link
* Correct dependent dirs
* Fix conditional checks
* Disable test if not built
* fix path to executable
* Use fixture for swmr_vfd check
* Add release note
* Correct shell tests and c++ flag warning
* Update autotools c++ warning
* Fix typo
* Avoid maintenance headaches: delete the `noerror-` variants of the
compiler flags files, since they essentially duplicate the `error-`
files modulo the replacement of `-Werror=` with `-W` and any changes in
comments. (I verified the duplication with a script.)
For autoconf, reinstate the use of the `demote_errors` shell function to
derive the `noerror-` content from the `error-` content. `demote_errors`
replaces `-Werror=` with `-W` when `WARNINGS_AS_ERRORS` is `no`.
Slightly reorder `configure.ac` so that the setting of
`WARNINGS_AS_ERRORS` takes effect before the `error-` files are sourced.
* Take a stab at updating the CMake files to match the changes I made to
the autoconf files to remove `noerror-` files. I'm not much of a CMake
user so these changes are quite rough.
Looks like the duplication can be reduced with the introduction of a new
macro.
* Delete `noerror-` files from the MANIFEST.
* Reduce duplication in the CMake files: perform the
HDF5_ENABLE_WARNINGS_AS_ERRORS test once in the ADD_H5_FLAGS macro.
* Add a release note.
* Update compact dataset I/O routines to handle driver-level memory copy
* Combine compact I/O read/write struct into single I/O struct
Rename CTL memory copy flag and H5Fquery routine to get file driver
structure
* Add RELEASE.txt entry for new "ctl" memory copy operation
* Update Platforms Tested in RELEASE.txt.
Remove Tested Configuration Features Summary for updated replacement.
* Set version to 1.13.0-6 for rc6 snapshot.
* Added "ctl" callback to the VFD interface, and the associated
H5FDctl() and H5FD_ctl() calls.
Modified the MPIO VFD accordingly -- specifically:
Added ctl() call with op-code support to expose rank, size,
and communicator.
Modified H5FD_mpi_get_rank(), H5FD_mpi_get_size(),
and H5FD_mpi_get_comm() to use the new ctl() callback. In passing
removed the const qualifier from the file parameter of these
functions, as the file parameter of the ctl callback is not
const.
Deleted the old H5FD__mpio_mpi_rank(), H5FD__mpio_mpi_size(), and
H5FD__mpio_communicator() calls from the MPIO VFD.
Deleted H5FD_class_mpi_t from H5FDprivate.h, and modified the
MPIO VFD accordingly. Note that all VFDs now use H5FD_class_t,
with no special class for VFDs that that support MPI.
Some minor touch ups to the Neil's selection I/O mods in passing.
Tested serial and parallel, debug and production on charis and
jelly.
* Reserve a range of VFD "ctl" opcodes for library and experimental usage
* Add "ctl" callbacks to passthrough VFDs
* Add RELEASE.txt entry for "ctl" callback
* Use H5FDopen with H5F_ACC_RDWR flag instead of H5F_ACC_TRUNC in vfd test
* Remove handling of passthrough "ctl" flag from multi VFD
* Move logic for testing H5FD_CTL__TEST_OPCODE into a testing VFD
Revise description of "ctl" callback in RELEASE.txt
Remove unused H5FD_CTL__NUM_OPCODES definition
Fix some warnings in multi VFD
Co-authored-by: mainzer <mainzer#hdfgroup.org>
* H5Ztrans: (feature) Improved H5Z_xform_noop() and H5Z_xform_create() function
- Made a small improvement for H5Z_xform_noop() function. Now,
the function will also return TRUE if the data transform function
expression = "x". For this case, the HDF5 library behaves in a
similar fashion as the case when no data transform function has been
specified.
- Improved the inline documentation of the function
H5Z_xform_create() such it is more inline with the rest of the
code.
* Committing clang-format changes
* H5Ztrans: (feature) Added 3 tests for improved H5Z_xform_noop() function
- Added serial test with data transform expression = "x" to
verify the improved H5Z_xform_noop() function behaves as expected.
- Added 2 parallel tests with data transform expression = "x"
in combination with a filter. Before, these tests will fail but
with the improved H5Z_xform_noop() function they work and result
in the expected behavior.
- Small bug fix for one of parallel filter tests.
* Committing clang-format changes
* H5Ztrans: (feature) Added release note about detection of the
simple data transform function "x".
- Added a brief explanation about the implemented improvement
of the detection of the simple data transform function "x"
to the RELEASE.txt file.
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* add ULL attr set/get signatures to LT public API
* add LT implementation of ULL set/get attribute
* fix copy-paste typo in test comment
* add ULL test data
* define ullong attr name global variable
* add LT tests for ULL attr set/get
* Committing clang-format changes
* correct type passed to lt get attr test for ULL
* update RELEASE.txt with added ULL set/get functionality
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fixes a bad memory read and unfreed memory in fsinfo code
The segfaul from CVE-2020-10810 was fixed some time ago, but the
illegal memory read and unfreed memory were not.
This fix tracks some buffer sizes and errors out gracefully on errors,
ensuring buffers are cleaned up and avoiding the H5FL infinite loop +
abort on library close.
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Changes from PR#3
* HDFFV-11213 added option to control gcc10 warnings diagnostics
* HDFFV-11212 Use the new references correctly in JNI utility and tests
* format source
* Fix typo
* Add new test file
* HDFFV-11212 - update test and remove unused arg
* Minor non-space formatting changes
* Use H5I_INVALID_ID instead of "-1"
* source formatting
* add missing testfile, update jni function
* Undo commit of debug code
* remove mislocated file
* Fix h5repack test for handling of fapls and id close
* Update h5diff test files usage text
* HDFFV-11212 add new ref tests for JNI export dataset
* src format update
* Remove blank line typo
* src format typo
* long double requires %Lg
* Another long double foramt specifer S.B. %Lg
* issue with t128bit test
* Windows issue with h5dump and type.
* Fix review issues
* refactor function nesting and fix error checks
* format fixes
* Remove untested functions and javadoc quiet comments
* Restore TRY block.
* Change string append errors to memory exception
* revert to H5_JNI_FATAL_ERROR - support functions need work
* Add assertion error for h5util functions
* remove duplicate function
* format fix
* Revert HD function error handling
* Update copyright comments
* GH #386 java folder copyright corrections
* Whitespace
* GH #359 implement and fix tools 1.6 API usage
* remove excessive comments
* Flip inits to correct ifdef section
* rework ifdef to be simpler
* format issue
* Reformat ifdef inits
* remove static attribute
* format compliance
* Update names
* Revert because logic relies on float not being int
* Changes noticed from creating merge of #412
* Double underscore change
* Correct compiler version variable used
* Remove header guard underscores
* Whitespace cleanup
* Split format source and commit changes on repo push
* remove pre-split setting
* Change windows TS to use older VS.
* correct window os name
* HDFFV-11212 JNI export util and Javadoc
* Suggested review changes
* Another change found
* Committing clang-format changes
* HDFFV-11113 long double in tools
* HDFFV-11113 add note
* Disable long double tests for now
* HDFFV-11228 remove arbitrary CMake warning groups.
* Make each flag a string
* Some Javadoc warning fixes
* Updated javadoc fixes
* # WARNING: head commit changed in the meantime
HDFFV-11229 Fix long double usage in tools and java
Changed h5dump and h5ls to just print 128-bit for long double type.
Added test and file for dataset and attributes with all three float
types.
* Committing clang-format changes
* HDFFV-11229 Add release note
* HDFFV-11229 windows testfile needed
* fix typo
* Remove non supported message text
* HDFFV-11229 - change ldouble test to check both native and general
* HDFFV-11229 add second file compare
* HDFFV-11229 fix reference file
* HDFFV-11229 autotools check two refs
* HDFFV-11229 revert back to removal of NATIVE_LDOUBLE in tools output
* Committing clang-format changes
* Update release note
* Update attribute type of ref file
* Change source of ninja for macs
* try port instead of brew
* Recommended is to use brew.
* Undo non long double changes
* remove unneeded file
* Fix perl and doxygen CMake code
* Add "option" command for clang options
* Rework CMake add_custom to use the BYPRODUCTS argument
* Add stamp files to BYPRODUCTS
* Only one copy of file exists
* Fix custom cmmand depends targets
* Fix fortran custom command DEPENDS
* Add LD_LIBRARY_PATH to tests
* Add custom target for DEPENDS
* Add h5detect conditionaly to generated target DEPENDS
* Correct DEPENDS targets
* Parallel builds need the mpi compiler for pkgconfig scripts.
* install only if MPI build
* Fortran target depends
* Remove incorrect source attribute
* doxygen adjustments
* doxygen build updates
* Correct version
* Correct function version - function has been merged to 1.12
* Correct version string for map functions
* Cleanup warnings for tools debug builds
* TRILAB-227 - fix indexing for h5diff selections
* Correct location of pos to index function call
* TRILAB-227 Enable test
* Quote subset args
* Use MATCHES because of AppleClang
* if blocks needed for build modes
* Update list of DT platforms
* VS2019 correctly displays float values
* revert VS2019 change
* Issue #669 remove version from pkgcfg filename
* remove version from h5cc script
* Java reference functions updated
enabled fortran in cmake gcc action yaml file
java reference test changed to correctly test refs
jni reference functions that create ids changed to use jni id logging
* Correct BYPRODUCTS argument
* Correct more genereated files BYPRODUCTS
* BYPRODUCTS must have unique locations
* Fix typo
* Fix fortran configure checks
* Rework H5_PAC_C_MAX_REAL_PRECISION setting logic
* Add note about fortran configure change
* Adds a quick for for some egregious chunk_info badness (#722)
* Fixes issue with ccmake that prevents building Fortran (#723)
ccmake runs iteratively, and the check_fortran_source_runs
macros were clobbering a single output file that did not
get updated on further configure iterations
* Fix conflicts with merge
* Move MAX_PRECISION back to HDF5UseFortran.cmake
* Use STREQUAL to test macro argument
* Move C language test to ConfigureChecks from HDF5UseFortran
* MAX_PRECISION defines must be defined
* Organize flags and align autotools and cmake
* Fix comment in no-error-general warnings files.
* Flag cleanup and fix typos
* Add comment
* Correct VAR used to find configure time file
* Set the path correctly
* Update missing release note info.
* Update code owners
* Correct JIRA note
* add known problem.
* Use only core library for testing dynamic plugins.
* Reorg tools perform to provide h5perf for installation
* Correct file paths
* Correct path
* Add new src folder to makefile list
* Remove bin_PROGRAMS from TEST_PROG
* Default h5perf executables to static build
* Remove test lib dependency from h5perf
* format adjustments
* Remove test lib from autotools makefile
* Add note
* h5perf needs the math library
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Some of the command-line tools would allow partial long options (e.g.,
--datas instead of --dataset). These were inconsistently implemented,
difficult to maintain, and occasionally blocked useful long options.
They have been removed from all the tools.
NOTE: This change should NOT be merged to 1.12 or earlier.
* Remove version-check for relnum in H5check.
* Add in release exception code check
* Fix typos
* Fix more typos
* Rework comments
* format change
* format whitespace
* Library version must be less than or equal to headers
* Need the NOT version of the compare
* Enable release+1 check for PASS
* Add release note
* Update note
