* When evicting driver info block, NULL the corresponding entry
Since H5C_expunge_entry() called (from H5AC_expunge_entry()) sets the flag
H5C__FLUSH_INVALIDATE_FLAG, the driver info block will be freed. NULLing the
pointer in f->shared->drvinfo will prevent use-after-free when it is used in other
functions (like H5F__dest()) - as other places will check whether the pointer is
initialized before using its value.
This fixes CVE-2021-46242 / Bug #2254
Signed-off-by: Egbert Eich <eich@suse.com>
* When evicting the superblock, NULL the corresponding entry
The call to H5AC_expunge_entry() will free the corresonding structure,
to avoid a use-after-free, the corrsponding pointer entry will be NULLed.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.com>
For Data Layout Messages version 1 & 2 the specification state
that the value stored in the data field is 1 greater than the
number of dimensions in the dataspace. For version 3 this is
not explicitly stated but the implementation suggests it to be
the case.
Thus the set value needs to be at least 2. For dimensionality
< 2 an out-of-bounds access occurs as in CVE-2021-45833.
This fixes CVE-2021-45833 / Bug #2240.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.com>
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Commit afc54d75a1 to the test suite to eliminate a separate
shell script did not take into account that there may be
multiple dependent test scripts which resulted in the
message:
`/bin/sh: line 7: [: too many arguments`.
when running the test suite.
Beware that this new version still makes use of non-posix GNU
extensions to `test`.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.com>
* Use ubuntu-20.04 for github autotools parallel tests until parallel
configure is fixed on ubuntu 22.04.
* Remove inactive developers from CODEOWNERS.
* Temporarily cat config.log for autotools build in github actions.
* Try ubuntu-latest.
* Attempt to cat config.log even when configure fails.
* Fix syntax.
* Try again
* Try again
* Try again.
* Back to original temporary fix.
* Install libopenmpi-dev with openmpi-bin, which was installed as a
dependency prior to ubuntu-22.04.
* Jira HDFFV-10881: Segfault in H5CX_get_vol_wrap_ctx when H5VLwrap_register is called from an application. A quick and simple fix to make it fail with a relevant error message.
* Format changes.
* Committing clang-format changes
* Minor change: split one condition check into two for clarity.
* Adding doxygen comment for H5VLwrap_register.
* Minor change: adding a little more detail to the Doxygen comment for H5VLwrap_register.
Return the result rather than setting the exit code. "return" is a
language keyword whereas "exit" is a function for which the <stdlib.h>
header has to be included which it wasn't in this test, therefore the
test would previously fail to identify that the encoder was enabled if
"-Werror=implicit-function-declaration" was used, which it is by default
with clang from Xcode 12 and later.
Fixes#2262
The size of the enumeration values is determined by the size of the parent.
Functions accessing the enumeration values use the size of the enumeration
to determine the size of each element and how much data to copy. Thus the
size of the enumeration and its parent need to match.
Check here to avoid unpleasant surprises later.
This fixes CVE-2018-14031 / Bug #2236.
Signed-off-by: Egbert Eich <eich@suse.com>
According to the specification, the information block for external links
contains 1 byte of version/flag information and two 0 terminated strings
for the object linked to and the full path.
Although not very useful, the minimum string length for each (with
terminating 0) would be one byte.
Checking this will help to avoid SEGVs triggered by bogus files.
This fixes CVE-2018-16438 / Bug #2233.
Signed-off-by: Egbert Eich <eich@suse.com>
Initially, the accumulated metadata location is initialized to HADDR_UNDEF
- the highest available address. Bogus input files may provide a location
or size matching this value. Comparing this address against such bogus
values may provide false positives. This make sure, the value has been
initilized or fail the comparison early and let other parts of the
code deal with the bogus address/size.
Note: To avoid unnecessary checks, we have assumed that if the 'dirty'
member in the same structure is true the location is valid.
This fixes CVE-2018-13867 / Bug #2230.
Signed-off-by: Egbert Eich <eich@suse.com>
When decoding a file space info message in H5O__fsinfo_decode() make
sure each element to be decoded is still within the message. Malformed
hdf5 files may have trunkated content which does not match the
expected size. Checking this will prevent attempting to decode
unrelated data and heap overflows. So far, only free space manager
address data was checked before decoding.
This fixes CVE-2021-45830 / Bug #2228.
Signed-off-by: Egbert Eich <eich@suse.com>
Additions
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Malformed hdf5 files may provide more dimensions than the array dim[] in
H5IMget_image_info() is able to hold. Check number of elements first by calling
H5Sget_simple_extent_dims() with NULL for both 'dims' and 'maxdims' arguments.
This will cause the function to return only the number of dimensions.
The fix addresse a stack overflow on write.
This fixes CVE-2018-17439 / HDFFV-10589 / Bug #2226.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.com>
* Removes MPE instrumentation support.
The Autotools will no longer accept --with-mpe= and the logging commands
have been removed from the FUNC_ENTER macros. CMake has never supported
instrumenting for MPE.
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Malformed hdf5 files may have trunkated content which does not match
the expected size. When this function attempts to decode these it may
read past the end of the allocated space leading to heap overflows
as bounds checking is incomplete.
Make sure each element is within bounds before reading.
This fixes CVE-2019-8396 / HDFFV-10712 / github bug #2209.
Signed-off-by: Egbert Eich <eich@suse.com>
Signed-off-by: Egbert Eich <eich@suse.com>
This has not been used to debug the library in a very long time. Most
developers use valgrind, -fsanitize=address, or some other memory checker
instead of this library.
This removes:
* dmalloc.h include from H5private.h
* --with-dmalloc= Autotools configure option
* HDF5_ENABLE_USING_DMALLOC CMake option
* Bug fix: make sure the test case supports the pass_through connector.
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Change how HPC tests are executed
* Move ctest_test commands for serial and parallel tests into cmake files.
* Update checks for uploading Test.xml to CDash.
* Correct cmake scripots.
* Remove extraneous characters.
* Add ctest_start (APPEND).
* Add binary directories to scripts running ctest_test.
* Correct parameters for appending Test.xml.
* Set CTEST_SITE and CTEST_BUILD_NAME.
* Try to pass site and build_name to ctest_test sripts.
* Add site name and build name for ctest_test commands vi environment.
* Revert "Try to pass site and build_name to ctest_test sripts."
This reverts commit d364aaf0fa.
* Correct typo in src/CMakeLists.txt.
* Update batch scripts to run ctest_test scripts.
Co-authored-by: Allen Byrne <byrn@hdfgroup.org>
* Update link for Software Changes in RELEASE.txt.
Add RELEASE.txt contents from 1.13.3 release to HISTORY-1_13.txt.
* Reverse order of entries in HISTORY-1_13.txt to match the order in previous HISTORY
files.
* Updated source file copyright headers to remove "Copyright by the Board of Trustees
of the University of Illinois", which is kept in the top-level COPYING file.
* Added a test case for H5VLget_connector_name.
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Some Javadoc warning fixes
* Updated javadoc fixes
* # WARNING: head commit changed in the meantime
HDFFV-11229 Fix long double usage in tools and java
Changed h5dump and h5ls to just print 128-bit for long double type.
Added test and file for dataset and attributes with all three float
types.
* Committing clang-format changes
* HDFFV-11229 Add release note
* HDFFV-11229 windows testfile needed
* fix typo
* Remove non supported message text
* HDFFV-11229 - change ldouble test to check both native and general
* HDFFV-11229 add second file compare
* HDFFV-11229 fix reference file
* HDFFV-11229 autotools check two refs
* HDFFV-11229 revert back to removal of NATIVE_LDOUBLE in tools output
* Committing clang-format changes
* Update release note
* Update attribute type of ref file
* Change source of ninja for macs
* try port instead of brew
* Recommended is to use brew.
* Undo non long double changes
* remove unneeded file
* Fix perl and doxygen CMake code
* Add "option" command for clang options
* Rework CMake add_custom to use the BYPRODUCTS argument
* Add stamp files to BYPRODUCTS
* Only one copy of file exists
* Fix custom cmmand depends targets
* Fix fortran custom command DEPENDS
* Add LD_LIBRARY_PATH to tests
* Add custom target for DEPENDS
* Add h5detect conditionaly to generated target DEPENDS
* Correct DEPENDS targets
* Parallel builds need the mpi compiler for pkgconfig scripts.
* install only if MPI build
* Fortran target depends
* Remove incorrect source attribute
* doxygen adjustments
* doxygen build updates
* Correct version
* Correct function version - function has been merged to 1.12
* Correct version string for map functions
* Cleanup warnings for tools debug builds
* TRILAB-227 - fix indexing for h5diff selections
* Correct location of pos to index function call
* TRILAB-227 Enable test
* Quote subset args
* Use MATCHES because of AppleClang
* if blocks needed for build modes
* Update list of DT platforms
* VS2019 correctly displays float values
* revert VS2019 change
* Issue #669 remove version from pkgcfg filename
* remove version from h5cc script
* Java reference functions updated
enabled fortran in cmake gcc action yaml file
java reference test changed to correctly test refs
jni reference functions that create ids changed to use jni id logging
* Correct BYPRODUCTS argument
* Correct more genereated files BYPRODUCTS
* BYPRODUCTS must have unique locations
* Fix typo
* Fix fortran configure checks
* Rework H5_PAC_C_MAX_REAL_PRECISION setting logic
* Add note about fortran configure change
* Adds a quick for for some egregious chunk_info badness (#722)
* Fixes issue with ccmake that prevents building Fortran (#723)
ccmake runs iteratively, and the check_fortran_source_runs
macros were clobbering a single output file that did not
get updated on further configure iterations
* Fix conflicts with merge
* Move MAX_PRECISION back to HDF5UseFortran.cmake
* Use STREQUAL to test macro argument
* Move C language test to ConfigureChecks from HDF5UseFortran
* MAX_PRECISION defines must be defined
* Organize flags and align autotools and cmake
* Fix comment in no-error-general warnings files.
* Flag cleanup and fix typos
* Add comment
* Correct VAR used to find configure time file
* Set the path correctly
* Update missing release note info.
* Update code owners
* Correct JIRA note
* add known problem.
* Use only core library for testing dynamic plugins.
* Committing clang-format changes
* Update main.yml
* Update HISTORY-1_13.txt
Missing release note added for changed location of CMake configuration files.
* Update HISTORY-1_13.txt
Fix typo
* Update main.yml
revert change
* Updated version to 1.13.4-1 after creating hdf5_1_13_3 branch for 1.13.3
release.
Added new version to exceptions in lines 74-5 of src/H5.c
* Cleared RELEASE.txt entries following HDF5 1.13.3 release.
Co-authored-by: Allen Byrne <byrn@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Allen Byrne <50328838+byrnHDF@users.noreply.github.com>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
* Removed the: file, func, line args in the _async APIs from the doxygen arguments. Documented H5ESclean.
* format fixes
* fixed DOXYGEN_PREDEFINED
* Fixed the #ifdet logic for doxygen, added missing multi-dataset va_args
* format fixes
* Use H5_DOXYGEN instead of H5_DOXYGEN_FORTRAN
* reordered multidataset declarations
* alt. for H5Aclose_async and H5Acreate_async
* made doxgyen API into seperate blocks
* updated codespell
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Added a test case to verify the VOL capacity flags work correctly.
* Committing clang-format changes
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Fix bug with cross platform compatibility of references within vlens.
No testing yet.
* Merge from multi_rd_wd_coll_io to a more recent branch from develop.
Untested, probably does not work yet.
* Committing clang-format changes
* Committing clang-format changes
* Fix many bugs in multi dataset branch. Mostly works, some issues in
SWMR tests.
* Committing clang-format changes
* Disable test in swmr.c that was failing due to bug in HDF5 unrelated to
multi dataset.
* Committing clang-format changes
* Fixed fortran multi-dataset tests
* Fixed xlf errors
* Added benchmark code for multi-datasets
* loops over datasets
* added missing error arg.
* Added gnuplot formatting
* Jonathan Kim original MD benchmarking code
* updated MD benchmarking code
* code clean-up
* Only make files in feature test mode
* misc clean-up
* removed TEST_MDSET_NO_LAST_DSET_2ND_PROC option
* Committing clang-format changes
* Change multi dataset API to use arrays of individual parameters instead
of the parameter struct.
* Committing clang-format changes
* Update to new multi dataset Fortran API and tests. (#1724)
* Update to new multi dataset Fortran API and tests.
* Sync Fortran with develop.
* skipping h5pget_mpio_actual_io_mode_f for now
* Fixed issue with dxpl_id, changed to variable size dim. (#1770)
* Remove "is_coll_broken" field from H5D_io_info_t struct
* Committing clang-format changes
* Minor cleanup in multi dataset code.
* Committing clang-format changes
* Clean up in multi dataset code.
* Committing clang-format changes
* Committing clang-format changes
* Fix speeling
* Fix bug in parallel compression. Switch base_maddr in io_info to be a
union.
* Committing clang-format changes
* Implement selection I/O support with multi dataset. Will be broken in
parallel until PR 1803 is merged to develop then the MDS branch.
* Committing clang-format changes
* Spelling
* Fix bug in multi dataset that could cause errors when only some of the
datasets in the multi dataset I/O used type conversion.
* Committing clang-format changes
* Integrate multi dataset APIs with VOL layer. Add async versions of
multi dataset APIs.
* Committing clang-format changes
* Spelling fixes
* Fix bug in non-parallel HDF5 compilation.
* Committing clang-format changes
* Fix potential memory/free list error. Minor performance fix. Other minor
changes.
* Committing clang-format changes
* Fix memory leak with memory dataspace for I/O.
* Committing clang-format changes
* Fix stack variables too large. Rename H5D_dset_info_t to
H5D_dset_io_info_t.
* Committing clang-format changes
* Remove mem_space_alloc field from H5D_dset_io_info_t. Each function is
now responsible for freeing any spaces it adds to dset_info.
* Committing clang-format changes
* fixed _multi Fortran declaration
* Refactor various things in (mostly) the serial I/O code path to make
things more maintainable.
* Committing clang-format changes
* updated to array based, doxygen, and examples
* Reinstate H5D_chunk_map_t, stored (via pointer) inside
H5D_dset_io_info_t.
* Change from calloc to malloc for H5D_dset_io_info_t and H5D_chunk_map_t.
Switch temporary dset_infos to be local stack variables.
* Committing clang-format changes
* format cleanup
* format cleanup
* added coll and ind
* Modify all parallel I/O paths to take dset_info instead of assuming
dset_info[0].
* Committing clang-format changes
* fixed output
* Rework parallel I/O code to work properly with multi dataset in more
cases. Fix bug in parallel compression.
* Committing clang-format changes
* Prevent H5D__multi_chunk_collective_io() from messing up collective opt
property for other datasets in I/O. Other minor cleanup. Add new test
case to t_pmulti_dset.c for H5FD_MPIO_INDIVIDUAL_IO, disabled for now
due to failures apparently unrelated to multi dataset code.
* Fix spelling
* Committing clang-format changes
* Replace N log N algorithm for finding chunk in
H5D__multi_chunk_collective_io() with O(N) algorithm, and remove use of
io_info->sel_pieces in that function.
* Committing clang-format changes
* Replace sel_pieces skiplist in io_info with flat array of pointers, use
qsort in I/O routine only when necessary.
* Committing clang-format changes
* Add new test case to mdset.c
* Committing clang-format changes
* Fix spelling
* Very minor fix in H5VL__native_dataset_read()
* Fix bug that could affect filtered parallel multi-dataset I/O.
* Add RM entries for H5Dread_multi(), H5Dread_multi_async(),
H5Dwrite_multi(), and H5Dwrite_multi_async()
* Unskip test in swmr.c
* Committing clang-format changes
* Eliminate H5D__pre_read and H5D__pre_write
* Remove examples/ph5mdsettest.c. Will fix and re-add as a test.
* Enable hyperslab combinations in mdset test
* Committing clang-format changes
* Clarify H5Dread/write_multi documentation.
* Fix bugs in multi-dataset I/O. Expand serial multi dataset test.
Update macro in parallel multi dataset test.
* Committing clang-format changes
* Spelling
* Remove obsolete entry in bin/trace
* Rework type conversion buffer allocation. Only one buffer is shared
between datasets in mdset mode, and it is malloced instead of calloced.
* Committing clang-format changes
* Fix bug in error handling in H5D__read/write
* added multi-dataset fortran check with optional dataset creation id (#2150)
* removed dup. dll entry
* Address comments from code review.
* Remove spurious changes in H5Fmpi.c
* Fix issue with reading unallocated datasets in multi-dataset mode.
Address other comments from code review.
* Committing clang-format changes
* Delay chunk index lookup from io_init to mdio_init so it doesn't add
overhead to single dataset I/O.
* Committing clang-format changes
* Fix inappropriate use of piece_count
* updated copyright on new file, removed benchmark from testing dir.
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: M. Scot Breitenfeld <brtnfld@hdfgroup.org>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
