Changes made so that raw data for dataset objects are copied from cached info when possible instead of flushing objects to file and read them back in again.
* commit '57f0a296c0b727e3d86cea312ba46efc98d76227':
Minor warning fix.
More minor warning fixes.
Minor warning fixes in the tools.
Minor type fixes to h5watch.
A few more warnings fixed.
Fixed additional minor warnings.
Fixed many minor warnings.
* Changed return type of H5FD_driver_query() to herr_t.
* Updated swmr.c to use H5FDdriver_query() instead of the private call.
* Added some flags tests to vfd.c.
* Moved H5FD_driver_query() to H5FDint.c.
* Minor typo fixes in H5I.c.
* Updated swmr.c to check the VFD feature flags and skip the
retries test when the VFD is not compatible with H5P_DEFAULT.
* Updated comments in the library to accurately reflect the semantics
of the H5FD_FEAT_POSIX_COMPAT_HANDLE flag.
* Removed the check for the POSIX compatible handle feature flag
when determining if a VFD supports SWMR.
Partial work for HDFFV-10197:
* Added a new H5FD_FEAT_DEFAULT_VFD_COMPATIBLE feature flag that is
set when a VFD creates output that is compatible with the default
VFD. Will be used in the testing but might also be generally useful
to users.
This is the similar set of changes that was checked in to 1.8 branch:
Incorporate the code changes that were missing from the original patch:
(1) Additional tests in hl/test/test_dset_opt.c
(2) Fix in src/H5Dchunk.c for direct access when an entry is in chunk cache but not dirty
Tested on platypus, ostrich, kituo, mayll, osx1010test, moohan, quail, emu.
* commit 'd436db6c6efdfe7e9dbe06a642715116ef0eb5e3':
Fix for the two issues reported in HDFFV-10051 Modifications made based on the review comments from pull request #494 Tested on moohan, mayll, kituo, platypus, ostrich, osx1010test, quail, emu.
Fix for the two issues reported in HDFFV-10051: (1) Repeated open/close of a compact dataset fails due to the increment of ndims in the dataset structure for every open. --This is done only for chunked dataset via H5D__chunk_set_sizes(). (2) layout "dirty" flag for a compact dataset is not reset properly after flushing the data at dataset close. --Reset the "dirty" flag before flushing the message to the object header via H5O_msg_write(). Tested on moohan, kituo, platypus, ostrich, osx1010test, quail, emu.
(1) Repeated open/close of a compact dataset fails due to the
increment of ndims in the dataset structure for every open.
--This is done only for chunked dataset via H5D__chunk_set_sizes().
(2) layout "dirty" flag for a compact dataset is not reset
properly after flushing the data at dataset close.
--Reset the "dirty" flag before flushing the message to the object header
via H5O_msg_write().
Tested on moohan, kituo, platypus, ostrich, osx1010test, quail, emu.
This is the same patch that had already been integrated to 1.8 branch (pull request #387).
Tested on moohan, ostrich, platypus, emu, osx1010test, quail, kituo, mayll.
* commit '54957d37f5aa73912763dbb6e308555e863c43f4':
Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes.
Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh.
Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1
Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
* commit 'a995a63085d88e1a633770536efeff5b01569445':
HDFFV-9655 fix test and some warnings
HDFFV-9655 fix compile and test
HDFFV-9655 - Add functions for '@' - untested
* commit '906c1819188755a04fde6cc0ad7346775a6e61b8':
Skip tests for page buffering in test/fheap.c for parallel build Skip the two tests for page buffering in test/fheap.c because this feature is disabled in parallel. Activate full testing when this feature is re-enabled in the future for parallel build.
Due to HDFVIEW-19 add read VL as string function
Fix URL name
updated with README changes
DAILYTEST-250 change test props to reduce timeout
Combined the README_DEVEL and README, and removed out-dated README material
Fix for H5Dset_extent test failure with extensive array indexing (HDFFV-9771) 1) Calculate chunk index for extensive array index based on swizzled max chunks when unlim_dim > 0 2) Minor fixes to test/fheap.c that somehow were missed from last check in. See pull request #396 review comments.
Modified test/cache.c to skip the smoke checks with paged aggregation
for all express test levels other than 0
Change to src/H5PL.c seems to be the result of autogen.sh. As it
seems to be correct, I am checking it in.
Tested serial and parallel (debug only) on jelly.
1) Calculate chunk index for extensive array index based on swizzled max chunks when unlim_dim > 0
2) Minor fixes to test/fheap.c that somehow were missed from last check in. See pull request #396 review comments.
* commit '7883c0d01b4eadba84697c7bb3322a304d242ed6': (38 commits)
Add comments
In current implementation, list must be gathered to all
Correct error with usage of array_gather function
Make array_gather routine more general
Updated comments
Add comments to H5D_filtered_collective_io_info_t data structure
Remove debugging statements
Revert "Merge latest changes from develop"
Merge latest changes from develop
removed erroneous casts from non-rvalue H5MM_free() calls. Replaced assignment of void return values with explicit value of NULL
Fix issue with uninitialized chunk index during chunk re-insertion into index
Remove potential infinite loops caused by goto in error label region
Remove note about src_type_size vs dst_type_size
Trigger error if library tries to break to independent I/O with filters in the pipeline
Fix issue in H5D__mpio_array_gather when no one contributes anything
Iterator for initial chunk entry IO operation should use source datatype size
Fix bug where data doesn't write correctly when MEMSPACE isnt the same as FILESPACE
Check return value of H5S_GET_EXTENT_NPOINTS macro
Fix macro complaint in H5Dmpio.c
Test code for user passing one-chunk-per-process hint to library
...
(https://jira.hdfgroup.org/browse/HDFFV-10055).
Briefly, in H5C_collective_write() in H5Cmpio.c,
the metadata cache attempts to perform a collective
write of metadata cache entries.
This worked fine as long as all processes had at
least one entry to write.
However, when the process has no entries, the
function tries to participate in the collective write
by calling MPI_File_set_view(),
MPI_File_write_all() and then MPI_File_set_view()
again, to match the calls in H5FD_mpio_write().
After pull request 183, the CGNS test benchmark_hdf5
started failing. On investigation, I determined that
the failure occurred in the first call to MPI_File_set_view()
in the "no data to write" path through H5C_collective_write().
Note that pull request 183 did not create the problem,
it only exposed it. The bug can be observed after pull
request 182 if one executes the CGNS progam
src/ptests/benchmark_hdf5 with 90 processes.
The problem appears to have been that the calls to
MPI_File_set_view() in H5C_collective_write() and
H5FD_mpio_write() were using different values for the
info parameter. I patched the problem by adding a
MPI specific VFD call allowing me to get the MPI_Info
used in H5FD_mpio_write() for use in
MPI_File_set_view() calls in H5C_collective_write().
Tested serial & parallel, debug & production on
Jelly.
* commit 'b56fb149c9a3c9dca11b406b7a2488f0c93ee187':
Updated the H5L.c error message after additional thought. Fix for HDFFV-10141.
Updated an error message in H5L.c to be more helpful. Fixes HDFFV-10141.
* commit '2412158ed8326a3f3d62fbd947e470667d0b5951':
Add new file COPYING_LBNL_HDF5.
Revert "Clear hdf5 1.10 entries from RELEASE.txt in the develop branch. Entries"
Add LBNL license file and modify COPYING file accordingly.
Omnibus checkin for several relatively minor modifications:
Clear hdf5 1.10 entries from RELEASE.txt in the develop branch. Entries in this branch version of RELEASE.txt should be intended for the future 1.12.0 release.
Fix HDFFV-8089 Description: Some code within an "ifdef H5D_CHUNK_DEBUG" block was using outdated data structure but not caught because the case of H5D_CHUNK_DEBUG being defined was never tested. It was commented out. I defined H5D_CHUNK_DEBUG, tested, and commented out again. Platforms tested: Linux/32 2.6 (jam) Linux/64 (platypus) Darwin (osx1010test)
Fix HDFFV-8089
* commit '52f8c2ed494ea1b89374981ecc6901abe8fd5fed':
Fix HDFFV-8089 Description: Some code within an "ifdef H5D_CHUNK_DEBUG" block was using outdated data structure but not caught because the case of H5D_CHUNK_DEBUG being defined was never tested. It was commented out. I defined H5D_CHUNK_DEBUG, tested, and commented out again. Platforms tested: Linux/32 2.6 (jam) Linux/64 (platypus) Darwin (osx1010test)
1) Added code test/page_buffer.c to verify that page buffering is
disabled in parallel builds.
2) Added code to test/cache_image.c to verify correct interaction
between evict on close and cache image -- in particular management
of a file containing a cache image containing dirty metadata that
has been opened R/O. Also fix for the bug exposed.
3) Added code to testpar/t_cache_image.c to verify expected procedure
for reading cache images, and also supporting stats collection code
needed for the test.
4) Repair of an overactive sanity check in H5C__reconstruct_cache_contents().
5) Other minor tidies in passing.
Tested serial and parallel, debug and production on Jelly.
Description:
Some code within an "ifdef H5D_CHUNK_DEBUG" block was using outdated
data structure but not caught because the case of H5D_CHUNK_DEBUG being
defined was never tested. It was commented out.
I defined H5D_CHUNK_DEBUG, tested, and commented out again.
Platforms tested:
Linux/32 2.6 (jam)
Linux/64 (platypus)
Darwin (osx1010test)
* commit 'b7c58f7dfca66d4caa26d7f8b318ad3514b8c46f':
Minor code changes to address comments in pull request
Checkin of additions to cache image parallel test code and associated bug fixes.
* commit 'b359e8f1a2349d177e70d34560d0a089ce37c4c8':
Description: Removed "#ifndef H5_NO_DEPRECATED_SYMBOLS" in file space tests, because the wrappers only use the latest functions now. Platforms tested: Linux/32 2.6 (jam) Linux/64 (platypus) Darwin (osx1010test)
Purpose: Add new C++ wrappers Description: Because H5Pset_file_space and H5Pget_file_space are deprecated, changed to make wrappers for the new functions instead: H5Ps/get_file_space_strategy H5Ps/get_file_space_page_size
Description: Fixed typos. Platforms tested: Linux/64 (jelly)
Description: Deprecating versions of PropList::setProperty that have arguments that miss "const" Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1010test)
Description: Miscellaneous clean-up: format and comments Platforms tested: Linux/64 (jelly) Darwin (osx1010test)
Add toolset option to cmake configure
Update cmake pubconf to match autotools and add strtoll checks
Switch from HDatoll to HDstrtoll, for Windows compatibility.
output_filter.sh: Comment added to address HDFFV-8270. The sample ontput in the file's comments are not up-to-date with the scripts in the file that remove output unique to certain systems when running test scripts. This output doesn't match expected output files for the tests, causing them to fail. Ther output_filter.sh file removes such output. Currently we don't have access to these systems to update the comments.
Correct double sourcedir name on copied datafiles.
HDFFV-10138 Merge app framework to examples
Final merge of page buffering branch to develop
Purpose: Add new C++ wrappers Description: Added wrappers for H5Iis_valid, H5Ps/get_nlinks, H5Tget_create_plist, H5Oopen, H5Oclose and H5Pset_virtual
Bring changes to I/O parameters from page_buffering branch.
Merge in reentrency changes to "make space in cache" from page_buffering branch.
Minor cleanups and bring over "prefetched dirty" fixes for entries loaded from a cache image.
Remove some usage of "prefetched_dirty" flag (which hasn't been merged from the page_buffering branch yet. Also, bring over improvements to flush candidate entries for parallel code.
Align with incoming page buffering changes: minor cleanups, centralize removing entries from collective metadata read list
associated bug fixes.
Also, modifications to H5PB_dest() to flush the page buffer before
destroying the page buffer. This is necessary, as when persistant
free space managers are enabled, the page buffer will typically contain
dirty FSM data at page buffer destroy time.
Tested serial/debug, serial/production, serial/check-vfd/debug,
parallel/debug, parallel/production on Jelly.
* commit '56e5b4ed05fb3bad51c226ec5b567871a45e0bce': (112 commits)
updated AM_SILENT_RULES syntax
Added a check for AM_SILENT_RULES before enabling it.
Fix alignment
Fix vds tests
Correct result code for strings test
Add extension to filename
HDFFV-10128 fix string compare and add tests
Description: Only format changes: mostly tabs vs. spaces Platforms tested: Linux/64 (jelly) - very minor
Lowercase math and set CMake commands
Added "flush" and "refresh" to testflushrefresh.sh error output so it's easier to see where errors in the script occur.
Misc. small cleanups to sync against incoming page buffering changes.
Update CMake formatting
Update CMake files and formatting
Normalize against incoming page buffering changes.
HDFFV-10120 verify nozlib checking with tools tests
cmake non-code typo
Correct issues from pull request reviews.
Corrected version of 4b5e05c084: Fix some more signed -> unsigned value issues with cache data structures, also misc. style cleanups. All to align w/incoming page_buffering changes.
Revert "Fix some more signed -> unsigned value issues with cache data structures, also"
Added virtual to list
...
Fix bug where a process with no selection during Multi-chunk IO could
cause other processes to copy back incorrect chunk information after
chunk re-allocation
