Fix bug where a process with no selection during Multi-chunk IO could
cause other processes to copy back incorrect chunk information after
chunk re-allocation
Modify single chunk entry function to handle both read and write cases
Store array of MPI derived types in Multi-chunk IO so that all freeing
can be done at end instead of during processing
Add read support for Multi-chunk IO only currently
Separate update phase for chunk entry into its own function, since code
for multi-chunk IO and link-chunk IO is exactly the same
Remove last IO mode code from multi-chunk IO, since filtered collective
writes cannot break to independent IO mode
Fix collective overlapping IO for multi-chunk IO by iterating equal to
the max number of chunks on any process
Make hard separation between collective read and write since trying to
mix the two into one loop becomes messy
Add preliminary code for asynchronous sending of chunk modification data
to new owning process when redistributing chunks
Merged naming check improvement in function enter macros.
* commit '441c8ea43ab6a6559d1f167bdfbbaf3720621804':
Added text to the function check macros so that a more helpful error message is emitted when there is a problem.
Final fix for evict-on-close issues with new-style groups.
* commit '02879bc4b2ba161dc259bf28a71e7a99b44be0e3':
Updated the H5FS cache code to grab the correct tag and modified the freespace test to use dxpls that have been tagged with the H5AC__FREESPACE_TAG global tag instead of H5AC_ind_read_dxpl_id. The library code now expects the owner of the free space manager to tag it so the owner-less free space managers in the freespace tag had to be tagged with *something* to avoid cache errors.
Changed the name of H5MF_alloc_close() to H5MF__alloc_close() to make the FUNC_ENTER macro happy.
Reverted the freespace test. The TAG macros in H5MF_get_freespace() make the changes unnecessary.
Added missing TAG macros to H5MF_get_freespace().
Updated the freespace test so functions take a common, tagged dxpl so the test will work with the new H5FS tag changes.
Updated the cache_tagging test so it correctly analyzes the new fractal heap free space manager behavior.
Added TAG macros to the H5MF package code so H5AC__FREESPACE_TAG gets set on the dxpl correctly. Also restored the TAG macro to H5FS__cache_hdr_pre_serialize() since otherwise the tag won't be initialized.
First cut at fixing the tagging issue with the free space managers. * Converted the tag macros to regular ones so the parent's tag is used via the dxpl.
the freespace test to use dxpls that have been tagged with
the H5AC__FREESPACE_TAG global tag instead of H5AC_ind_read_dxpl_id.
The library code now expects the owner of the free space manager
to tag it so the owner-less free space managers in the freespace
tag had to be tagged with *something* to avoid cache errors.
* Converted the tag macros to regular ones so the parent's tag
is used via the dxpl.
* Updated the tag sanity check logic so for free space managers.
* Turned new-style group checks back on in test/evict_on_close.c
Updated H5AC_dump_cache() to emit more information.
* commit '7a8c7c6063e328fdac28aa3f77ead4bdc91860e5':
Updated the cache dump to use the stored type name instead of the (redundant) array of names I previously created.
Updated the format of H5AC/C_dump_cache() to include more information and be easier to read.
Adds /*out*/ parameters to H5O_close() and H5F_try_close() so
that H5D/G_close() will know when H5O_close() has triggered a
file close and thus the file struct is not reliable.
Also removes the H5F_CLOSING() macro and related which were
formerly used to check if the file was closing.
Applied the evict-on-close "check for file closing" change that was successful for groups to datasets.
* commit '88b6f2e24cafcfaffe7d99ab6c5afc2290ce93b4':
Added the H5F_CLOSING() check that makes valgrind happy in the evict-on-close code in H5G_close() to H5D_close().
Two minor evict-on-close changes:
* Fix for a valgrind-reported issue where invoking EoC behavior when the file is closing leads to garbage file data being passed to downstream calls. The solution is to ignore EoC when the file is closing anyway.
* Added brackets so an infinite loop that arises when evicting tagged entries (due to pinned entries preventing progress) raises an error instead.
* commit '4e7b19f8260a7adb0e3b3df30e2d8289fccdcca6':
Two minor cache changes:
* Evict-on-close behavior is now skipped when the file is closing.
This fixes a potential issue discovered by Valgind on Windows
where potentially garbage data would be availble for manipulation.
* Added brackets to some code in the tagged entry evict iterators
so that the flag that determines if progress was made only gets
set if things are actually evicted. This prevents an infinite
loop (and emits an error) when pinned entries prevent eviction
and thus progress.
Changed the return type of H5Pget_driver_info() from void * to const void *
* commit '8eaa0af30585adbbab29686541cd33e058abd6e8':
Changed the return type of H5Pget_driver_info() from void * to const void *. Fixes HDFFV-10017.
* commit 'b2878dec04c120e4e3cdf00e943283e359862c84':
Move cache debugging routines into separate module.
Change file memory type for extending to default, and correct error in backward compatibility for multi VFD.
Warning and whitespace cleanup.
Improvements to the log VFD, for helping with SWMR debugging.
Minor code cleanups.
Uncomment line in test/objcopy.c, clean up whitespace and POSIX call wrapping in tools code.
Merged EoC for groups. Will expand the tests as discussed in the code review at a later date.
* commit '30f5f7d9f97e7e4e331894c3574cc1b6c4df6d9e':
Removed commented-out lines that clean up test files.
Added a test for old- and new-style groups under evict-on-close.
Evict-on-close now works with groups.
* commit '5a7880183025f56421cf6f2274d9f1ac36f59641':
Clean up hardcoded constants and check return values better. (Comments from group code review)
Description: Cleanups from Dana's review.
Description: Further warning cleanups: from 667 warnings to 503.
* commit '748d6f318c5ebe4fa7ee5d8e4ca2681d69792b70':
Change check for number of dimensions for old-style arrays in datatype decoding routine from an assertion to an if/HGOTO_ERROR check, since it is inappropriate to assert the contents of a file will be what we expect.
* commit '866d17f57f47414b6e778d63fcdcc2a82b208683':
Fix bug in "nooptype" decode in fix for TALOS-0177.
Fix issues in H5Znbit.c where the decompression algorithm would not check the compressed data for validity, potentially causing a buffer overflow.
More warning cleanups, bringing the build down to 25 unique types of
warnings, with 550 warnings in 122 files (down from 28, 770, and 134).
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial & parallel
(h5committest forthcoming)
with an alignment threshold set to be larger than the chunk size but smaller
than the size of the small data aggregator.
Tested: koala, ostrich (h5committest); jelly, ummon
More warning cleaups: down to 770 warnings (from ~940) in 134 files (from
148), with 28 unique kinds of warnings (from 31).
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial & parallel
(h5committest forthcoming)
Clean up more warnings: drop the warning count from ~1310 down to ~940,
with only 31 types of warnings in 148 files (down from 38 types in 167 files).
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial & parallel
(h5committest forthcoming)
More warning cleanups. This brings us down to ~1300 warnings in 167 files
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial & parallel
(h5committest forthcoming)
Use internal rounding routine, if the StdC ones aren't available (like on
Windows).
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial & parallel
(h5committest forthcoming)
Clean up warnings (from 2774 -> 1560, with my standard debug build)
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Move updated flush dependency code in metadata cache from revise_chunks
branch to trunk. Also many of the cleanups from r30111 in the revise_chunks
branch.
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
The features is controlled via H5Pset/get_evict_on_close()
and is currently enabled by default (it will be disabled
by default in the final implementation).
There is a bug in the code where the eviction of tagged
metadata fails due to some of the metadata being dirty,
resulting in error return values and test failures.
Bring object/dataset/group/named datatype features from revise_chunks
branch to trunk. Also CMake support for h5format_convert and a bunch of
misc. cleanups.
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Create iterator routine for tagged entries and refactor current routines
to use it.
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring a few code cleanups in from thr revise_chunks branch.
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Extract query routines into separate source module.
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial & parallel
(h5committest not required on this branch)
Bring r29914 from revise_chunks branch to trunk:
Banished -Wformat= warnings from the library, tools, and tests.
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring over remaining piece of r29675 from revise_chunks branch:
Refactor coding in H5D__farray_idx_delete() to be similar to H5D__earray_idx_delete()
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring r29934 from revise_chunks branch to trunk:
(1) Fix for HDFFV-9434: throw an error instead of assertion when v1 btree level hits the 1 byte limit.
(2) Modifications to better handle error recovery when conversion by h5format_convert fails.
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring h5format_convert tool from revise_chunks branch to trunk.
Tested on:
MacoSX/64 10.11.4 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring H5DOappend(), H5P[s|g]et_object_flush_cb, and H5P[s|g]et_append_flush
from revise_chunks branch to trunk. Brings along updated metadata cache
entry tagging, and the internal object flush routine.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring "file pointer patching" code & test from revise_chunks to trunk.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring 'none' chunk index from revise_chunks branch to trunk.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production (w/check-vfd)
(h5committest forthcoming)
Bring "single" chunk index from revise_chunks branch to trunk.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production (w/check-vfd)
(h5committest forthcoming)
Bring over more dataset tests from the revise_chunks branch.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Correct chunk index computation for earrays and farrays, particularly when
copying datasets with H5Ocopy() and other operations (like H5Dget_storage_size)
that require iterating over all chunks.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial & parallel
(h5committest forthcoming)
Clean up coding to increment/decrement underlying FAPL for multi/split
VFDs, to avoid copying property lists as much.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial & parallel
(h5committest forthcoming)
Finish correcting initializer for chunked layouts.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production
(to minor to require h5committest)
Bring support for earray and v2 B-trees from revise_chunks branch to
trunk.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring updated code for setting chunk size information from revise_chunks
branch.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Bring updated layout and EFL property comparisons and new "latest format"
flags from revise_chunks branch.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Corrects a problem where H5Dget_access_plist() returns a dapl that
does not contain all dapl fields.
Tested on: 64-bit Ubuntu 15.10 w/ gcc 5.2.1
autotools serial
Svn revisions reverted: 29341, 29326
Branch features/h5check_version was created for further development.
Files changed
test/tcheck_version.c
test/testcheck_version.sh.in
src/H5public.h
src/H5.c
config/lt_vers.am
bin/h5vers
Tested with h5committest.new
bin/h5vers:
Changed to copy the shared lib version from lt_vers.am to H5public.h
(Thanks to Larry who made the changes.)
config/lt_vers.am:
cosmetic change, nothing material.
src/H5public.h:
Changed by bin/h5vers.
src/H5.c:
Changed H5check_version() to use shared library version for compatibility
checking.
test/tcheck_version.c:
test/testcheck_version.sh.in:
Changed to use shared library version numbers instead of the HDF5 library
version numbers for testing.
Tested: tested in Jam (C only), platypus and osx1010dev (enable fortran and C++).
external dataset storage path behavior.
This check-in fixes a bug in the original check-in where the
external path stored in the file struct was not copied on reopen
causing subsequent dataset operations to fail.
Tested on: 64-bit Ubuntu 15.10 (Linux 4.2.0 x86_64) gcc 5.2.1
autotools serial w/ fortran and C++
autotools parallel (MPICH 3.1.4) w/ fortran
can be configured to generate extra debugging output in both
the autotools and CMake.
This can still be set by defining H5B_DEBUG manually or by
using a custom string in configure.
Tested on: 64-bit Ubuntu 15.10 (Linux 4.2.0 x86_64) gcc 5.2.1
autotools serial
CMake serial
r29069 and 72 (gheap cleanup)
r29041 and 43 (HDFFV-8740 external storage)
These cause problems in the daily tests. They will be
re-introduced when their respective problems have been
addressed.
Tested on: 64-bit Ubuntu 15.10 (Linux 4.2.0 x86_64) gcc 5.2.1
autotools serial (check-vfd) w/ Fortran and C++
autotools parallel w/ Fortran
CMake serial
