Changes made so that raw data for dataset objects are copied from cached info when possible instead of flushing objects to file and read them back in again.
* commit '54957d37f5aa73912763dbb6e308555e863c43f4':
Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes.
Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh.
Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1
Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
Bring 'none' chunk index from revise_chunks branch to trunk.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production (w/check-vfd)
(h5committest forthcoming)
Bring "single" chunk index from revise_chunks branch to trunk.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production (w/check-vfd)
(h5committest forthcoming)
Bring updated layout and EFL property comparisons and new "latest format"
flags from revise_chunks branch.
Tested on:
MacOSX/64 10.11.4 (amazon) w/serial, parallel & production
(h5committest forthcoming)
Buffers that will be written to disk will now always be cleared since
not doing this has huge security implications.
Tested on: 64-bit Ubuntu 15.10 (Linux 4.2.0, x86_64) gcc 5.2.1
serial autotools
parallel autotools (MPICH 3.1.4)
serial CMake
Clean up many mismatches between malloc/free and H5MM_malloc/H5MM_xfree
in the library and tests (and use of H5free_memory and H5Dvlen_reclaim). Also
make H5Ocopy use a private version of H5Lexists, which doesn't internally
throw (and suppress) errors when an object (or the path to it) isn't found in
the destination.
Tested on:
MacOSX/64 10.11.2 (amazon) w/serial & parallel
(h5committest forthcoming)
Fix tests accordingly and fix misuse of hbool_t in various places
Fix initialization of H5Pgcpl/ocpl structs in property decoding routines
Tested on:
Linux/32 (jam)
Linux/64 (platypus)
Linux/PPC64 (ostrich)
MacOSX/64 10.11
Refactor property list code to "deep copy" properties in the correct way,
retraining the rest of the library to copy & release things correctly. This
cleans up another batch of memory leaks, etc. within the library.
Tested on:
MacOSX/64 10.10.5 (amazon) w/serial & parallel
Linux/32 2.6.x (jam) w/serial & parallel
(h5committest forthcoming)
Complete revamp of package initialization/shutdown mechanism in the library.
Each package now has a single init/term routine.
This new way should avoid packages being re-initialized during library
shutdown and is also be _much_ more proactive about giving feedback for
resource leaks internal to the library.
Introduces a new "module" header file for packages in the library
(e.g src/H5Fmodule.h) which sets up some necessary package configuration macros
for the FUNC_ENTER/LEAVE macros. (The VFL drivers have their own slightly
modified version of this header, src/H5FDdrvr_module.h)
Also cleaned up a bunch of resources leaks all across the library and tests,
along with addressing many warnings, as I encountered them.
Tested on:
MacOSX/64 10.10.5 (amazon) w/serial & parallel
Linux/64 3.10.x (kituo) w/serial & parallel
Linux/64 2.6.x (ostrich) w/serial
The only remaining code consists of a few floating-point tests
that rely on pre-generated and checked-in VMS files. These have
been left alone, even though they will not be possible to
recreate, since testing VMS float behavior is still important.
Tested on: h5committest
- Adds 'F' suffixes for most float constants.
- A few constants MUST be of type double. These now receive the long
double L suffix and are then cast to double. I do this via a new
H5_DOUBLE() macro which was added to H5private.h.
Fixes: HDFFV-9148
Tested on: h5committest
Merge 64-bit ID changes from branch to trunk. (Plus a few minor cleanups
that aren't on the branch)
Tested on:
Mac OSX/64 10.9.4 (amazon) w/C++ & FORTRAN
(h5committested on branch already for a week)
Changes resulting from Klocwork static analysis tool, from Mark Miller
@ LLNL (miller86@llnl.gov).
Tested on:
Mac OS X/64 10.7.4 (amazon) w/debug, C++ & FORTRAN, using gcc 4.7.x
(too minor to require h5committest)
Description:
When jumping out from between H5_BEGIN_TAG and H5_END_TAG macros using
HGOTO_ERROR or HGOTO_DONE, the previous metadata tag is not reset on the dxpl.
This could cause problems when, for example, calling H5Ocopy within an
H5Literate callback.
Added new HGOTO_ERROR_TAG and HGOTO_DONE_TAG macros which must be used in place
of the above between H5_BEGIN_TAG and H5_END_TAG.
Tested:
jam, koala, ostrich (h5committest), durandal
Description:
When copying an object with attribute creation order tracked, the attribute
creation order was not copied correctly to the destination file, causing an
error if the creation order was also indexed (due to attempting to insert
duplicate keys) or incorrect creation orders otherwise. Fixed to copy the
creation order correctly.
Also fixed the attribute character set not being copied, and fixed an issue
where an attribute opened with H5Aopen (or similar, but not by_idx), from an
object using the latest format but without creation order being tracked, would
always report the creation order as 0 (and marked as valid).
Tested: jam, koala, ostrich (h5committest), durandal
Bring "merge committed datatypes during H5Ocopy" feature from branch to
trunk. (Also has some minor bugfixes with it)
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug
(h5committest coming up)
Don't check dataset storage size for compressed datasets with region
reference datatypes. (The address of the region reference type in the file
varies and affects the compressed size)
Tested on:
Mac OS X/32 10.7.2 (amazon) w/debug & production + check-vfd
Recalculate the size of destination attribute message when the source and
destination versions are different during an object copy operation.
(Jira: HDFF-7718)
Tested on:
Mac OS X/32 10.7.2 (amazon) w/debug
(h5committest upcoming)
Description:
H5Ocopy could get confused when copying a named datatype containing an
attribute which used that named datatype as its datatype. This happened
because H5Ocopy would recurse into the attribute's datatype before the object
the attribute was in was fully copied (i.e. before the "post-copy" pass).
Modified H5Ocopy to avoid recursing before the post-copy step in this case.
Required many changes, including to how non-committed shared messages are
copied.
Tested: jam, koala, heiwa (h5committest); durandal
Switch from H5P_DATASET_ACCESS_DEFAULT to H5P_DEFAULT for calls to
H5Rdereference2().
Tested on:
Mac OS X/32 10.6.8 (amazon) w/debug
(too minor to require h5committest)
Description:
Added internal tesitng routines to traverse a file and verify that symbol table
information is *always* cached, whenever possible. Added this check to the end
of many tests to check all the test files, right before the call to h5_cleanup.
Tested: jam, amani, heiwa (h5committest)
Close out various resource leaks and usages of uninitialized memory that
were flagged by valgrind. [There's still some more valgrind warnings, but
it's better now... :-/ ]
Also clean up warnings and code formatting.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & valgrind
(h5committest forthcoming)
Description:
Attempting to copy an object with NULL references (all bytes zero) with the
H5O_COPY_EXPAND_REFERENCE_FLAG flag set would cause a failure or an assertion
(depending on whether it was in debug mode). Changed copy routine to detect
NULL references (object and region) and avoid attempting to expand the reference
in this case.
Tested: jam, linew, amani (h5committest)
Clean up srcdir querying code more, extracting it into single header file,
to avoid compiler warnings.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug & prod
Linux/32 2.6 (jam)
Unify srcdir handling for test executables and allow them to use the srcdir
setting from configure time without requiring the 'srcdir' environment variable
be set (although you still can, to override the built in setting). Attempted
to get this right for Windows builds also.
Also add dependency between src/H5Tinit.c and src/libhdf5.settings, so
that the test/testcheck_version.sh script works correctly.
Tested on:
Linux/32 2.6 (jam)
Mac OS X/32 10.6.2 (amazon)
Description:
Support for expanding external links was not implemented in H5Ocopy, even though
a flag existed for H5Pset_copy_object to enable this. This patch implements
this feature.
Tested: jam, amani, linew (h5committest); Fedora
Bring Coverity changes into the trunk:
r17877:
Error 266: Uninitialized memspace set to -1. Changed malloc and free to HDmalloc and HDfree. Removed unused dtype var.
r17878:
Error 265: Uninitialized mem_space set to -1. Changed malloc and free to HDmalloc and HDfree.
Error 267: Uninitialized smspace set to -1. Changed malloc and free to HDmalloc and HDfree.
r17879:
Error 242: Uninitialized aid set to -1.
Error 243: Uninitialized sid set to -1.
Uninitialized tid set to -1 for consistency
r17880:
Error 242: reinitialized aid to -1 after close to be ready for reuse.
Error 243: reinitialized sid to -1 after close to be ready for reuse.
reinitialized tid to -1 for consistency after close to be ready for reuse.
r17881:
use valgrind to check there is a memory leak. The fix is to free ptrstr in line 5838 at xml_dump_group() of h5dump.c after it is used. run the valgrind after the fix, no memory leak for that part of the code.
r17882:
Fix Coverity items 256 and 269.
r17883:
Error 222-230: initialized hid_t vars to -1 after close to be ready for reuse.
Also added H5Tclose for tid in gent_bigdims
r17884:
Bug fix (Coverity run2, view 23, dead code)
(this is pair-program done by Albert, Elena and Larry).
Confirmed and fixed the deadcode in hyperslab read branch of function
diff_datasetid.
(Discovered other bad code that diff_datasetid() should be recoded.
Bug 1693 is entered for this.)
r17906:
Fix Coverity item 260.
r17907:
262: Initialized hid_t's dtype, dtype_tmp and file to -1. Initialized H5T_t * dt to NULL.
r17908:
Fix Coverity item 261.
r17909:
Fix Coverity item 248.
r17910:
Revise fix for Coverity item 248.
r17911:
Resolved coverity issues #s 263, 162, 163, 164. All issues in dsets.c. Initialized fid and did hid_t's. filter_corrupt function was returning in the middle of an if statement, bypassing free calls. Updated error handling to free buffers and hid_t's appropriately.
r17912:
(done by Larry and Albert)
Cleanup Coverity view warnings (#231-241) about
using uninitialized variables. Initialized all
of them.
r17913:
Resolved issue 251 and 264. Initialized tid2 to -1 and initialized buffers that were freed in case of an error.
r17914:
Resolved coverity issues 66, 220, and 221:
66: Negative Return assignment ignored
220,221: Initialized hid_t's.
r17915:
Fix Coverity item 247.
r17916:
Fix Coverity item 246.
r17917:
Fix Coverity item 245.
r17918:
Fix Coverity item 244.
r17919:
Coverity Issue #84: Moved asserts in H5T_cmp to the top of the function, and converted them to HDassert. Coverity complaining about using potentially NULL pointer without checking it. Want to see if Coverity will accept Assertions as acceptable checking before using the value.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.2 (amazon) in debug mode
Mac OS X/32 10.6.2 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
When copying a dataset using a vlen inside a compound, the various dataset
copying callbacks would allocate a background buffer but would not use it when
converting from disk to memory, only memory to disk. This caused an assertion
failure as compounds always need a background buffer. These callbacks have
been modified to use the background buffer for both conversions.
Tested: jam, linew, smirom (h5committest)
Description:
Versions of the library between 1.3.0 and 1.6.3 have a bug which prevents them
from opening any file that does not have the root group's symbol table
information cached in the root group's entry in the superblock. Prior to 1.8
this was not an issue as this information was always cached. However, 1.8.0
stopped writing this information (which is not required by the file format
specification), and these older versions can therefore not read files created or
last written by versions 1.8.0 to 1.8.2. This fix modifies the library to once
again add this information to the superblock (when using the old file format).
Tested: jam, linew, smirom (h5committest)
Bring r16435 from revise_chunks branch back to trunk:
Expand object copy tests for chunked datasets to include 1-D datasets
with an unlimited dimension. (Fix typo in comment for test/links.c)
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(more thoroughly tested already on revise_chunks branch)
Description:
Attribute object copy routines have been moved from H5Oattr.c to H5Aint.c.
These routines are now shared between compact and densely stored attributes.
New routines written to support the copying of dense attributes. This patch wasmostly written by Peter Cao.
Tested: jam, smirom (h5committest)
