Some systems, like Mac, strings inspect library files and older versions
of strings maybe know newer library format, resulting in errors.
Make it read the file as stdin, avoiding this problem.
Tested: h5committest. all passed.
The ERROR macro had a typo of missing a separator comma. When used, it seg.
fault. Added the comma back in.
Fix is reviewed in Crucible HDF5-107.
Tested: h5committest passed. Also tested using the ERROR macro in the
Mac (Owl) platform. Before fix, it seg. fault. After fix, it prints the
message as intended.
Description:
Adds 2 new API functions, H5Dscatter and H5Dgather. H5Dscatter retrieves data
from a specified callback function and scatters it into a selection, defined by
a supplied dataspace, within a supplied memory buffer. H5Dgather gathers data
from a selection within a supplied memory buffer and passes it in a contiguous
form to a supplied callback function. Added tests for these functions
Tested: jam, ostrich, koala (h5committest); ummon
Description:
Adds 2 new API functions, H5Dscatter and H5Dgather. H5Dscatter retrieves data
from a specified callback function and scatters it into a selection, defined by
a supplied dataspace, within a supplied memory buffer. H5Dgather gathers data
from a selection within a supplied memory buffer and passes it in a contiguous
form to a supplied callback function. Added tests for these functions
Tested: jam, ostrich, koala (h5committest); ummon
Bring reviewed changes from Coverity branch back to trunk (QK & JK):
r20457:
Coverity issue 691: return of H5duo could be negative. Fixed by using
STDOUT_FILENO and redesign parse_command_line and main to cleanup file
allocations. The output_file var is null when using stdout. In cleanup do not
close output_file if NULL.
r20510:
Initialize ufid = -1 and predicate HDclose call on ufid != -1
r20511:
Purpose: Fix coverity issue 1715
Description: Free "file" and nested data on failure in H5FD_core_open.
r20512:
Initialize ifid = -1 and predicate HDclose call on ifid != -1
r20514:
Initialize h5fid = -1 and predicate HDclose call on h5fid != -1
r20516:
Added else branch to the if (ret_value < 0) check.
r20522:
Addressed coverity issues 930-933, 850, 836, 835, 1307. All minor
potential buffer overwrite bugs, or coverity errors. Fixed by replacing
strcpy and sprintf with strncpy and snprintf.
r20523:
fixed coverity issues 68, 1120, 1116i
r20524:
Check H5Z_SZIP->encoder_present < 1 assuming 0 represents absence.
r20601:
Purpose: Fix coverity issues 1703-1705
Description: Modified the cleanup code in test_free in accum.c to reset
allocated buffers to NULL after they are freed, and modified the error cleanup
code to check if these buffers are NULL before freeing them. Also fixed some
unrelated warnings in accum.c.
r20602:
Use HDsnprintf and HDstrncat
r20603:
Purpose: Fix coverity issues 808-809
Description: Modified test_core in vfd.c to check the returns from malloc, and
keep track of whether points and check are allocated by setting them to NULL
when they are not. Added code to free points and check on error if they are
not NULL. Also fixed unrelated warnings in vfd.c.
r20604:
Use HDstrncpy.
r20605:
Use HDstrncpy and HDstrncat.
r20606:
Purpose: Fix coverity issue 807
Description: Modified long_compact in stab.c to keep track of whether objname is
allocated by setting it to NULL when it is not. Added code to free objname on
error if it is not NULL.
r20607:
Changed string function calls to use versions that specify the string length
to fix coverity issues 832 and 839.
Tested on:
Mac OSX/64 10.8.2 (amazon)
(Too minor to require h5committest)
Have free space manager use temporary address space for storing the
section info, until the file is flushed or closed.
Tested on:
Mac OSX/64 10.7.4 (amazon) w/debug, gcc 4.7.x, C++, FORTRAN & threadsafe
(h5committest forthcoming)
Changes resulting from Klocwork static analysis tool, from Mark Miller
@ LLNL (miller86@llnl.gov).
Tested on:
Mac OS X/64 10.7.4 (amazon) w/debug, C++ & FORTRAN, using gcc 4.7.x
(too minor to require h5committest)
Description:
When jumping out from between H5_BEGIN_TAG and H5_END_TAG macros using
HGOTO_ERROR or HGOTO_DONE, the previous metadata tag is not reset on the dxpl.
This could cause problems when, for example, calling H5Ocopy within an
H5Literate callback.
Added new HGOTO_ERROR_TAG and HGOTO_DONE_TAG macros which must be used in place
of the above between H5_BEGIN_TAG and H5_END_TAG.
Tested:
jam, koala, ostrich (h5committest), durandal
Refactor ID class registration to be more like how other "class"s in the
library are registered, and clean up compiler warnings.
Tested on:
Mac OSX/64 10.7.4 (amazon) w/debug
(Too minor to require h5comittest)
Copy the file image test file from the source directory to the build
directory, instead of trying to open the [in-memory] file R/W.
Also, clean up a few other test files that were left lying around from
other tests.
Tested on:
MacOSX/64 10.7.4 (amazon) w/debug
Too minor to require h5committest
(2) fix to extend into the aggregator based on threshold
(3) fix for the "flavor" logging problem
(4) fix for test failures due to the above fixes
(5) bug fix for h5stat when h5stat against a non-existing file
Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward
the final design (as exemplified by the H5EA & H5FA code).
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & parallel
Correct several errors in fractal heap code: root indirect block was
getting pinned/protected more than once, "single" free space sections weren't
getting "re-parented" correctly when the heap transitioned between having a
root indirect block and a root direct block, and several related issues. Also
cleaned up some warnings in library/tests.
Tested on:
FreeBSD/32 8.2 (loyalty) w/gcc4.6, w/C++ & FORTRAN, in debug mode
FreeBSD/64 8.2 (freedom) w/gcc4.6, w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (koala) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Mac OSX/64 10.7.3 (amazon) w/debug
Description:
When copying an object with attribute creation order tracked, the attribute
creation order was not copied correctly to the destination file, causing an
error if the creation order was also indexed (due to attempting to insert
duplicate keys) or incorrect creation orders otherwise. Fixed to copy the
creation order correctly.
Also fixed the attribute character set not being copied, and fixed an issue
where an attribute opened with H5Aopen (or similar, but not by_idx), from an
object using the latest format but without creation order being tracked, would
always report the creation order as 0 (and marked as valid).
Tested: jam, koala, ostrich (h5committest), durandal
Bring "merge committed datatypes during H5Ocopy" feature from branch to
trunk. (Also has some minor bugfixes with it)
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug
(h5committest coming up)
Minor code safety issue in test/fheap.c and whitespace in other files.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug
(Too minor to require h5committest)
This occurs due to the istore test creating very large files on systems which
do not have POSIX-like sparse file semantics. The large amount of I/O causes
the test to run for a very long period of time.
The fix was to copy the "big" test's sparse file check and only run the
largest sparse file test when POSIX-like sparse file semantics are found.
Tested on:
jam (nfs)
ostrich (nfs)
loyalty (ufs)
64-bit linux VM (ext4)
64-bit OS-X Lion (hfs, detected as not POSIX sparse)
64-bit Windows 7 (NTFS, detected as not POSIX sparse)
The OS-X failure to pass the sparse check is likely an error and will be
entered as a new bug.
The parser mistaked "-" as substraction. I fixed it and also fixed another problem
with some special cases like 100-x and 2/x.
Tested on jam, koala, and ostrich.
Description:
When using the new object header format, it was possible for corruption to occur
if the first object header chunk changed size such that the lenght of the "chunk
0 size" field changed. This only occurred if there were messages that had not
been decoded. The original algorithm that changed the object header chunk size
marked all messages as dirty, causing those that had not been decoded to have
both the raw and native form invalidated. Changed the algorithm to avoid
marking messages dirty and added assertions to catch the case where messages
are dirtied without being decoded (or recently created) first.
Tested: jam, koala, ostrich (h5committest), durandal
Description:
When using the new object header format and adding an attribute with a size near
64K, it was possible for file corruption to occur. This happened only if the
first object header chunk was smaller than 256 bytes and then grew to larger
than 64K after the attribute was added.
Tested: ostrich, jam, koala (h5committest), durandal
Correct corner case for creating a contiguous dataset with a zero-sized
dataspace, when the allocation time is set to early.
Also clean up a few compiler warnings in the dataspace code.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & parallel
Remove some leftover uses of the __FUNCTION__ macro, replacing them with
FUNC macro, as used everywhere else.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
(too minor to require h5committest)
Add FUNC_ENTER macros for package-private routines and begin process of
switching package routines to use them. All H5G routines are currently
finished.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
Description:
When shrinking a chunked dataset, the library fills in the unused parts of
chunks that have been shrunk. The fill value buffer allocated for this purpose
had a maximum size of 1 MB, but the fill was performed in a single operation.
Therefore, if the amount of unused space in a chunk after being shrunk was
greater than 1 MB, the library would read off the end of the fill value buffer.
Changed the maximum fill buffer size to be equal to the chunk size.
Tested: durandal; jam, koala, heiwa (h5committest)
Don't check dataset storage size for compressed datasets with region
reference datatypes. (The address of the region reference type in the file
varies and affects the compressed size)
Tested on:
Mac OS X/32 10.7.2 (amazon) w/debug & production + check-vfd
Recalculate the size of destination attribute message when the source and
destination versions are different during an object copy operation.
(Jira: HDFF-7718)
Tested on:
Mac OS X/32 10.7.2 (amazon) w/debug
(h5committest upcoming)
Description:
Added new H5SL_TYPE_GENERIC skip list type, which uses void *'s as keys and a
client-supplied callback for key comparison. This was added to support the
upcoming "merge named datatype" feature for H5Ocopy, but may be used in other
places as well. Also added testing.
Also fixed a potential bug with the H5SL_TYPE_OBJ implementation, and added
testing for that.
Tested: jam, koala, heiwa (h5committest), durandal
Correct error in loading local heap prefix & data block from the file.
Sometimes the local heap's prefix could be loaded before the data block (e.g.
using H5Oget_info), but then when the data block was loaded later, the free
list information would get lost, causing the heap's size to grow larger than
necessary. This is Jira bug #HDFFV-7767
Tested on:
Mac OS X/32 10.7.2 (amazon) w/debug
(h5committest coming up)
Description:
H5Ocopy could get confused when copying a named datatype containing an
attribute which used that named datatype as its datatype. This happened
because H5Ocopy would recurse into the attribute's datatype before the object
the attribute was in was fully copied (i.e. before the "post-copy" pass).
Modified H5Ocopy to avoid recursing before the post-copy step in this case.
Required many changes, including to how non-committed shared messages are
copied.
Tested: jam, koala, heiwa (h5committest); durandal
Some machines, like LLNL udawn, a blue-gene machine, requires all executables,
be launched by some command like mpirun.
Solution:
Added $RUNSERIAL to launch the executable.
Tested: LLNL uDawn.
them is in the INIT_INTEGER macro definition in dt_arith.c. It complained about line 150 where it tried to subtract 1 from
the negative minimal value of "int", causing it to overflow (or underflow). So I revised the code to avoid it.
Tested on jam, koala, linew, and Mac OS Lion with CLANG compiler.
temp_point->l = (unsigned long long)((i * 100 + j * 1000) * n);
The value can overflow the signed int before being converted to unsigned long long. So I changed it to
temp_point->l = (unsigned long long)((i * 40 + j * 400) * n);
to keep it under the maximal value.
Tested on jam. Simple change.
Ran bin/reconfigure to update the Makefile.in in directories not part of the fortran directory check=in. Updates Makefile.in due to changes made in configure.in for the Fortran 2003 additions.
Tested on all platforms run under daily tests.
i_mask = ~((unsigned)~0 << (precision[0] + offset[0])) & ((unsigned)~0 << offset[0]);
in line 3680 left shifted a 32-bit integer for 32-bit. The result is undefined by C language. A user
discovered it using clang compiler with -fcatch-undefined-behavior option (see Issue 7674
in Jira). So I changed it in a funny way to avoid it.
Tested on jam, koala, and heiwa.
Further patches for Jira 7638, to ensure that H5D/Aget_type will
be attached to the correct top file. (This change to the library should
reverted when Jira 7638 is finally finished)
Tested on:
FreeBSD/32 8.2 (loyalty) w/gcc4.6, w/C++ & FORTRAN, in debug mode
FreeBSD/64 8.2 (freedom) w/gcc4.6, w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (koala) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Mac OS X/32 10.6.8 (amazon) in debug mode
Mac OS X/32 10.6.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
1. Quincey changed H5P_DATASET_ACCESS_DEFAULT to H5P_DEFAULT for the tests that call H5Rdereference. Inside H5R_dereference, I assigned H5P_DATASET_ACCESS_DEFAULT to OAPL_ID if it is H5P_DEFAULT.
2. I added the test for invalid OAPL_ID in trefer.c.
Tested on jam, koala, heiwa.
Switch from H5P_DATASET_ACCESS_DEFAULT to H5P_DEFAULT for calls to
H5Rdereference2().
Tested on:
Mac OS X/32 10.6.8 (amazon) w/debug
(too minor to require h5committest)
Check in ugly version of fix for Jira #7638. I'm going to leave the issue
open in Jira and when the library has been refactored to use shared file
pointers (instead of top file pointers) for file operations, I'll uncomment
the check in src/H5Oshared.c
Tested on:
Mac OS X/32 10.6.8 (amazon) w/debug
(too small for h5committest)
super block. We took it out in v1.8 library because it's meaningless
for the MULTI file. v1.8 library saves the EOA for the metadata file,
instead. But this caused some backward compatibility problem.
v1.8 library couldn't open the file created with v1.6 library. I
fixed the problem by checking the EOA value to detect the file
created with v1.6 library.
Tested on jam, koala, and heiwa.
data, overflowing values are retained. When conversion happens, the values become -1. The conversion function puts -1
when overflow happens. I added two new dataset transfer property to control whether to fill 0xff in the destination
data or convert to the destination data when overflow happens. The two new functions are H5Pset(get)_enum_conv_overflow. I also added test cases in enum.c and dtypes.c.
Tested on jam, koala, and heiwa.
Description:
When closing an object that had two open id's, the library would previously fail
to account for the status of the "holding_file" field, potentially causing the
file's "nopen_objs" field to become inconsistent. This caused problems when
opening the same object twice through an external link using th external file
cache. Modified the library to properly account for the status of the
"holding_file" field when closing an object id, even when the underlying
"shared" object isn't closed.
Tested: jam, koala, heiwa (h5committest)
The problem was that the test was a clone of the SEC2 test. The SEC2 test accepts a file descriptor (int) via a void pointer from H5Fget_vfd_handle() whereas the STDIO test needs to accept a FILE pointer. Storing the FILE pointer in an integer failed on some systems. The STDIO test now stores the file handle in FILE pointer and checks it for NULL instead of < 0.
Clean up various warnings & code formatting issues.
Bring changes from Coverity branch to trunk:
r20085:
Purpose: Fix coverity issue 793
Description: Modified H5S_hyper_project_simple_higher() to free the entire span
list in new_space on failure.
r20091:
This is a fix for coverity bug #1683.
Changed the two printfs to use %lu (unsigned long) for printing "dset_size".
r20162:
Purpose: Fix coverity issue 785
Description: Modified H5T_enum_nameof() to free "name" on failure if it was
allocated. Also clarified some code in H5S_hyper_rebuild_helper().
r20189:
Addressed coverity defect 783.
H5SL_new_node() in H5SL.c was failing to free space allocated in its
first alloc if the second alloc failed. Added a call to H5FL_FREE
to address this issue.
This is purely to keep coverity happy -- if this code is ever triggered,
we have much larger problems.
Note that this fix will trigger an unused return value complaint
from coverity next week.
r20190:
Fixed Coverity issues 1561 1565 and 1678 (UNUSED_VALUES) by moving checks of return values to after the function call.
r20191:
Fixed coverity issues 643 644 and 1678 (CHECKED_RETURN).
r20232:
Addressed coverity issues 923-925. Replaced calls to sprintf with calls
to HDsnprintf.
r20233:
Fix coverity issue 662. Don't try to sort 0 attributes in H5Aint.c.
r20234:
Fix coverity issue 664. Check for NULL before dereferencing in H5Gdeprec.c.
r20271:
Purpose: Fix coverity issue 784
Description: Modified H5_debug_mask() to keep a list of files opened for use as
a debugging output stream, and modified H5_term_library to close these files on
exit.
r20272:
addressed coverity issues 838 & 955. Issue was use of strcpy() -- existing
code was safe, but modified to use strncpy() to keep coverity happy.
r20273:
Addresed coverity issues 1388 and 1389.
Initialized sel_iter->type to NULL in H5S_select_iter_init.
r20275:
Purpose: Fix valgrind issue in mf.c
Description: Fixed bug (incomplete if statement) in test_mf_fs_alloc_free() so
the retrieved node gets freed.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Add explicit test that checks that the 'fileno' field in H5O_info_t
is the same for objects in the same file, whether the file was opened twice
or not.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Description:
Added internal tesitng routines to traverse a file and verify that symbol table
information is *always* cached, whenever possible. Added this check to the end
of many tests to check all the test files, right before the call to h5_cleanup.
Tested: jam, amani, heiwa (h5committest)
Correct several problems with compound datatypes that don't have any
fields added:
- Change assertion to error report when a file is encountered which has this
situation.
- Added check to attribute creation to avoid creating attributes with a
datatype like this (datasets and named datatypes already have the check)
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
General shared library improvements for CYGWIN / AIX
Description:
Shared libraries are disabled on both CYGWIN and AIX due
to inability to build them correctly. Part of the problem
in both of these situations is the lack of the libtool
flag -no-undefined, which tells libtool that all needed
symbols are defined at link time (a requirement on these
systems) and that it's okay to build shared libraries.
Another problem are lack of dependencies between wrapper
libraries and core C HDF5 library.
This patch addresses both of these by fixing configure to
add in -no-undefined flag for libtool during linking and
adds automake dependencies in the Makefile.am files.
After testing, both CYGWIN and AIX now generate shared
libraries, but there are still some test failures in each.
(cache_api, dt_arith, and testerror.sh on CYGWIN, and
fortran tests on AIX).
Even though the shared libraries are not quite perfect,
this is a general improvement to what we had before, so
I'm applying the patch anyways. Note that default behavior
of shared libraries on these systems being disabled has
NOT been changed and requires the use of the
--enable-unsupported to attempt to build them.
We will need to address the test failures in each
architecture prior to formally supporting shared
libraries on each.
Tested:
h5committested & CYGWIN tested (on bangan)
(AIX tested by Albert on bp-login2)
Add "silent make" mode configure option.
Description:
Automake 1.11 has a new option available that allows for a
silent make mode. This functionality needs to be explicitly
enabled in configure.in via the use of the automake macro
AM_SILENT_RULES, which is what this commit is adding.
This introduces a new configure option:
--{en|dis}able-silent-rules
This option is on by default, and simplies compile and link
line outputs when building the library. Disabling this option
will print full "verbose" output (i.e., full compile and
linking lines for each target).
Tested:
This was tested on jam & h5committested
forbid setting the dimension size bigger than existing maximal size. In this checkin, I restored it to the previous
behavior that any change will wipe out previous record of dimensionality.
Tested on jam - a simple change.
r20440 and r20469:
1. The dataspace code has another bug - when the maximal dimension isn't passed in for H5Sset_extent_simple, it
is supposed to be same as the dimension. The current library sets NULL to it. I corrected it and added a
test case to it.
2. I corrected the tests of Fortran and C++ for this problem.
Tested on heiwa, jam, and amani.
Remove global variable for the path name in the group traversal code. Also
clean up a bunch of compiler warnings.
Tested on:
Mac OS X/32 10.6.7 (amazon) w/debug & production
FreeBSD/32 6.3 (duty) w/debug
FreeBSD/64 6.3 (liberty) w/debug
Solaris/32 2.7 (linew) w/debug
- Revise shared Fortran library disabling scenarios in configure
- Improve configure output summary
Description:
Shared Fortran libraries are not supported on Mac, but were being
disabled by configure in a way that also forced the C libraries
to be static-only. This has been fixed, so now only shared Fortran
is disabled while shared C can remain.
This prompted two additional changes:
1. While working on the check that addresses whether or not
shared Fortran libraries are allowed, removed old and no
longer needed check(s) that disable shared Fortran
libraries with HP, Intel 8, PGI, and Absoft compilers.
(Essentially, Mac is the only situation in which Fortran
shared are disabled by configure.)
2. Having two different states of libraries (i.e. shared C
library with static-only Fortran library) was not apparent
in the configure summary, which labeled all libraries as
either shared and/or static. I've added lines to both the
C++ and Fortran output sections to list shared/static-ness
of these libraries specifically.
Additionally, I've made sure that the new --enable-unsupported
configure option correctly overrides configure if it tries to
disable a shared library.
Tested:
jam, fred, & h5committest
1. I added a test case of extending dataset of zero dimension size and shrinking back to zero dimension size.
2. I updated the Makefile to include the new data file to be cleaned up.
Tested on jam - relatively simple.
Progress on bug 2138 (bugzilla) / HDFFV-4298 (JIRA)
Description:
- Added basic VFD tests (i.e. cloned the SEC2 test) for the Windows, STDIO and log VFDs.
- Added H5Pget_driver() sanity check to all VFD's tests.
- Added log output file to list of files to clean up in Makefile.am.
Tested on:
- h5committest script platforms (jam, amani, heiwa)
- Windows 7 (32- and 64-bit)
Description:
Implements a cache of files opened through external links. Adds the public
functions H5Pset_elink_file_cache_size(), H5Pget_elink_file_cache_size(), and
H5Frelease_file_cache().
Tested: jam, amani, heiwa (h5committest), fedora 64.
Bring changes from Coverity branch to trunk:
r19930:
Fix memory leaks involving VL attributes in h5repack and h5diff. The buffers in
copy_attr and diff_attr were not checked for the presence of a vlen before being
freed, and vlen storage was never reclaimed. Added checks and calls to
H5D_vlen_reclaim().
r19933:
Purpose: Fix memory leak in H5L_move_cb()
Description: H5L_move_cb copied the source link using H5O_msg_copy() but freed
it manually using H5MM_xfree(). Since H5O_link_copy allocates the link using
H5FL_MALLOC, this causes the link to be allocated from the free list but is
never put back on the free list when it is freed. This prevents the link free
list from shutting down properly. Modified H5L_move_cb() and H5L_move_dest_cb()
to free the link properly using H5O_msg_free().
r19973:
Fix resource leaks by freeing string created by HD5f2string
r19974:
Issue #345: Inialize buf variable to null
Tested on:
Mac OS X/32 10.6.6 (amazon) w/debug & production
(h5committested on Coverity branch)
Clean up Coverity warnings, and fix some style issues:
r19735:
Fix for memory leak in test/mf found by valgrind.
r19736:
Fix memory leak in h5repack. The buffer in copy_objects, when copying the
entire dataset at once, was not checked for the presence of a vlen, and vlen
storage was never reclaimed. Added check and call to H5D_vlen_reclaim().
r19772:
Change H5assert() to
if (H5T_VLEN != src->shared->type || H5T_VLEN != dst->shared->type)
HGOTO_ERROR(H5E_ARGS, H5E_BADTYPE, FAIL, "not a H5T_VLEN datatype")
r19774:
removed unused priv.
r19775:
removed unused variables
r19778:
Fix memory leak comparing for variable length data types.
r19834:
Fixed memory leaks found by valgrind. Memory errors remain for another day.
Tested on:
Mac OS X/32 10.6.6 (amazon) w/debug & production
(h5committested on branch)
Description:
When using the scale-offset filter with floating point data or fill values, big
endian machines would save some metadata in the wrong byte order. This caused
such datasets to yield incorrect data when read on little endian machines.
Fixed the scale-offset filter to always save this metadata in the right byte order (i.e. little endian).
Tested: jam, amani, heiwa (h5committedt); fedora, linew
"*buffer_size = j + 1;" was mistakenly taken out by someone. It is necessary to update the new size. I
put it back and made 2 test cases for integer and float to verify the correct dataset size.
I'm bringing the fix from 1.8 branch. The changes to configure.in, tools/misc, config, Makefile.am are only property changes.
Tested on jam. But I tested 1.8 on jam, heiwa, and amani.
Changed the test to run Huge Dataset (tens of GB), Xtra large dataset(4GB big),
Large dataset (2GB big), or merely 1GB big, depending on if the file system
supports sparse file or if it supports larger than 32bigs I/O.
Tested: h5committest, jam (serial), Windows (Bangan which is Windows 7, 32bits).
file name listed for DISTCLEANFILES should be testlinks_env.sh
2. Add comments to test_4() in test/external.c about the link name "/ link".
3. Fix memory leak as reported by valgrind in src/H5Lexternal.c:
free memory pointed to by tmp_env_prefix for HDF5_EXT_PREFIX case.
Add additional error checking to catch erroneous user input.
Description:
Attempting to retrieve a links's name by index in the case
where the link is external and the file that the object is
located in doesn't exist was causing a segmentation fault
(in production) and an assertion failure (in debug).
The segfault wasn't occuring until the metadata accumulator
attempted a write, so I've added error checking higher in
the pipeline in H5O_protect (where there was previously just
an assert) to catch this. I've also added additional asserts
in the H5F layer where there were none.
Additionally, I added another case to the links.c test to
test that this fails gracefully instead of segfaulting or
asserting out.
Tested:
h5committest and gandalf (mac os x)
Also updated root CMakeLists.txt to output a message when unsopported options are configured with the PARALLEL option. CMake will still generate files.
Tested: windows and local linux
1) Move the test for H5D_EXT_PREFIX in links.c to a separate file: links_env.c
2) links_env.c will be used by testlinks_env.sh to test for the environmental
variable H5D_EXT_PREFIX in searching for the external linked file.
Add another test for shutting down open object IDs without using the
standard "close" routines.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(too minor to require h5committest)
double to (unsigned) long and from (unsigned) long to long double. I added tests in configure.in
to detect these algorithms. Before I can figure out them, I disable the tests in dt_arith.c.
There are property changes to tools/misc, config, and Makefile.am when I brought the fix from 1.8.
Tested on jam, heiwa, amani, IBM Power6 Linux machine in Holland (huygens.sara.nl).
Do some extra range-checking on H5Pset_elink_acc_flags() calls.
Clean up some minor compiler warnings also.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(too minor to require h5committest)
Bring Coverity revisions from branch back to trunk, and clean up some other
misc. compiler warnings also.
r19500:
Fix coverity items 1446 and 1447. Moved up calls to memset in test_cont in
ohdr.c so the test never tries to close uninitialized locations.
r19501:
Fix coverity items 1398-1445. Various uninitialized variable errors in fheap.c.
r19502:
Fixed coverity issue 579 and some additional warnings in the file as well.
r19503:
Bug fix: This fix addressed the "RESOURCE_LEAK" problems #789 and 790, run 26
r19504:
minor mods to try to keep coverity from flagging false positives.
r19505:
Fixed coverity issues 566 - 571. Declared variables that are passed to functions that use them as arrays to be arrays of size 1.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug, production & parallel
(h5committested on trunk)
- Fix a bug in each of the metadata accumulator source and test code
Description:
- In accum.c test file, switch dxpl_id used in H5F_accum_*
function calls to H5P_DATASET_XFER_DEFAULT (instead of
H5AC_dxpl_id), to fix compilation on windows.
- Changed boundary checking from <= to < when checking
if a read from disk with overlapping dirty metadata in the
accumulator has the read ending such that it aligns exactly
with the dirty accumulator (line 234 of H5Faccum.c).
Tested:
- h5committested
Merge accum_tests branch back to the trunk.
Description:
Changes consist of addition of tests for H5Faccum.c source code,
as well as some fixes that address some discovered bugs in the
metadata accumulator on several corner cases.
Tested:
h5committested
Address issue with object headers being created getting evicted from
the metadata cache cache before they are completely initialized. This is
done by pinning the object header in the cache until it is completely
initialized and attached to a group.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.4 (amazon) in debug mode
Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
with the 1.8 library to compile. The cause is from the mismatch of H5Eprint1 and H5Eprint2
set through H5Eset_auto. I changed the structure H5E_auto_t. I added a IS_DEDAULT flag in
it. Both H5Eprint1/2 are the default now. If the user sets his/her own printing function.
Then a call to H5Eget_auto1/2 will have to match H5Eset_auto1/2.
Tested on heiwa, jam, and amani.
The property change in configure.in, config, and Makefile.am came from the merge of the 1.8
library change.
flag IS_DEDAULT in the H5E_auto_t structure. Both H5Eprint1/2 are the default now. If the user
sets his/her own printing function. Then a call to H5Eget_auto1/2 will have to match
H5Eset_auto1/2.
Tested on jam, heiwa, and amani.
Bug 1707 is that H5Eset_auto causes a seg fault when an application uses -DH5_USE_16_API with
the 1.8 library to compile.
I created a branch off the trunk called set_auto to continue working on the problem.
Tested on jam - backing out, simple.
Create a new, more streamlined method of generating I/O sequences when
there is only a single block defined by a hyperslab. This improves the
performance of Ger Van Diepen's (LOFAR team) benchmark by about 20%.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.4 (amazon) in debug mode
Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
Description:
The AIX platform does not install the lib*.so for some reason but libhdf5.a
acts like a shared library. Until we can find out how to fix this, patch
up tetlibinfo.sh to not look for an lib*.so in AIX. (Note the test results
look wierd as it says libhdf5.a passed and skipped.)
Tested: NCSA BP for both enable and disable-shared.
through H5Eset_auto. I changed the union in the structure H5E_auto_t. Another change is to
make H5Eget_auto fail if H5Eset_auto is called to set the printing function. I'll write a
document for it.
Tested on heiwa, jam, and amani.
The property change in configure.in, config, and Makefile.am came from the merge of the 1.8
library change.
Correct Bz#1968 (Problem when opening new format file with
H5F_LIBVER_EARLIEST) by initializing symbol table leaf information in a
more universal way. Also cleaned up other superblock initialization a
little bit while I'm here.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.4 (amazon) in debug mode
Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
Clean up the internal usage H5I_dec_ref() to eliminate the sequence of
flags that has been creeping in.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.4 (amazon) in debug mode
Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
Bring back various modifications to the v2 B-trees that were introduced
in the revise_chunks branch but aren't dependent on any file format or API
changes.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.4 (amazon) in debug mode
Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
Description:
The fractal heap header structure keeps a pointer to the file associated with
it. However, it is possible for that file pointer to be closed while the
header is still in cache (through the shared file pointer). Previously, the
header's file pointer was not updated and subsequently pointed to an invalid
file structure. Modified fractal heap code to update the file pointer every
time the header is accessed.
Tested: jam, linew, amani (h5committest)
Add windows threads support to HDF5.
Description:
Added calls to the windows threads library to the H5TS layer, and wrapped
most calls to either pthreads or windows threads library with portable
H5TS-style defines. Modified tests to use portable function definitions
as well.
This can be configured via CMake with the HDF5_ENABLE_THREADSAFE
option, and should work on windows vista and later operating systems.
Tested:
h5committest, plus threadsafe with pthreads on jam and amani,
and tested on a Windows Vista VM with threadsafe using windows threads.
Add some extra range checking for invalid chunk sizes.
Tweak default chunk sizes to be zero, so that it's more likely that
application developers will be forced to set them correctly.
Add a few tests to verify these checks.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(too minor to require h5committest)
Corrected use/name of source folder aliases.
Duplicated FindMPI.cmake so that non-c++ compiler is found first (recommemded commands did not work).
Tested: local linux with mpich
Clean up a few compiler warnings and style issues in/near recent changes.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(too minor to require h5committest)
Bring revisions from Coverity branch back to trunk:
r19191:
Fix coverity items 104 and 105. Added calls to H5HF_sect_indirect_free to
H5HF_sect_indirect_reduce and H5HF_sect_indirect_reduce_row if there is an errorbefore "peer_sect" is linked into the main free space structure via its direct
sections. Also delayed call to H5HF_sect_indirect_first to prevent peer_sect
from being left in an inconsistent state.
r19268:
Added fix to disallow extendible compact dataset.
This was the same check as in H5D_contig_construct() in H5Dcontig.c.
Added test to verify the creation of extendible dataset with various layouts.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(h5committested on branch)
Whack a few more memory errors exposed by valgrind.
Make the debugging dump output a little bit prettier.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug
(h5committest forthcoming)
H5T_ORDER_MIXED specifically for compound types and its derived types. They report mixed
orders if they have members of different orders.
There is no change to configure.in, config, and Makefile.am in the top directory. They have
some property changes when I merged the code from 1.8.
Tested on jam. I tested the same change for 1.8 with h5committest.
More fixes to memory allocation, etc. exposed by valgrind.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.4 (amazon) in debug mode
Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
Close out various resource leaks and usages of uninitialized memory that
were flagged by valgrind. [There's still some more valgrind warnings, but
it's better now... :-/ ]
Also clean up warnings and code formatting.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & valgrind
(h5committest forthcoming)
1. For enum type, members shouldn't be defined yet.
2. H5T_ORDER_NONE only works for reference and fixed-length
string.
3. For opaque type, the order will be ignored.
4. For compound type, all restrictions above apply to the
members.
I'll change H5Tget_order and do another commit.
There is no change to configure.in, config, and Makefile.am. There is some property change for
these files when I did a merge from 1.8.
Tested on jam. But I tested the 1.8 on heiwa, and amani.
Add in override for testing timeout alarm to allow "HDF5_ALARM_SECONDS"
environment variable to change the timeout value from the default value.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(Too minor to require h5committest)
Clean up misc. compiler warnings.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
Description:
honest3 v1.8 failed in parallel test. It got stuck in the same
testpar/testphdf5 subtest (cbhsssdrpio). This is an old problem.
Upon closer inspection, the testphdf5, when terminated, had clocked
up 1hr 9min 46 sec wall clock time. Honest1 system also sent a message
that an mpi process has used up 30+ CPU minutes which exceeded their login
node cpu time limit and they killed the process. I also did a hand-run
of testphdf5. All subtests before cbhsssdrpio completed in a few minutes.
Therefore, it is safe to say the majority of the 70 minutes of wall clock
time are spent in the sub-test cbhsssdrpio. It also used up lots of CPU
time. cbhsssdrpio is likely infinite looping.
Since MPI application is prone to infinite looping due to message deadlock,
the testphdf5 has a built-in protection to give each subtest at most 20 minutes
of wall-clock time to run. When the 20 minutes wall-clock time is exceeded,
the testphdf5 will attempt to terminate itself. This prevents unnecessary
CPU time consumption in infinite looping.
But that clock limit was changed to 30 and then 60 minutes. I should have
but failed to, noticed the change mentioned by Quincey. IMO, 20 wall clock
time is more than sufficient for each subtest of testphdf5 to complete.
If a subtest takes longer than 20 minutes, it is likely infinite looping.
Giving it more time will not help.
If a subtest of testphdf5 takes more than 20 minutes, it should be broken
down to small tests that will finish way under 20 minutes so that it is
much easier to see progress and identify any deadlock problems.
In view of this, I am changing the testphdf5 time limit back to 20 minutes.
This will at least stop the CPU TIME exceeding limits and annoying the
system administrators.
Maybe there could be a provision, such as environment variable like
$HDF5_ALARM_SECOND to modify the alarm duration on individual execution.
Even so, that should be used temporary to see if an execution just needs
a little more time.
Tested: just eyeballed as the change is trivia.
herr_t. To minimize the change of the library's behavior, in the function
H5Z_prelude_callback of H5Z.c, if the return value of can_apply is FALSE and
the filter is MANDATE, this function returns a FAILURE. If the return value is FALSE
but the filter is OPTIONAL, this function returns a SUCCEED. During the IO, the filter
will fail and return a size of zero. But the pipeline will skip this filter.
Tested the same change for 1.8 on jam, linew, and amani. Tested on jam with szip.
Description:
test/big incorrectly determined not able to write files larger than 2GB and
skipped the SEC2 and STDIO driver tests. The reason was because it was using
off_t while the SEC2 driver is using lseek64 which expects off64_t type.
Solution:
Created a new HDoff_t which is set to off_t or off64_t or other appropriate
type depending on which of lseek or lseek64 is available. Changed SEC2 file
driver and the big test to use this common definition.
Tested:
In BP (AIX), using --enable and --disable-largefile, for both 32 and 64 bits
modes. Did not do h5committest because: 1. the error was exposed in the remote
BP machine; 2. the change is trivial.
Note that STDIO driver failed when --disable-largefile is used. That is an
error in the STDIO driver code that is being fixed.
Bring revisions from Coverity branch back to trunk:
r19044:
Coverity #449 - Line 1560 called function H5O_chunk_protect for 2 pointers to allocate. But when there's failure on the second one, the first wasn't freed
(H5O_chunk_unprotect). We fixed it by freeing the pointers when an error happens.
r19045:
Fixed coverity issue # 319. Free sec_node in done if it is not NULL.
r19046:
Add intended but missing assignments to initialize pointers to NULL (coverity issue fixes).
r19049:
Hdf5_1_8_coverity branch was recreated from hdf5_1_8 branch in revision 18839 without fix for Coverity issue #84 having been propagated to the hdf5_1_8 branch. This revision adds the fix again.
r19060:
added parentheses to see if they will keep subversion from getting confused
r19061:
Fix coverity item 139. Fixed incorrect condition for freeing buffer on error.
Fix coverity items 20 and 21. Removed unused NTESTS facility from dtypes.c.
Cleanup in H5Shyper.c.
r19062:
Fix coverity item 450. Check to see if chk_proxy has been allocated before
attempting to free it.
Fix coverity item 454. Check to see if allocation of buf failed in
H5D_fill_refill_vl.
Fix coverity items 455-457. Initilize hid_t's to -1, check their value before
attempting to close them, and check if the close failed.
r19063:
New fix to address coverity issue #84. Check that pointers in H5Z_xform_find_type are not NULL before passing them to H5T_cmp.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(Too minor to require h5committest)
Correct traversal of user-defined links (including external links) to
retain path information of object, allowing H5Iget_name() queries to work
quickly (without searching entire destination file). This required some
refactoring and addition of a mechanism to detect if a "fast" query was
performed (for the tests).
Minor code cleanups, etc.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.4 (amazon) in debug mode
Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
Rename H5AC_set() to H5AC_insert_entry()
Get rid of H5C_set_skip_flags() & related flags
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug, production & parallel
(too simple to require h5committest)
Fix const pointer issues for projection construction routine and also
bump time before alarm kicks in to terminate a test from 20 minutes to 30
minutes, to give the PGI compiler tests w/debugging enabled a chance to finish.
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug
Linux/32 2.6.18 (jam) w/PGI & debug
Bring "shape same" changes from LBL branch to trunk. These changes
allow shapes that are the same, but projected into dataspaces with different
ranks to be detected correctly, and also contains code to project a dataspace
into greater/lesser number of dimensions, so the I/O can proceed in a faster
way.
These changes also contain several bug fixes and _lots_ of code
cleanups to the MPI datatype creation code.
Many other misc. code cleanup are included as well...
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
A bug introduced in 1.8.5 causes local heap data blocks to be mis-aligned when
sizeof_offsets + 2*sizeof_lengths is not a multiple of 8. In this case, the
address of the data block as stored in the heap prefix is aligned but the actual
data block is not. This causes files created with these sizes to be corrupted,
and prevents uncorrupted files with these sizes to be unreadable. Modified
local heap code to account for alignment.
Tested: jam, amani, linew (h5committest)
Minor code cleanup
Description:
Add missing declaration of H5AC_ignore_tags()
to H5ACprivate.h
Clean up some warnings in cache_tagging.c
test file
Tested:
jam, gandalf
Fix some tagging issues.
Description:
- changed H5Gcreate to H5Gcreate2 in the cache_tagging.c
test file to work properly with 1.6 API default
mapping turned on.
- Added FUNC_ENTER/LEAVE_*TAG macros to the
H5G_stab_valid function.
Tested:
gandalf, jam, amani (w/ production mode, 1.6 API mapping)
Commit metadata tagging framework.
Description:
This check-in contains a new framework whose goal is to apply
a tag value to each new entry in the metadata cache as it is created.
This tag value is such that it relates each piece of metadata to the
HDF5 object that it belongs to (dataset, group, et cetera).
This changeset includes the framework that applies the tags as
well as a suite of tests to verify correct tag application, though does
not yet make use of the tag values to flush/evict individual objects.
Please refer to the "flush/evict individual objects" RFC for further
discussion of these changes.
Tested:
jam, amani, linew (h5committest)
liberty, abe, blue print
Problem: some launching command like yod (and sometimes mpirun), do not pass
back the exit code of the executables back to the shell. It always returns 0
if it can launch the executable. This cause tests that expect non-zero exit
code to fail incorrectly.
Short term Solution:
Added the feature that if $RUNSERIAL_NOEXITCODE is defined, do not
test conditions that rely on non-zero exit code.
Tests:
dove-linux using RUNSERIAL=thgyod to see the failures and using
$RUNSERIAL_NOEXITCODE=yes to bypass the errors.
Also h5committested.
This also affected the use of the #include <winsock.h> needed by gethostname. The new include: winsock2.h has been added to h5private.h and either removed or updated in the other files.
Tested: Windows
Bring r18911 (plus some adaptions to match the code on the trunk)
from the metadata journaling "merging" branch to the trunk:
More general changes to align trunk with eventual changes from
metadata journaling branch.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
fill values.
Description:
Previously, H5D_chunk_allocate would, if a dataset were filtered and had a non-default VL fill value, allocate a new buffer for every chunk. Changed
H5D_chunk_allocate to reuse the existing buffer if possible. Also other
miscellaneous cleanup.
Tested: jam, linew, amani (h5committest)
Description:
Previously, it was possible for a chunk to be flushed due to chunk operations in
the callback from H5B_iterate in H5D_chunk_prune_by_extent. Because flushing
the chunk can force it to be reallocated if it is filtered, this can change the
contents of the chunk b-tree in the middle of H5B_iterate. Because H5B_iterate
uses a locally cached copy of the b-tree, this causes subsequent operations
to be passed incorrect data. Rewrote H5D_chunk_prune_by_extent to avoid
H5B_iterate entirely.
Also fixed a bug in the dataset expand algorithm that could cause extra chunks
to be created.
Tested: jam, linew, smirom (h5committest)
Remove the size parameter from H5[A]C_unprotect() and the
H5[A]C__SIZE_CHANGED_FLAG, switching all clients & tests over to using
H5[A]C_resize_entry appropriately.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring r18738 from metadata journaling 'merging' branch to trunk:
Switch H5AC_resize_pinned_entry() to work on protected entries as well,
and rename to H5AC_resize_entry()
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring r18724 from metadata journaling 'merging' branch to trunk:
Rename H5[A]C_rename to H5[A]C_move_entry.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring r18720 from metadata journaling merging branch to trunk:
Bring changes from metadata journaling branch to 'merging' branch:
Rename H5[A]C_mark_pinned_or_protected_entry_dirty() to
H5[A]C_mark_entry_dirty() and get rid of H5[A]C_mark_pinned_entry_dirty().
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Improve configure's large-file support control.
Description:
Modified configure to now attempt to add defines necessary for
supporting largefiles on all systems, instead of solely on linux. This
is in response to user requests to enable largefile support on Solaris
by default, as well as to give extra control on AIX (instead of just
jamming the necessary flag into the config files).
The old --enable-linux-lfs flag has been removed in favor of the
--enable-largefile flag (enabled by default), which can be used on all
platforms.
On systems where large files cannot be supported in this manner,
configure will report as such.
Tested:
h5committest
AIX (NCSA's blue_print machine)
duty, liberty, and linew.
Bring r18672 from metadata journaling "merging" branch to trunk:
Mostly changes to move to only using one 'user data' parameter for
calls to H5AC_protect(), along with some minor reformatting code cleanups.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Add a few H5Tequal() calls to make certain that the native datatype
constructed with H5Tget_native_type() is identical to the memory datatype
created from the struct in memory.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.3 (amazon) in debug mode
Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
size of nested compound type are calculated by using H5T_cmp_offset. The old way had a bug
in it (see bug #1850).
Tested on jam. I tested the same change for 1.8 on amani, linew, and jam.
