This occurs due to the istore test creating very large files on systems which
do not have POSIX-like sparse file semantics. The large amount of I/O causes
the test to run for a very long period of time.
The fix was to copy the "big" test's sparse file check and only run the
largest sparse file test when POSIX-like sparse file semantics are found.
Tested on:
jam (nfs)
ostrich (nfs)
loyalty (ufs)
64-bit linux VM (ext4)
64-bit OS-X Lion (hfs, detected as not POSIX sparse)
64-bit Windows 7 (NTFS, detected as not POSIX sparse)
The OS-X failure to pass the sparse check is likely an error and will be
entered as a new bug.
The parser mistaked "-" as substraction. I fixed it and also fixed another problem
with some special cases like 100-x and 2/x.
Tested on jam, koala, and ostrich.
Description:
When using the new object header format, it was possible for corruption to occur
if the first object header chunk changed size such that the lenght of the "chunk
0 size" field changed. This only occurred if there were messages that had not
been decoded. The original algorithm that changed the object header chunk size
marked all messages as dirty, causing those that had not been decoded to have
both the raw and native form invalidated. Changed the algorithm to avoid
marking messages dirty and added assertions to catch the case where messages
are dirtied without being decoded (or recently created) first.
Tested: jam, koala, ostrich (h5committest), durandal
Description:
When using the new object header format and adding an attribute with a size near
64K, it was possible for file corruption to occur. This happened only if the
first object header chunk was smaller than 256 bytes and then grew to larger
than 64K after the attribute was added.
Tested: ostrich, jam, koala (h5committest), durandal
Correct corner case for creating a contiguous dataset with a zero-sized
dataspace, when the allocation time is set to early.
Also clean up a few compiler warnings in the dataspace code.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & parallel
Remove some leftover uses of the __FUNCTION__ macro, replacing them with
FUNC macro, as used everywhere else.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
(too minor to require h5committest)
Add FUNC_ENTER macros for package-private routines and begin process of
switching package routines to use them. All H5G routines are currently
finished.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
Description:
When shrinking a chunked dataset, the library fills in the unused parts of
chunks that have been shrunk. The fill value buffer allocated for this purpose
had a maximum size of 1 MB, but the fill was performed in a single operation.
Therefore, if the amount of unused space in a chunk after being shrunk was
greater than 1 MB, the library would read off the end of the fill value buffer.
Changed the maximum fill buffer size to be equal to the chunk size.
Tested: durandal; jam, koala, heiwa (h5committest)
Don't check dataset storage size for compressed datasets with region
reference datatypes. (The address of the region reference type in the file
varies and affects the compressed size)
Tested on:
Mac OS X/32 10.7.2 (amazon) w/debug & production + check-vfd
Recalculate the size of destination attribute message when the source and
destination versions are different during an object copy operation.
(Jira: HDFF-7718)
Tested on:
Mac OS X/32 10.7.2 (amazon) w/debug
(h5committest upcoming)
Description:
Added new H5SL_TYPE_GENERIC skip list type, which uses void *'s as keys and a
client-supplied callback for key comparison. This was added to support the
upcoming "merge named datatype" feature for H5Ocopy, but may be used in other
places as well. Also added testing.
Also fixed a potential bug with the H5SL_TYPE_OBJ implementation, and added
testing for that.
Tested: jam, koala, heiwa (h5committest), durandal
Correct error in loading local heap prefix & data block from the file.
Sometimes the local heap's prefix could be loaded before the data block (e.g.
using H5Oget_info), but then when the data block was loaded later, the free
list information would get lost, causing the heap's size to grow larger than
necessary. This is Jira bug #HDFFV-7767
Tested on:
Mac OS X/32 10.7.2 (amazon) w/debug
(h5committest coming up)
Description:
H5Ocopy could get confused when copying a named datatype containing an
attribute which used that named datatype as its datatype. This happened
because H5Ocopy would recurse into the attribute's datatype before the object
the attribute was in was fully copied (i.e. before the "post-copy" pass).
Modified H5Ocopy to avoid recursing before the post-copy step in this case.
Required many changes, including to how non-committed shared messages are
copied.
Tested: jam, koala, heiwa (h5committest); durandal
Some machines, like LLNL udawn, a blue-gene machine, requires all executables,
be launched by some command like mpirun.
Solution:
Added $RUNSERIAL to launch the executable.
Tested: LLNL uDawn.
them is in the INIT_INTEGER macro definition in dt_arith.c. It complained about line 150 where it tried to subtract 1 from
the negative minimal value of "int", causing it to overflow (or underflow). So I revised the code to avoid it.
Tested on jam, koala, linew, and Mac OS Lion with CLANG compiler.
temp_point->l = (unsigned long long)((i * 100 + j * 1000) * n);
The value can overflow the signed int before being converted to unsigned long long. So I changed it to
temp_point->l = (unsigned long long)((i * 40 + j * 400) * n);
to keep it under the maximal value.
Tested on jam. Simple change.
Ran bin/reconfigure to update the Makefile.in in directories not part of the fortran directory check=in. Updates Makefile.in due to changes made in configure.in for the Fortran 2003 additions.
Tested on all platforms run under daily tests.
i_mask = ~((unsigned)~0 << (precision[0] + offset[0])) & ((unsigned)~0 << offset[0]);
in line 3680 left shifted a 32-bit integer for 32-bit. The result is undefined by C language. A user
discovered it using clang compiler with -fcatch-undefined-behavior option (see Issue 7674
in Jira). So I changed it in a funny way to avoid it.
Tested on jam, koala, and heiwa.
Further patches for Jira 7638, to ensure that H5D/Aget_type will
be attached to the correct top file. (This change to the library should
reverted when Jira 7638 is finally finished)
Tested on:
FreeBSD/32 8.2 (loyalty) w/gcc4.6, w/C++ & FORTRAN, in debug mode
FreeBSD/64 8.2 (freedom) w/gcc4.6, w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (koala) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Mac OS X/32 10.6.8 (amazon) in debug mode
Mac OS X/32 10.6.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
1. Quincey changed H5P_DATASET_ACCESS_DEFAULT to H5P_DEFAULT for the tests that call H5Rdereference. Inside H5R_dereference, I assigned H5P_DATASET_ACCESS_DEFAULT to OAPL_ID if it is H5P_DEFAULT.
2. I added the test for invalid OAPL_ID in trefer.c.
Tested on jam, koala, heiwa.
Switch from H5P_DATASET_ACCESS_DEFAULT to H5P_DEFAULT for calls to
H5Rdereference2().
Tested on:
Mac OS X/32 10.6.8 (amazon) w/debug
(too minor to require h5committest)
Check in ugly version of fix for Jira #7638. I'm going to leave the issue
open in Jira and when the library has been refactored to use shared file
pointers (instead of top file pointers) for file operations, I'll uncomment
the check in src/H5Oshared.c
Tested on:
Mac OS X/32 10.6.8 (amazon) w/debug
(too small for h5committest)
super block. We took it out in v1.8 library because it's meaningless
for the MULTI file. v1.8 library saves the EOA for the metadata file,
instead. But this caused some backward compatibility problem.
v1.8 library couldn't open the file created with v1.6 library. I
fixed the problem by checking the EOA value to detect the file
created with v1.6 library.
Tested on jam, koala, and heiwa.
data, overflowing values are retained. When conversion happens, the values become -1. The conversion function puts -1
when overflow happens. I added two new dataset transfer property to control whether to fill 0xff in the destination
data or convert to the destination data when overflow happens. The two new functions are H5Pset(get)_enum_conv_overflow. I also added test cases in enum.c and dtypes.c.
Tested on jam, koala, and heiwa.
Description:
When closing an object that had two open id's, the library would previously fail
to account for the status of the "holding_file" field, potentially causing the
file's "nopen_objs" field to become inconsistent. This caused problems when
opening the same object twice through an external link using th external file
cache. Modified the library to properly account for the status of the
"holding_file" field when closing an object id, even when the underlying
"shared" object isn't closed.
Tested: jam, koala, heiwa (h5committest)
The problem was that the test was a clone of the SEC2 test. The SEC2 test accepts a file descriptor (int) via a void pointer from H5Fget_vfd_handle() whereas the STDIO test needs to accept a FILE pointer. Storing the FILE pointer in an integer failed on some systems. The STDIO test now stores the file handle in FILE pointer and checks it for NULL instead of < 0.
Clean up various warnings & code formatting issues.
Bring changes from Coverity branch to trunk:
r20085:
Purpose: Fix coverity issue 793
Description: Modified H5S_hyper_project_simple_higher() to free the entire span
list in new_space on failure.
r20091:
This is a fix for coverity bug #1683.
Changed the two printfs to use %lu (unsigned long) for printing "dset_size".
r20162:
Purpose: Fix coverity issue 785
Description: Modified H5T_enum_nameof() to free "name" on failure if it was
allocated. Also clarified some code in H5S_hyper_rebuild_helper().
r20189:
Addressed coverity defect 783.
H5SL_new_node() in H5SL.c was failing to free space allocated in its
first alloc if the second alloc failed. Added a call to H5FL_FREE
to address this issue.
This is purely to keep coverity happy -- if this code is ever triggered,
we have much larger problems.
Note that this fix will trigger an unused return value complaint
from coverity next week.
r20190:
Fixed Coverity issues 1561 1565 and 1678 (UNUSED_VALUES) by moving checks of return values to after the function call.
r20191:
Fixed coverity issues 643 644 and 1678 (CHECKED_RETURN).
r20232:
Addressed coverity issues 923-925. Replaced calls to sprintf with calls
to HDsnprintf.
r20233:
Fix coverity issue 662. Don't try to sort 0 attributes in H5Aint.c.
r20234:
Fix coverity issue 664. Check for NULL before dereferencing in H5Gdeprec.c.
r20271:
Purpose: Fix coverity issue 784
Description: Modified H5_debug_mask() to keep a list of files opened for use as
a debugging output stream, and modified H5_term_library to close these files on
exit.
r20272:
addressed coverity issues 838 & 955. Issue was use of strcpy() -- existing
code was safe, but modified to use strncpy() to keep coverity happy.
r20273:
Addresed coverity issues 1388 and 1389.
Initialized sel_iter->type to NULL in H5S_select_iter_init.
r20275:
Purpose: Fix valgrind issue in mf.c
Description: Fixed bug (incomplete if statement) in test_mf_fs_alloc_free() so
the retrieved node gets freed.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Add explicit test that checks that the 'fileno' field in H5O_info_t
is the same for objects in the same file, whether the file was opened twice
or not.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
Description:
Added internal tesitng routines to traverse a file and verify that symbol table
information is *always* cached, whenever possible. Added this check to the end
of many tests to check all the test files, right before the call to h5_cleanup.
Tested: jam, amani, heiwa (h5committest)
Correct several problems with compound datatypes that don't have any
fields added:
- Change assertion to error report when a file is encountered which has this
situation.
- Added check to attribute creation to avoid creating attributes with a
datatype like this (datasets and named datatypes already have the check)
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
General shared library improvements for CYGWIN / AIX
Description:
Shared libraries are disabled on both CYGWIN and AIX due
to inability to build them correctly. Part of the problem
in both of these situations is the lack of the libtool
flag -no-undefined, which tells libtool that all needed
symbols are defined at link time (a requirement on these
systems) and that it's okay to build shared libraries.
Another problem are lack of dependencies between wrapper
libraries and core C HDF5 library.
This patch addresses both of these by fixing configure to
add in -no-undefined flag for libtool during linking and
adds automake dependencies in the Makefile.am files.
After testing, both CYGWIN and AIX now generate shared
libraries, but there are still some test failures in each.
(cache_api, dt_arith, and testerror.sh on CYGWIN, and
fortran tests on AIX).
Even though the shared libraries are not quite perfect,
this is a general improvement to what we had before, so
I'm applying the patch anyways. Note that default behavior
of shared libraries on these systems being disabled has
NOT been changed and requires the use of the
--enable-unsupported to attempt to build them.
We will need to address the test failures in each
architecture prior to formally supporting shared
libraries on each.
Tested:
h5committested & CYGWIN tested (on bangan)
(AIX tested by Albert on bp-login2)
Add "silent make" mode configure option.
Description:
Automake 1.11 has a new option available that allows for a
silent make mode. This functionality needs to be explicitly
enabled in configure.in via the use of the automake macro
AM_SILENT_RULES, which is what this commit is adding.
This introduces a new configure option:
--{en|dis}able-silent-rules
This option is on by default, and simplies compile and link
line outputs when building the library. Disabling this option
will print full "verbose" output (i.e., full compile and
linking lines for each target).
Tested:
This was tested on jam & h5committested
forbid setting the dimension size bigger than existing maximal size. In this checkin, I restored it to the previous
behavior that any change will wipe out previous record of dimensionality.
Tested on jam - a simple change.
r20440 and r20469:
1. The dataspace code has another bug - when the maximal dimension isn't passed in for H5Sset_extent_simple, it
is supposed to be same as the dimension. The current library sets NULL to it. I corrected it and added a
test case to it.
2. I corrected the tests of Fortran and C++ for this problem.
Tested on heiwa, jam, and amani.
Remove global variable for the path name in the group traversal code. Also
clean up a bunch of compiler warnings.
Tested on:
Mac OS X/32 10.6.7 (amazon) w/debug & production
FreeBSD/32 6.3 (duty) w/debug
FreeBSD/64 6.3 (liberty) w/debug
Solaris/32 2.7 (linew) w/debug
- Revise shared Fortran library disabling scenarios in configure
- Improve configure output summary
Description:
Shared Fortran libraries are not supported on Mac, but were being
disabled by configure in a way that also forced the C libraries
to be static-only. This has been fixed, so now only shared Fortran
is disabled while shared C can remain.
This prompted two additional changes:
1. While working on the check that addresses whether or not
shared Fortran libraries are allowed, removed old and no
longer needed check(s) that disable shared Fortran
libraries with HP, Intel 8, PGI, and Absoft compilers.
(Essentially, Mac is the only situation in which Fortran
shared are disabled by configure.)
2. Having two different states of libraries (i.e. shared C
library with static-only Fortran library) was not apparent
in the configure summary, which labeled all libraries as
either shared and/or static. I've added lines to both the
C++ and Fortran output sections to list shared/static-ness
of these libraries specifically.
Additionally, I've made sure that the new --enable-unsupported
configure option correctly overrides configure if it tries to
disable a shared library.
Tested:
jam, fred, & h5committest
1. I added a test case of extending dataset of zero dimension size and shrinking back to zero dimension size.
2. I updated the Makefile to include the new data file to be cleaned up.
Tested on jam - relatively simple.
Progress on bug 2138 (bugzilla) / HDFFV-4298 (JIRA)
Description:
- Added basic VFD tests (i.e. cloned the SEC2 test) for the Windows, STDIO and log VFDs.
- Added H5Pget_driver() sanity check to all VFD's tests.
- Added log output file to list of files to clean up in Makefile.am.
Tested on:
- h5committest script platforms (jam, amani, heiwa)
- Windows 7 (32- and 64-bit)
Description:
Implements a cache of files opened through external links. Adds the public
functions H5Pset_elink_file_cache_size(), H5Pget_elink_file_cache_size(), and
H5Frelease_file_cache().
Tested: jam, amani, heiwa (h5committest), fedora 64.
Bring changes from Coverity branch to trunk:
r19930:
Fix memory leaks involving VL attributes in h5repack and h5diff. The buffers in
copy_attr and diff_attr were not checked for the presence of a vlen before being
freed, and vlen storage was never reclaimed. Added checks and calls to
H5D_vlen_reclaim().
r19933:
Purpose: Fix memory leak in H5L_move_cb()
Description: H5L_move_cb copied the source link using H5O_msg_copy() but freed
it manually using H5MM_xfree(). Since H5O_link_copy allocates the link using
H5FL_MALLOC, this causes the link to be allocated from the free list but is
never put back on the free list when it is freed. This prevents the link free
list from shutting down properly. Modified H5L_move_cb() and H5L_move_dest_cb()
to free the link properly using H5O_msg_free().
r19973:
Fix resource leaks by freeing string created by HD5f2string
r19974:
Issue #345: Inialize buf variable to null
Tested on:
Mac OS X/32 10.6.6 (amazon) w/debug & production
(h5committested on Coverity branch)
Clean up Coverity warnings, and fix some style issues:
r19735:
Fix for memory leak in test/mf found by valgrind.
r19736:
Fix memory leak in h5repack. The buffer in copy_objects, when copying the
entire dataset at once, was not checked for the presence of a vlen, and vlen
storage was never reclaimed. Added check and call to H5D_vlen_reclaim().
r19772:
Change H5assert() to
if (H5T_VLEN != src->shared->type || H5T_VLEN != dst->shared->type)
HGOTO_ERROR(H5E_ARGS, H5E_BADTYPE, FAIL, "not a H5T_VLEN datatype")
r19774:
removed unused priv.
r19775:
removed unused variables
r19778:
Fix memory leak comparing for variable length data types.
r19834:
Fixed memory leaks found by valgrind. Memory errors remain for another day.
Tested on:
Mac OS X/32 10.6.6 (amazon) w/debug & production
(h5committested on branch)
Description:
When using the scale-offset filter with floating point data or fill values, big
endian machines would save some metadata in the wrong byte order. This caused
such datasets to yield incorrect data when read on little endian machines.
Fixed the scale-offset filter to always save this metadata in the right byte order (i.e. little endian).
Tested: jam, amani, heiwa (h5committedt); fedora, linew
