Clean up warnings, enable new compiler warning flag(s) and bring back
changes from Coverity branch:
r20813:
Remove the dead code as listed for coverity bug #1722. h5committested.
r20814:
Issue 69: Check return value and throw error if negative return. Also free datatype id on error
r20815:
Use HDstrncpy. --gh
r20816:
Replaced one last HDstrcat call with HDstrncat to resolve coverity issue 832.
r20817:
Use HDstrncpy and HDstrncat. --gh
r20818:
Purpose: Fix valgrind issues with h5jam
Description:
Modified h5jam to free strings strdup'd in parse_command_line before exit. Note
that they may still not be freed in case of error, due to the widespread use of
exit().
r20819:
Issue 80: change loop to use int as loop index.
r20820:
Maintenance: Fixed the bug found by coverity CID 788
There were two problems with this function:
1) it tried to unnecessary free NULL pointer
2) it tried to allocate c_name buffer that is done by H5Pget_class_name
Tested on:
Mac OSX 10.8.4 (amazon) w/gcc 4.8.1, C++ & FORTRAN
(too minor to require h5committest)
Found and removed two more "//" comments.
Tested: in Koala by
$ env CFLAGS="-ansi" ../hdf5/bin/buildhdf5 -fortran -cxx -config # enable fortran and cxx, configure only
$ make AM_CFLAGS=-ansi # compile all source file by setting HDF5 build CFLAGS as "-ansi" only
Clean up warnings in H5Tconv.c (down to _only_ 9000 lines of output now!)
Merge changes from Coverity branch back to trunk:
r20684:
Fix for coverity bug #1721 which was due to the fix for coverity bug #943.
r20685:
Use HDstrncpy. --gh
r20761:
Purpose: Fix valgrind issues
Description:
Free image_data and data as appropriate in test_image.
r20762:
Purpose: Fix coverity issue 600
Description:
Add check for return value of H5O_close in H5Ocopy. Also cleaned up various
warnings.
r20763:
Purpose: Fix valgrind issues with h5stat
Description:
Modified h5stat to free "iter" before exit, and free "hand" before exit if
parse_command_line exits directly.
r20764:
fixed coverity issues:
69, 327, 614, 684, 685, 696, 697, 1681, 967, 826, 660, 80
r20765:
Fixed coverity bug 668
Pulled x * y * z multiply out of malloc operand into a separate n_elements variable to quiet integer overflow warning.
No actual integer overflow tests are performed since it's just a test program but I did add a check that n_elements is >= 1.
I also changed an error condition from doing its own close and returning -1 to "goto out;" like the rest of the program.
r20766:
Fixed coverity bug 667
Pulled x * y * z multiply out of malloc operand into a separate n_elements variable to quiet integer overflow warning.
No actual integer overflow tests are performed since it's just a test program.
Tested on:
Mac OSX/64 10.8.3 (amazon) w/C++ & FORTRAN
(h5committest upcoming)
HDFFV-7643 - h5diff - incorrect exit-code returns for extra attribute exist
Description:
h5diff: Fixed to return correct exit code 1 when detect unique
attribute. Prior to this fix, h5diff returned exit code 0 indicating
two files are identical.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), fred (mac64-LE), emu (solaris-BE), Windows (32-LE cmake), cmake (jam)
Bring reviewed changes from Coverity branch back to trunk (QK & JK):
r20457:
Coverity issue 691: return of H5duo could be negative. Fixed by using
STDOUT_FILENO and redesign parse_command_line and main to cleanup file
allocations. The output_file var is null when using stdout. In cleanup do not
close output_file if NULL.
r20510:
Initialize ufid = -1 and predicate HDclose call on ufid != -1
r20511:
Purpose: Fix coverity issue 1715
Description: Free "file" and nested data on failure in H5FD_core_open.
r20512:
Initialize ifid = -1 and predicate HDclose call on ifid != -1
r20514:
Initialize h5fid = -1 and predicate HDclose call on h5fid != -1
r20516:
Added else branch to the if (ret_value < 0) check.
r20522:
Addressed coverity issues 930-933, 850, 836, 835, 1307. All minor
potential buffer overwrite bugs, or coverity errors. Fixed by replacing
strcpy and sprintf with strncpy and snprintf.
r20523:
fixed coverity issues 68, 1120, 1116i
r20524:
Check H5Z_SZIP->encoder_present < 1 assuming 0 represents absence.
r20601:
Purpose: Fix coverity issues 1703-1705
Description: Modified the cleanup code in test_free in accum.c to reset
allocated buffers to NULL after they are freed, and modified the error cleanup
code to check if these buffers are NULL before freeing them. Also fixed some
unrelated warnings in accum.c.
r20602:
Use HDsnprintf and HDstrncat
r20603:
Purpose: Fix coverity issues 808-809
Description: Modified test_core in vfd.c to check the returns from malloc, and
keep track of whether points and check are allocated by setting them to NULL
when they are not. Added code to free points and check on error if they are
not NULL. Also fixed unrelated warnings in vfd.c.
r20604:
Use HDstrncpy.
r20605:
Use HDstrncpy and HDstrncat.
r20606:
Purpose: Fix coverity issue 807
Description: Modified long_compact in stab.c to keep track of whether objname is
allocated by setting it to NULL when it is not. Added code to free objname on
error if it is not NULL.
r20607:
Changed string function calls to use versions that specify the string length
to fix coverity issues 832 and 839.
Tested on:
Mac OSX/64 10.8.2 (amazon)
(Too minor to require h5committest)
Clean up warnings and bring changes from Coverity branch back to trunk:
r20451:
Fixed coverity 810
r20454:
Purpose: Fix coverity issue 816
Description: Modified various file drivers to always check the return value from
H5P_get_driver_info.
r20455:
Purpose: Cleanup fix for coverity issue 816
Description: Removed redundant check for "fa" in H5FD_core_open.
Tested on:
Mac OSX/64 10.8.2 (amazon) w/debug
(Too minor to require h5committest)
HDFFV-8145 h5diff slowness with version >= 1.8.7
Description:
Improved speed when comparing HDF5 files with lots of attributes. Much slower performance was identified with release version from 1.8.7 to 1.8.10 compared to 1.8.6.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), Windows (32-LE cmake), cmake (jam)
Review Coverity changes and bring them back to trunk. (QK & JK)
r20402:
Added #includes for h5tools.y or h5tools_utils.h as required to remedy implicit
function declarations which caused compiler warnings and coverity issues 703-4
and 708-11.
r20414:
This is related to the previous checkin r20399. There were incorrect updates
which caused incorrect behavior when no file was given. Also possible segfault
when handling hyperslab options. Simplify the code changes.
r20449:
Description: Modified H5E_walk2_cb to check return value of H5I_object_verify.
r20450:
fixed coverity 813
Tested on:
Mac OSX/64 10.8.2 (amazon) w/C++, FORTRAN, debug & threadsafe
(too minor to require h5committest)
HDFFV-5919 - GMQS: h5diff - The number of object difference is not consistent between dataset and group/type when attribute(s) have differences
Description:
Object differences are not consistent between dataset and group/datatype when their attribute(s) have differences. This is because attribute(s) differences is not accumulated to group or datatype object’s difference, but accumulated to dataset difference.
To fix, do not accumulate attribute difference to dataset difference. This is referred to h5diff’s default behavior and also past report from users that users were confused by the accumulated behavior. (also can’t figure out only for dataset difference , also hard to spot dataset difference when it has lots of attributes or differences)
This also lead to fix inconsistent format indicating difference between dataset and group/datatype object.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
Address HDFFV-7942 - h5diff: incorrect result for comparing attribute data with different type size in same class
Description:
When comparing attribute data values with same type class but different size, the result was incorrect. It was due to the size difference and got truncated.
Fixed to match up the smaller type size to big type size like what dataset does.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
Purpose:
HDFFV-8003 - ph5diff (parallel h5diff): daily test failure on ember intermittently during non comparable test file comparison
HDFFV-7755 - parallel h5diff : hanging on koala intermittently during non comparable test file comparison
Description:
non-comparable test intermittently hung on koala and ember, but not on jam. it didn't occur until -np reaches 4 or bigger. it occurred once out of many repeated attempts of the same test.
There was a incorrectly (mistakenly?) duplicated code in MPI section which caused such hang in a certain condition. The test used more processes than other tests, which increased chance to trigger more undone processes, and such process could enter the incorrect code section and wait for wrong pair of send. it explains why it occurred intermittently according to machine condition and using a certain feature.
Removed incorrect code which blocked correct code.
Tested: some manually repeated test performed
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE)
Create setters for indexing options in h5trav.c to be set before the obj table is created.
Revert h5trav_print to remove now duplicated parameters.
Added test and testfiles.
Tested: local linux with cmake
Added static local variables in h5trav to hold the index_by and index_order values. Defaults are set.
H5dump only use of the trav_print function, added two parameters and passed in the sort_by and sort_order variables.
Tested: local linux
Fix for HDFFV-7998-h5diff: incorrect behavior and output for comparing symbolic dangling links without following-symlinks option
Description:
Fix not to check and display dangling link status without --follow-symlinks option.
Berfor the fix, when comparing symbolic link to symbolic link without follow-symlinks option, h5diff followed to check if those links are dangling or not. It caused to display output incorrectly (dangling link instead of symbolic link).
The fix also improved performance when comparing lots of symbolic links without the --follow-symlinks option.
Test cases were added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward
the final design (as exemplified by the H5EA & H5FA code).
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & parallel
Fix for HDFFV-7644 GMQS: h5diff - incorrect behaviors when comparing HDF5 files with different type of objects (dataset, group, type) as common objects
Description:
Fixed failure for comparing same named object with different object types in comparing groups. (common object comparison)
Prior to the fix, h5diff resulted in error. After the fix, h5diff detects such case as non-comparable and display messages accordingly.
Test cases were added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
Fix for HDFFV-7837 - h5diff: incorrect behavior with exclude-path option when unique object exist only in one file
Description:
If unique objects exists only in one file and try to exclude
the unique objects with --exclude-path option, h5diff missed
excluding some objects.
Fixed to exclude objects correctly in such case.
Test cases were added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
by using an escape character. For example, for a dataset "/dset"
containing attribute "speed(m/h)", use "h5dump -a "/dset/speed(\/h)"
to show the content of the attribute. See details at HDFFV-7523
Fix for HDFFV-7835 h5diff: incorrect result for comparing the two same type symlinks as dangling links.
Description:
When two symbolic dangling links are compared with --follow-symlinks option,
the result should be same. It works for comparing two files, but didn't work
for comparing two objects.
Test cases were added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
Task for HDFFV-7862 - Select data by chunk direction to improve performance in h5repack
Description:
h5repack sometimes became very slow when handling big chunked datasets in
certain cases. (when chunk boundary doesn't match with a hyperslab boundary.)
The main issue was from figuring out a hypeslab without considering chunk
boundary to read from and write to such datasets.
The update was made to figure out a better hyperslab unit with considering
chunk boundary to improve performance for such cases prior to the update.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows
Fix for HDFFV-7836 h5diff: displays error stack message for comparing the two dangling symlink with follow-symlinks option
Description:
While ago, to improve performance, skipping same object checking
(h5tools_is_obj_same()) was added.
However the checking function doesn't understand about the dangling link and
caused the issue.
Since handling dangling link code section already implemented, move the
checking function after handling dangling-links to address the problem.
Test was added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
HDFFV-7770 - h5diff segmentation fault on JPSS files
Description:
Fixed segfault over non-comparable attribute with different dimention or
rank, along with '-c' option to display details.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows, Cmake
Description:
Added new H5SL_TYPE_GENERIC skip list type, which uses void *'s as keys and a
client-supplied callback for key comparison. This was added to support the
upcoming "merge named datatype" feature for H5Ocopy, but may be used in other
places as well. Also added testing.
Also fixed a potential bug with the H5SL_TYPE_OBJ implementation, and added
testing for that.
Tested: jam, koala, heiwa (h5committest), durandal
Added a funtion to reset dataset & hyperslab buffer size for h5repack from an
environment variable.
This is performance debugging purpose for now.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), Windows (32-LE), cmake
HDFFV-7693 - h5diff produces different behavior between versions: 1.8.3 and 1.8.6
Description:
There were two issues on this.
One was not displaying all the comparable objects if non-comparable
object/attribute exist and compared before comparables ones. This issue
occurred after 1.8.4 release. This is the issue from user.
The other issue was not displaying all the comparable attributes,
if non-comparable object/attribute exist and compared ahead. This issue
were exist even before 1.8.4 release. So it's possible some attribute
comparison may have not displayed correctly in the past, if non-comparable
data were exist in common object.
Fixed h5diff to display all the comparable object and attribute regardless
of non-comparables.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Cmake (jam, Windows)
tempint = (tempint >> packed_data_offset) & packed_data_mask;
The right shift may cause undefined behavior if PACKED_DATA_OFFSET is
32-bit or more. For every kind of native integers, I changed the code
to make it zero if PACKED_DATA_OFFSET is greater than or equal to the
size of integer.
Tested on Mac OS Lion with CLANG compiler and jam, koala, and linew.
