Revert back one of the coverity updates from previous checkin. "r20457: Coverity issue 691" in previous svn log. Only revert the change about using "STDOUT_FILENO".
Description:
Using "STDOUT_FILENO" caused error on Windows. On unix it's defined in <unistd.h>, but not in Windows. After a little discussion with Allen, he remembered how the similar case were handled for h5dump redirecting those streams. (in h5dump.c) However it would requires more attention to apply, so this issue is entered as a separate task in jira HDDFV-8306.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), fred (mac64-LE)
Bring reviewed changes from Coverity branch back to trunk (QK & JK):
r20457:
Coverity issue 691: return of H5duo could be negative. Fixed by using
STDOUT_FILENO and redesign parse_command_line and main to cleanup file
allocations. The output_file var is null when using stdout. In cleanup do not
close output_file if NULL.
r20510:
Initialize ufid = -1 and predicate HDclose call on ufid != -1
r20511:
Purpose: Fix coverity issue 1715
Description: Free "file" and nested data on failure in H5FD_core_open.
r20512:
Initialize ifid = -1 and predicate HDclose call on ifid != -1
r20514:
Initialize h5fid = -1 and predicate HDclose call on h5fid != -1
r20516:
Added else branch to the if (ret_value < 0) check.
r20522:
Addressed coverity issues 930-933, 850, 836, 835, 1307. All minor
potential buffer overwrite bugs, or coverity errors. Fixed by replacing
strcpy and sprintf with strncpy and snprintf.
r20523:
fixed coverity issues 68, 1120, 1116i
r20524:
Check H5Z_SZIP->encoder_present < 1 assuming 0 represents absence.
r20601:
Purpose: Fix coverity issues 1703-1705
Description: Modified the cleanup code in test_free in accum.c to reset
allocated buffers to NULL after they are freed, and modified the error cleanup
code to check if these buffers are NULL before freeing them. Also fixed some
unrelated warnings in accum.c.
r20602:
Use HDsnprintf and HDstrncat
r20603:
Purpose: Fix coverity issues 808-809
Description: Modified test_core in vfd.c to check the returns from malloc, and
keep track of whether points and check are allocated by setting them to NULL
when they are not. Added code to free points and check on error if they are
not NULL. Also fixed unrelated warnings in vfd.c.
r20604:
Use HDstrncpy.
r20605:
Use HDstrncpy and HDstrncat.
r20606:
Purpose: Fix coverity issue 807
Description: Modified long_compact in stab.c to keep track of whether objname is
allocated by setting it to NULL when it is not. Added code to free objname on
error if it is not NULL.
r20607:
Changed string function calls to use versions that specify the string length
to fix coverity issues 832 and 839.
Tested on:
Mac OSX/64 10.8.2 (amazon)
(Too minor to require h5committest)
testh5stat.sh fails in LLNL uDawn. The test actually ran as expected
but it did not prepare for the extra error messages from the MPI
process.
Solution:
This sort of problem has been fixed by using the output filters in
output_filter.sh. testh5stat.sh is updated to use the output filters.
Tested:
uDawn.
Clean up warnings and bring changes from Coverity branch back to trunk:
r20451:
Fixed coverity 810
r20454:
Purpose: Fix coverity issue 816
Description: Modified various file drivers to always check the return value from
H5P_get_driver_info.
r20455:
Purpose: Cleanup fix for coverity issue 816
Description: Removed redundant check for "fa" in H5FD_core_open.
Tested on:
Mac OSX/64 10.8.2 (amazon) w/debug
(Too minor to require h5committest)
HDFFV-8145 h5diff slowness with version >= 1.8.7
Description:
Improved speed when comparing HDF5 files with lots of attributes. Much slower performance was identified with release version from 1.8.7 to 1.8.10 compared to 1.8.6.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), Windows (32-LE cmake), cmake (jam)
Review Coverity changes and bring them back to trunk. (QK & JK)
r20402:
Added #includes for h5tools.y or h5tools_utils.h as required to remedy implicit
function declarations which caused compiler warnings and coverity issues 703-4
and 708-11.
r20414:
This is related to the previous checkin r20399. There were incorrect updates
which caused incorrect behavior when no file was given. Also possible segfault
when handling hyperslab options. Simplify the code changes.
r20449:
Description: Modified H5E_walk2_cb to check return value of H5I_object_verify.
r20450:
fixed coverity 813
Tested on:
Mac OSX/64 10.8.2 (amazon) w/C++, FORTRAN, debug & threadsafe
(too minor to require h5committest)
Bring Coverity changes back to trunk:
r20399:
Change parse_command_line to free hand structure on error and only "leave()" at end of main.
r20400:
Purpose: Fix coverity issue 592
Description: Added void cast to call to H5F_sfile_assert_num() in
H5F_term)interface(). There is no need to check the return value of this
function in general, and there would be nothing to do even if it did fail.
r20401:
Added free(src_sizes) and free(src_offset). --gh
Tested on:
Mac OSX/64 10.8.2 (amazon) w/C++, FORTRAN, threadsafe
(too minor for h5committest)
Larry mentioned that intel compiler was used on Mac for testing.
Allen requested comment it out.
"It is not scheduled for 1.8, I created the file for Java testing. Actually I
have to create windows versions because of the use of floating point." - Allen
HDFFV-8012 - h5repack changes max dims and cause failure if only "-f none" is used without changing layout for chunked dataset when a chunk dim is bigger than a dataset dim
Description:
"h5repack -f <obj>:NONE <file.h5> out.h5" command failed if source file contains chunked dataset and a chunk dim is bigger than a dataset dim.
Another issue is that the command changed max dims if chunk dim is smaller than the dataset dim.
These issue occurred when dataset size is smaller than 64k (compact size limit)
Fixed them.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
HDFFV-5919 - GMQS: h5diff - The number of object difference is not consistent between dataset and group/type when attribute(s) have differences
Description:
Object differences are not consistent between dataset and group/datatype when their attribute(s) have differences. This is because attribute(s) differences is not accumulated to group or datatype object’s difference, but accumulated to dataset difference.
To fix, do not accumulate attribute difference to dataset difference. This is referred to h5diff’s default behavior and also past report from users that users were confused by the accumulated behavior. (also can’t figure out only for dataset difference , also hard to spot dataset difference when it has lots of attributes or differences)
This also lead to fix inconsistent format indicating difference between dataset and group/datatype object.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
HDFFV-7942 - h5diff: incorrect result for comparing attribute data with different type size in same class
Description:
This is additional fix due to daily build fail against previous r22684.
Refer to the svn -r22684 log for previous details
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), cmake (jam)
Address HDFFV-7942 - h5diff: incorrect result for comparing attribute data with different type size in same class
Description:
When comparing attribute data values with same type class but different size, the result was incorrect. It was due to the size difference and got truncated.
Fixed to match up the smaller type size to big type size like what dataset does.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
Changes resulting from Klocwork static analysis tool, from Mark Miller
@ LLNL (miller86@llnl.gov).
Tested on:
Mac OS X/64 10.7.4 (amazon) w/debug, C++ & FORTRAN, using gcc 4.7.x
(too minor to require h5committest)
Purpose:
HDFFV-8003 - ph5diff (parallel h5diff): daily test failure on ember intermittently during non comparable test file comparison
HDFFV-7755 - parallel h5diff : hanging on koala intermittently during non comparable test file comparison
Description:
non-comparable test intermittently hung on koala and ember, but not on jam. it didn't occur until -np reaches 4 or bigger. it occurred once out of many repeated attempts of the same test.
There was a incorrectly (mistakenly?) duplicated code in MPI section which caused such hang in a certain condition. The test used more processes than other tests, which increased chance to trigger more undone processes, and such process could enter the incorrect code section and wait for wrong pair of send. it explains why it occurred intermittently according to machine condition and using a certain feature.
Removed incorrect code which blocked correct code.
Tested: some manually repeated test performed
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE)
Fix for HDFFV-8107 testh5diff will fail if build/test in HDF5 source tree
Description:
This is sub-task for "HDFFV-8105 testh5diff.sh uses the wrong operator (-a) in an if statement."
From the HDFFV-8105's update, h5diff test failed if build&test is performed in HDF5 source tree because 'cp' try to copy test files to self dir.
It's addressed by skipping if cp's src dir and dest dir is same.
Also this applied for all other tools under src/tools dir.
No change to the CMakeLists.txt files because CMake cautions/demands that in-source builds be avoided.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), some manual tests as well
Description:
The test command in COPY_TESTFILES_TO_TESTDIR used the wrong operator -a which
means and. It is not sufficient to test mere existence as $tstfile could have
been a directory or other non-file things. If the file is not readable, the
$CP command would fail but its return code was not checked.
Solution:
Get rid of the test command and go ahead to $CP it and then check its returned
code and report errors if detected.
Tested; committested. (note that ostrick still failed on errors committed by
other staff).
Create setters for indexing options in h5trav.c to be set before the obj table is created.
Revert h5trav_print to remove now duplicated parameters.
Added test and testfiles.
Tested: local linux with cmake
Create setters for indexing options in h5trav.c to be set before the obj table is created.
Revert h5trav_print to remove now duplicated parameters.
Added test and testfiles.
Tested: local linux with cmake
HDFFV-7602 - GMQS: HDF5 command tools: Provide framework for reusable test files among tools
Stage2 working for h5dump tool.
Description:
Update cmake script to fetch test files from any tools to a test directory
and run test in organized manner.
Also keep the apperance similiar to shell script for better maintainance.
Tested:
CMake (Windows and jam)
Added static local variables in h5trav to hold the index_by and index_order values. Defaults are set.
H5dump only use of the trav_print function, added two parameters and passed in the sort_by and sort_order variables.
Tested: local linux
(2) fix to extend into the aggregator based on threshold
(3) fix for the "flavor" logging problem
(4) fix for test failures due to the above fixes
(5) bug fix for h5stat when h5stat against a non-existing file
HDFFV-7602 - GMQS: HDF5 command tools: Provide framework for reusable test files among tools
Stage2 working for h5diff tool.
Description:
Update cmake script to fetch test files from any tools to a test directory
and run test in organized manner.
Also keep the apperance similiar to shell script for better maintainance.
Tested:
CMake (Windows and jam)
Fix h5repacktst (test file generating binary) to address HDFFV-8016 h5repacktst failed in AIX 32bit mode
Description:
Segmentation fault occurred when h5repacktst try to generate a dataset when
system memory is limited to 256MB.
Add error handling to display apporiate message to exit and adjust the test
dataset according to the original size which didn't causing out of memory.
Tested: test was done by limit memory in csh.
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE)
The ph5diff (parallel h5diff) can intermittently hang in parallel mode
when comparing the two HDF5 files that contains common name objects with
different object types. (HDFFV-8003)
Skip only the specific test for parallel until the fix has made.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Fix for HDFFV-7998-h5diff: incorrect behavior and output for comparing symbolic dangling links without following-symlinks option
Description:
Fix not to check and display dangling link status without --follow-symlinks option.
Berfor the fix, when comparing symbolic link to symbolic link without follow-symlinks option, h5diff followed to check if those links are dangling or not. It caused to display output incorrectly (dangling link instead of symbolic link).
The fix also improved performance when comparing lots of symbolic links without the --follow-symlinks option.
Test cases were added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
Fix for HDFFV-8001 - h5unjam: segfault when used -V (show version) option
Description:
h5unjam has an option '-V' to display version information.
"h5unjam -V" caused segfulat for 1.8.9 pre-release.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward
the final design (as exemplified by the H5EA & H5FA code).
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & parallel
Fix for HDFFV-7993 - h5repack fails with error "chunk size must be <= maximum dimension size for fixed-sized dimensions"
Description:
Fixed a failure when change the chunk size of a specified chunked dataset with unlimited max dims.
Also took care of converting to contiguous and compact from the dataset.
Test cases were added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
Clean up a few warnings and use constant values for free-space
parameters for the metadata block size test.
Tested on:
Linux/64 2.6.18 (koala) w/production & szip
(h5committested also)
Add new "metadata block size" command line option ('-M <x>' or
'--metadata_block_size=<x>') for h5repack.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug)
(h5committest upcoming)
Fix for HDFFV-7644 GMQS: h5diff - incorrect behaviors when comparing HDF5 files with different type of objects (dataset, group, type) as common objects
Description:
Fixed failure for comparing same named object with different object types in comparing groups. (common object comparison)
Prior to the fix, h5diff resulted in error. After the fix, h5diff detects such case as non-comparable and display messages accordingly.
Test cases were added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
HDFFV-398: h5cc doesn't work with automake
Description:
Fixed compiler wrapper scripts to correctly detect compilation when
-MT preprocessor flag is provided, fixing a bug in which its *.o
argument was added to link_objs but not compile_args.
This previously broke usage of h5cc as the provided compiler in
configure scripts (like with h5edit) as automake may supply the -MT
option to the compiler via the makefiles.
Tested:
Tested using h5edit and h5committested.
update for HDFFV-7837 - h5diff: incorrect behavior with exclude-path option when unique object exist only in one file
Description:
Add the new test output list to test shell script.
This is based on the previous commit r22114.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE)
Fix for HDFFV-7837 - h5diff: incorrect behavior with exclude-path option when unique object exist only in one file
Description:
If unique objects exists only in one file and try to exclude
the unique objects with --exclude-path option, h5diff missed
excluding some objects.
Fixed to exclude objects correctly in such case.
Test cases were added and tagged with jira#.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), Cmake (jam)
Minor code safety issue in test/fheap.c and whitespace in other files.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug
(Too minor to require h5committest)
HDFFV-7523 JPSS: h5dump fails to open an attribute when using the -a option to specify an attribute with a slash in the name
Description:
Update command line help page and test generating file.
Previous commits: r22052, r22063
Tested:
jam (linux32-LE), koala (linux64-LE), cmake (jam)
