HDFFV-7643 - h5diff - incorrect exit-code returns for extra attribute exist
Description:
h5diff: Fixed to return correct exit code 1 when detect unique
attribute. Prior to this fix, h5diff returned exit code 0 indicating
two files are identical.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), fred (mac64-LE), emu (solaris-BE), Windows (32-LE cmake), cmake (jam)
Since this is a testing code error that does not affect the function of the
library or tools, I started a new "Testing" section for this sort of changes.
Description:
Adds 2 new API functions, H5Dscatter and H5Dgather. H5Dscatter retrieves data
from a specified callback function and scatters it into a selection, defined by
a supplied dataspace, within a supplied memory buffer. H5Dgather gathers data
from a selection within a supplied memory buffer and passes it in a contiguous
form to a supplied callback function. Added tests for these functions
Tested: jam, ostrich, koala (h5committest); ummon
HDFFV-8145 h5diff slowness with version >= 1.8.7
Description:
Improved speed when comparing HDF5 files with lots of attributes. Much slower performance was identified with release version from 1.8.7 to 1.8.10 compared to 1.8.6.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), Windows (32-LE cmake), cmake (jam)
HDFFV-8012 - h5repack changes max dims and cause failure if only "-f none" is used without changing layout for chunked dataset when a chunk dim is bigger than a dataset dim
Description:
"h5repack -f <obj>:NONE <file.h5> out.h5" command failed if source file contains chunked dataset and a chunk dim is bigger than a dataset dim.
Another issue is that the command changed max dims if chunk dim is smaller than the dataset dim.
These issue occurred when dataset size is smaller than 64k (compact size limit)
Fixed them.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
HDFFV-5919 - GMQS: h5diff - The number of object difference is not consistent between dataset and group/type when attribute(s) have differences
Description:
Object differences are not consistent between dataset and group/datatype when their attribute(s) have differences. This is because attribute(s) differences is not accumulated to group or datatype object’s difference, but accumulated to dataset difference.
To fix, do not accumulate attribute difference to dataset difference. This is referred to h5diff’s default behavior and also past report from users that users were confused by the accumulated behavior. (also can’t figure out only for dataset difference , also hard to spot dataset difference when it has lots of attributes or differences)
This also lead to fix inconsistent format indicating difference between dataset and group/datatype object.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
HDFFV-8143 Provide a routine(s) for telling the user why the library broke collective data access
Description:
Added H5Pget_mpio_no_collective_cause() function that retrive reasons why the collective I/O was broken during Read/Write IO access.
Reasons to break collective I/O:
- SET_INDEPENDENT
- DATATYPE_CONVERSION
- DATA_TRANSFORMS
- MPIPOSIX
- NOT_SIMPLE_OR_SCALAR_DATASPACES (NULL Space)
- POINT_SELECTIONS
- NOT_CONTIGUOUS_OR_CHUNKED_DATASET (Compact or External-Storage)
- FILTERS
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Address HDFFV-7942 - h5diff: incorrect result for comparing attribute data with different type size in same class
Description:
When comparing attribute data values with same type class but different size, the result was incorrect. It was due to the size difference and got truncated.
Fixed to match up the smaller type size to big type size like what dataset does.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
Purpose:
HDFFV-8003 - ph5diff (parallel h5diff): daily test failure on ember intermittently during non comparable test file comparison
HDFFV-7755 - parallel h5diff : hanging on koala intermittently during non comparable test file comparison
Description:
non-comparable test intermittently hung on koala and ember, but not on jam. it didn't occur until -np reaches 4 or bigger. it occurred once out of many repeated attempts of the same test.
There was a incorrectly (mistakenly?) duplicated code in MPI section which caused such hang in a certain condition. The test used more processes than other tests, which increased chance to trigger more undone processes, and such process could enter the incorrect code section and wait for wrong pair of send. it explains why it occurred intermittently according to machine condition and using a certain feature.
Removed incorrect code which blocked correct code.
Tested: some manually repeated test performed
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE)
Fix for HDFFV-8107 testh5diff will fail if build/test in HDF5 source tree
Description:
This is sub-task for "HDFFV-8105 testh5diff.sh uses the wrong operator (-a) in an if statement."
From the HDFFV-8105's update, h5diff test failed if build&test is performed in HDF5 source tree because 'cp' try to copy test files to self dir.
It's addressed by skipping if cp's src dir and dest dir is same.
Also this applied for all other tools under src/tools dir.
No change to the CMakeLists.txt files because CMake cautions/demands that in-source builds be avoided.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), some manual tests as well
Description:
When jumping out from between H5_BEGIN_TAG and H5_END_TAG macros using
HGOTO_ERROR or HGOTO_DONE, the previous metadata tag is not reset on the dxpl.
This could cause problems when, for example, calling H5Ocopy within an
H5Literate callback.
Added new HGOTO_ERROR_TAG and HGOTO_DONE_TAG macros which must be used in place
of the above between H5_BEGIN_TAG and H5_END_TAG.
Tested:
jam, koala, ostrich (h5committest), durandal
Description:
The test command in COPY_TESTFILES_TO_TESTDIR used the wrong operator -a which
means and. It is not sufficient to test mere existence as $tstfile could have
been a directory or other non-file things. If the file is not readable, the
$CP command would fail but its return code was not checked.
Solution:
Get rid of the test command and go ahead to $CP it and then check its returned
code and report errors if detected.
Tested; committested. (note that ostrick still failed on errors committed by
other staff).
