HDFFV-7770 - h5diff segmentation fault on JPSS files
Description:
Fixed segfault over non-comparable attribute with different dimention or
rank, along with '-c' option to display details.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows, Cmake
Description:
H5Ocopy could get confused when copying a named datatype containing an
attribute which used that named datatype as its datatype. This happened
because H5Ocopy would recurse into the attribute's datatype before the object
the attribute was in was fully copied (i.e. before the "post-copy" pass).
Modified H5Ocopy to avoid recursing before the post-copy step in this case.
Required many changes, including to how non-committed shared messages are
copied.
Tested: jam, koala, heiwa (h5committest); durandal
Fortran wrappers.
Solutions: The calls were not needed and were removed from the C stubs h5open_c and h5close_c for the correspnding
Fortran subroutines h5open_f and h5close_f.
Platforms tested: jam with gcc and gfortran, PGI and Intel
koala with PGI and Intel
linew with the standard Sun compilers
HDFFV-7693 - h5diff produces different behavior between versions: 1.8.3 and 1.8.6
Description:
There were two issues on this.
One was not displaying all the comparable objects if non-comparable
object/attribute exist and compared before comparables ones. This issue
occurred after 1.8.4 release. This is the issue from user.
The other issue was not displaying all the comparable attributes,
if non-comparable object/attribute exist and compared ahead. This issue
were exist even before 1.8.4 release. So it's possible some attribute
comparison may have not displayed correctly in the past, if non-comparable
data were exist in common object.
Fixed h5diff to display all the comparable object and attribute regardless
of non-comparables.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Cmake (jam, Windows)
Work for:
HDFFV-7515 - GMQS: h5unjam - incorrect help page and should display when
no file is given.
and
HDFFV-5941 - h5jam: specifying a user block file with a proper HDF5 magic
number will result in a corrupted HDF5 file.
Description:
- Revised command help pages of h5jam and h5unjam. The descriptions
were not up to date and some were missing. (HDFFV-7515)
- Fixed h5jam not to allow specifying an HDF5 formatted file as input
file for -u (user block file) option, because the original HDF5 file
will not be accessible if allows. (HDFFV-5941)
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Cmake (jam, Wondows)
Replaced several old Linux platforms no longer in use with the current Linux test machine information.
No test - doc file only.
M release_docs/RELEASE.txt
- h5repack: h5repack failed to copy dataset if the layout is changed from chunked with
unlimited dims to contiguous. (PC -- 2011/07/15)
- h5diff: "--delta" option considers two NaN of the same type are different, which is wrong
based on http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Diff. (PC -- 2011/07/15)
Purpose:
Remove H5_MPI_SPECIAL_COLLECTIVE_IO_WORKS and
H5_MPI_COMPLEX_DERIVED_DATATYPE_WORKS #defines from source.
Description:
Two advanced parallel functionalities, special collective IO and
complex derived datatypes, are not supported by older
implementations of mpi, and thus our code limits the use of these
features with #ifdefs and has checks in configure to set them (or
not). Unfortunately, configure can't actually run a parallel check
to see if these features are working (nor not) so it resorts to
looking in the config files where they are explicity enabled or
disabled based on versions of mpi, sytems being built on, or for
no documented reason at all (i.e. just set to on or off as some
'default'). Overriding these settings is easy if need be, provided
it is known that it needs to be done to get improved performance,
and oftentimes it is not.
Most new MPI implementations successfully handle the functionality
requested when these #defines are set, and many of the "turn these
features off" cases in the config files are for old (> 5 years)
versions of MPI and retired systems (such as NCSA's tungsten).
Therefore, the decision has been made to remove the support for
these old versions of MPI and systems that cannot handle these
behaviors. The #ifdefs and supporting setup in the config/ files
and configure script has been removed, and the code executed when
these options were not set removed from the source.
In passing, this commit also cleans up some whitespace issues in
both t_mpi.c and H5Dmpio.c. Furthermore, in t_mpi.c, the special
collective IO test was not getting regularly run due to it being
written to work only with four processes (we regularly test with
six, previously with three), and thus it failed when actually run
due to an out of bounds data buffer assignment. It has been
modified to run at any number of processes greater than four, and
the memory problem has been fixed so the test passes.
Tested:
jam, h5committest, ember
Work for HDFFV-7600 - GMQS: h5diff - argument options -d, -p and --use-system-epsilon should be mutually exclusive.
Description:
Fixed h5diff to display instructive error message and exit with 1
when mutually exclusive options (-d, -p and --use-system-epsilon)
are used together.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Cmake (jam)
super block. We took it out in v1.8 library because it's meaningless
for the MULTI file. v1.8 library saves the EOA for the metadata file,
instead. But this caused some backward compatibility problem.
v1.8 library couldn't open the file created with v1.6 library. I
fixed the problem by checking the EOA value to detect the file
created with v1.6 library.
Tested on jam, koala, and heiwa.
HDFFV-5878 - GMQS: h5dump - incorrect output format for array type with multiple elements in an index
Description:
Fixed h5dump to display the first line of each element into correct
position for multiple dimention array type.
Before this fix, the first line of each element in array were displayed
after the last line of previous element without moving to the next
line (+indentation)
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE)
data, overflowing values are retained. When conversion happens, the values become -1. The conversion function puts -1
when overflow happens. I added two new dataset transfer property to control whether to fill 0xff in the destination
data or convert to the destination data when overflow happens. The two new functions are H5Pset(get)_enum_conv_overflow. I also added test cases in enum.c and dtypes.c.
Tested on jam, koala, and heiwa.
HDFFV-7594 - GMQS: h5dump shows wrong values for H5T_STD_I8LE dataset on Blue-gene (ppc64 linux BE +@)
Description:
For the STD_I8 type data, updated to use 'signed char' instead of 'char' as
'char' could be defined as unsigned on a certain system, which cause h5dump
to display data values incorrectly.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE)
HDFFV-5928 - GMQS: h5diff problem and improvement on comparsing the same objects
Description:
Improved performance by eliminating duplicated action for getting object
information in half from the previous fixe when comparing group vs group.
This is addition to the previous commit r20676.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), cmake
Small improvement to the core VFD's error reporting.
Update FreeBSD information
Tested on:
FreeBSD/32 8.2 (loyalty)
(too minor to require h5committest)
configure.in that tried to distinguish different versions of AIX. This is no
longer needed.
Tested:
NCSA Blue-print. No h5committest since this affected AIX system only.
Add "silent make" mode configure option.
Description:
Automake 1.11 has a new option available that allows for a
silent make mode. This functionality needs to be explicitly
enabled in configure.in via the use of the automake macro
AM_SILENT_RULES, which is what this commit is adding.
This introduces a new configure option:
--{en|dis}able-silent-rules
This option is on by default, and simplies compile and link
line outputs when building the library. Disabling this option
will print full "verbose" output (i.e., full compile and
linking lines for each target).
Tested:
This was tested on jam & h5committested
Fix HDFFV-4342 : GMQS: h5dump test fails when source directory is read-only
Description:
Updated h5dump test case script to prevent entire test failure upon
source code directory is read-only.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Fixed issue HDFFV-5866 (BZ 2156). Changed scripts to run examples to use specific names for compiled executable files instead of a.out, which did not work on Cywin as it produces a.exe by default. Removed issue from known problems section of RELEASE.txt.
Tested with Cygwin 1.7.8 on Windows 7.
This line, and those below, will be ignored--
M release_docs/RELEASE.txt
M hl/c++/examples/run-hlc++-ex.sh.in
M hl/fortran/examples/run-hlfortran-ex.sh.in
M hl/examples/run-hlc-ex.sh.in
M c++/examples/run-c++-ex.sh.in
M fortran/examples/run-fortran-ex.sh.in
M examples/run-c-ex.sh.in
- Revise shared Fortran library disabling scenarios in configure
- Improve configure output summary
Description:
Shared Fortran libraries are not supported on Mac, but were being
disabled by configure in a way that also forced the C libraries
to be static-only. This has been fixed, so now only shared Fortran
is disabled while shared C can remain.
This prompted two additional changes:
1. While working on the check that addresses whether or not
shared Fortran libraries are allowed, removed old and no
longer needed check(s) that disable shared Fortran
libraries with HP, Intel 8, PGI, and Absoft compilers.
(Essentially, Mac is the only situation in which Fortran
shared are disabled by configure.)
2. Having two different states of libraries (i.e. shared C
library with static-only Fortran library) was not apparent
in the configure summary, which labeled all libraries as
either shared and/or static. I've added lines to both the
C++ and Fortran output sections to list shared/static-ness
of these libraries specifically.
Additionally, I've made sure that the new --enable-unsupported
configure option correctly overrides configure if it tries to
disable a shared library.
Tested:
jam, fred, & h5committest
Fixed Bug 2184 - GMQS: h5diff - incorrect calculation code for
--use-system-epsilon option
Description:
Merged from HDF5 1.8 branch r20369.
Fixed h5diff for --use-system-epsilon option: the calculation changed
from ( |a - b| / b ) to ( |a - b| ). This was decided for better
performance and was corrected only in HDF5 trunk, so 1.8 got updated.
Also comments for equal_XXX() function were updated correctly.
Also help page and RM got updated correctly.
Also add test cases for testing the differences w/wo the option.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE)
Bug 2182 - h5ls and h5dump: update to use --enable-error-stack among tools, also update related descriptions
Description:
Decided to use --enable-error-stack long option name only to
display error stack messages from HDF5 lib among tools.
Updated to unify option name to '--enable-error-stack' for printing
HDF5 error stack messages for HDF5 tools. h5ls and h5dump for now.
For h5ls, this replaces "-e/--errors" option, which is deprecated.
For h5dump, remove -E from help page and RM , which was added by mistake
before release1.8.7.
Help page and RM got updated.
Updated test case.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE)
Fixed CHICAGO: Bug 2121 - h5diff - incorrect and lack of output for the
different set of attributes (different number and names)
Description:
This is related to previous checkin r20294.
Add help page update for verbose option level feature, which displays
useful information about attribute difference with -v1, --verbose=1 -v2
or --verbose=2 option.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Fixed CHICAGO: Bug 2121 - h5diff - incorrect and lack of output for the
different set of attributes (different number and names)
Description:
Previously h5diff compared attributes correctly only when two objects have
the same number of attributes and attribute names are identical.
This fix covers all other cases.
Also didn't display useful information about attribute difference.
This fixes both issues.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Improve the previous fix for Bug 2216 - GMQS: h5diff - memory leak when
compares vlen string in dataset or attributes
Description:
Improve the fix along with the previous checkin r20266.
Add a new function to tool lib, h5tools_detect_vlen_data() which return
TRUE if include any kind of vlen data all at once, either VLEN-data or
VLEN-string and so on.
Also updated h5ls and h5dump code accordingly.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Cmake - jam
Fixed Bug 1904 - h5ls : usage of '-a' option is not clear to display expected output
Description:
Fixed incorrect content and improved help page for -a option of h5ls.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), Cmake (jam)
Fix Bug 2120 - h5copy: improve to copy an object into same HDF file
Description:
Before the fix users can't copy an object (dataset or group) with
different name if input file (-i) and output file (-o) is same.
This feature would be useful to clone any object with different name and
reuse contents.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Cmake (jam)
Fixing Bug 2161 - GMQS: h5dump - only on Windows, skip displaying a data value every a certain lines in array type dataset
Description:
Fixed h5dump for skipping some values for long array type dataset on
Windows. This issue only occurred on Windows due to the different
return behavior from _vsnprintf() funtion.
Tested:
Windows, jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Fixing Bug 2092 - h5dump does not display index for a dataset
Description:
h5dump skip displaying array indices every certain number when the
array type dataset is relatively big. The certain number varies
according to the size of each array.
This checkin fix the skipping array indices problem. This fix also
correct indentation of the dataset data output.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Description:
When using the scale-offset filter with floating point data or fill values, big
endian machines would save some metadata in the wrong byte order. This caused
such datasets to yield incorrect data when read on little endian machines.
Fixed the scale-offset filter to always save this metadata in the right byte order (i.e. little endian).
Tested: jam, amani, heiwa (h5committedt); fedora, linew
Bug 2089 - GMQS: h5diff segfault on a compound dataset with fixed length +
vlen string type order
Description:
This is fix for the segfault when h5diff compares a compound dataset with
combination of fixed length string types and vlen string types in certain
orders. Optimized vlariable length string handling codes. The fix is
referred from h5dump handling vlen strings.
For testing, several compound datasets were added with various combinations.
Previous failed cases:
- Vlen string, Fixed length string, Vlen string, Fixed length string
- Fixed length string, Fixed length string, Vlen string, Vlen string
- Fixed length string, Vlen string, Fixed length string, Vlen string
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Add "--enable-unsupported" configure flag.
Description
The "--enable-unsupported" configure flag allows a user to
prevent configure from failing due to the use of incompatible
options, such as c++ with parallel.
Specifying --enable-unsupported will bypass all of configure's
checks for incompatible and unsupported combinations of flags.
There are no guarantees that the library will be configured
in any sort of working condition, but that's the risk of
using the --enable-unsupported flag.
I've changed all default error messages related to unsupported
option combinations to indicate that using --enable-unsupported
will allow configure to complete without error.
Tested:
by hand on jam, tested all unsupported configure option
combinations with and without the new flag, making sure
the flag allows configure to finish without error.
(h5committest wouldn't do any good here; it won't test
the new option, and since we're enabling unsupported
combinations, failures are likely to occur in build or
tests with --enable-unsupported turned on anyways.
That's why they're unsupported!)
Add additional error checking to catch erroneous user input.
Description:
Attempting to retrieve a links's name by index in the case
where the link is external and the file that the object is
located in doesn't exist was causing a segmentation fault
(in production) and an assertion failure (in debug).
The segfault wasn't occuring until the metadata accumulator
attempted a write, so I've added error checking higher in
the pipeline in H5O_protect (where there was previously just
an assert) to catch this. I've also added additional asserts
in the H5F layer where there were none.
Additionally, I added another case to the links.c test to
test that this fails gracefully instead of segfaulting or
asserting out.
Tested:
h5committest and gandalf (mac os x)
Improve h5diff performance.
Description:
The following changes for improving h5diff performance:
1) use HDmemcmp() before comparing each elements
(memcmp() is very fast at both linew and jam)
2) replace the expensive H5Tequals() calls
3) retrieve datatype information at dataset level not each element level
for compound datasets
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
Fixed Bug# 1979 Output from h5ls -rdlS on nested compound datatypes is
difficult to parse.
Description:
Update to add curly brackets for the nested compound members, when
S (--simple) option is used with -l (--label), so user can tell
which members blong to which compound type.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32)
Fixed h5diff to handle variable-length strings in a compound dataset
correctly. Also variable-length string array in a compound dataset.
Bug #1989.
Description:
Garbage values were displayed when h5diff compared variable-length
strings (or string array) in a compound type dataset.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (Mac32)
Fix for bug# 2040 - h5copy should fail gracefully for expected failure copying to non-exist nested group without -p option.
Description:
Fixed h5copy to fail gracefully when copying object to non-exist group without -p option. This is expected to be failed.
Tested:
jam, amani, heiwa
Backout the previous changes (r19598):
(Fix for bug# 2040 - h5copy should fail gracefully for expected failure copying to non-exist nested group without -p option.)
Description:
Some failure occurred on talwit, so backout the changes and will put it back
after the fix.
(Fixed h5copy to fail gracefully when copying object to non-exist group without -p option. This is expected to be failed.
Merged from hdf5 trunk r19598.)
