Fixed Bug# 1979 Output from h5ls -rdlS on nested compound datatypes is
difficult to parse.
Description:
Update to add curly brackets for the nested compound members, when
S (--simple) option is used with -l (--label), so user can tell
which members blong to which compound type.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32)
Fixed h5diff to handle variable-length strings in a compound dataset
correctly. Also variable-length string array in a compound dataset.
Bug #1989.
Description:
Garbage values were displayed when h5diff compared variable-length
strings (or string array) in a compound type dataset.
Tested:
jam (linux32-LE), amani (linux64-LE), heiwa (linuxppc64-BE), tejeda (Mac32)
Fix for bug# 2040 - h5copy should fail gracefully for expected failure copying to non-exist nested group without -p option.
Description:
Fixed h5copy to fail gracefully when copying object to non-exist group without -p option. This is expected to be failed.
Tested:
jam, amani, heiwa
Backout the previous changes (r19598):
(Fix for bug# 2040 - h5copy should fail gracefully for expected failure copying to non-exist nested group without -p option.)
Description:
Some failure occurred on talwit, so backout the changes and will put it back
after the fix.
(Fixed h5copy to fail gracefully when copying object to non-exist group without -p option. This is expected to be failed.
Merged from hdf5 trunk r19598.)
Fix for bug# 2040 - h5copy should fail gracefully for expected failure copying to non-exist nested group without -p option.
Description:
Fixed h5copy to fail gracefully when copying object to non-exist group without -p option. Expected to be failed.
Tested:
jam, amani, heiwa
unknown versions of the C Intel compilers.
Modified apple config file to display Intel compiler information.
Updated RELEASE.txt notes accordingly.
Platfroms tested: jam, amani and dubna (Mac 64-bit) with Intel C, Fortran and C++ compilers.
Bug 1707 is that H5Eset_auto causes a seg fault when an application uses -DH5_USE_16_API with
the 1.8 library to compile.
I created a branch off the trunk called set_auto to continue working on the problem.
Tested on jam - backing out, simple.
Description:
Library versions 1.6.3 and earlier contain a bug which causes them to be unable
to perform certain operations on a group if that group's symbol table
information is not cached in the parent group's symbol table. Versions 1.8.0
to 1.8.5 did not cache this information. Modified library to cache this
information.
Tested: jam, amani, heiwa (h5committest)
through H5Eset_auto. I changed the union in the structure H5E_auto_t. Another change is to
make H5Eget_auto fail if H5Eset_auto is called to set the printing function. I'll write a
document for it.
Tested on heiwa, jam, and amani.
The property change in configure.in, config, and Makefile.am came from the merge of the 1.8
library change.
Add --exclude-path option
Description:
Specified path to an object will be excluded from comparing the two files or two groups. If group is specified all the member objects will be excluded.
Related to "1890: h5diff excluding object for file comparison via command line"
Tested:
jam, amani and heiwa
Fix for Bug1975 h5diff - support recursive comparison on group when specified as an object
Description:
Compare member objects and groups recursively when two files or groups are specified to be compared. Support parallel diff and handling symbolic links accordingly.
Tested:
jam, amani, heiwa
Fix for Bug1896 h5repack - changing layout to COMPACT does not work
Description:
Make h5repack be able to convert a layout to COMPACT for small size dataset as default. Also add verifying layout changes in our test script.
Tested:
jam, amani, heiwa
Description:
The fractal heap header structure keeps a pointer to the file associated with
it. However, it is possible for that file pointer to be closed while the
header is still in cache (through the shared file pointer). Previously, the
header's file pointer was not updated and subsequently pointed to an invalid
file structure. Modified fractal heap code to update the file pointer every
time the header is accessed.
Tested: jam, linew, amani (h5committest)
Add windows threads support to HDF5.
Description:
Added calls to the windows threads library to the H5TS layer, and wrapped
most calls to either pthreads or windows threads library with portable
H5TS-style defines. Modified tests to use portable function definitions
as well.
This can be configured via CMake with the HDF5_ENABLE_THREADSAFE
option, and should work on windows vista and later operating systems.
Tested:
h5committest, plus threadsafe with pthreads on jam and amani,
and tested on a Windows Vista VM with threadsafe using windows threads.
H5T_ORDER_MIXED specifically for compound types and its derived types. They report mixed
orders if they have members of different orders.
There is no change to configure.in, config, and Makefile.am in the top directory. They have
some property changes when I merged the code from 1.8.
Tested on jam. I tested the same change for 1.8 with h5committest.
1. For enum type, members shouldn't be defined yet.
2. H5T_ORDER_NONE only works for reference and fixed-length
string.
3. For opaque type, the order will be ignored.
4. For compound type, all restrictions above apply to the
members.
I'll change H5Tget_order and do another commit.
There is no change to configure.in, config, and Makefile.am. There is some property change for
these files when I did a merge from 1.8.
Tested on jam. But I tested the 1.8 on heiwa, and amani.
herr_t. To minimize the change of the library's behavior, in the function
H5Z_prelude_callback of H5Z.c, if the return value of can_apply is FALSE and
the filter is MANDATE, this function returns a FAILURE. If the return value is FALSE
but the filter is OPTIONAL, this function returns a SUCCEED. During the IO, the filter
will fail and return a size of zero. But the pipeline will skip this filter.
Tested the same change for 1.8 on jam, linew, and amani. Tested on jam with szip.
Description:
In certain circumstances, the direct I/O driver did not perform correctly when
data was unaligned. The driver has been patched to fix this. Also added some
potential performance improvements for the unaligned case, and strengthened the
test for whether the data needs to be aligned.
Tested: cobalt
Description:
A bug introduced in 1.8.5 causes local heap data blocks to be mis-aligned when
sizeof_offsets + 2*sizeof_lengths is not a multiple of 8. In this case, the
address of the data block as stored in the heap prefix is aligned but the actual
data block is not. This causes files created with these sizes to be corrupted,
and prevents uncorrupted files with these sizes to be unreadable. Modified
local heap code to account for alignment.
Tested: jam, amani, linew (h5committest)
Add --no-dangling-links option to h5ls.
Description:
Related to "Bug 1830 - Following an dangling external link in h5ls should set non-zero return code."
If --no-dangling-links option is specified and any dangling link is found, return exit code 1 (error).
Tested:
jam, amani and heiwa
Change h5ls not to manipulate special characters in object name or attribute name for smart display.
Description:
Related to "But1784 h5ls has input and output issues with links that have '\' characters in them."
Tested:
jam, amani and linew
Description:
Previously, the chunk cache would, when looking for a chunk, alwaylook for the
chunk in the b-tree before checking the cache. Reworked the functions that
look up chunks to always check the cache first and avoid the b-tree lookup if
the chunk is in cache.
Tested: jam, linew, amani (h5committest)
PHDF5 changed to use "mpiexec", instead of mpirun, as the default MPI
applications startup command as defined in the MPI-2 definition, section
4.1. Note that only mpich related (using mpicc command) definitions are
changed to mpiexec. Other favors like hcc, AIX, champion, are not changed
since I don't have a way to verify them yet.
Tested:
jam and amani, using both current and new mpich.
Did not h5committested since this affects PHDF5 configure only.
Support follow symbolic links.
Description:
Add '--follow-symlinks' option to follow symbolic links (soft and external).
Update help page according to RM.
Remove some warning messages from compiler.
Tested:
jam, amani and linew
Fixed a bug (1855) in H5DSattach_scale, H5DSis_attached and H5DSdetach_scale
caused by using H5Tget_native_type function to determine the native
type for reading REFERENCE_LIST attribute. The bug was exposed on Mac PPC.
Platforms tested: juniper (Mac PPC), jam, amani, linew
Rename '--follow-links' to '--follow-symlinks'
Description:
The '--follow-links' option is to follow symbolic links (soft and external).
Make the name more intuitive and specific to the feature.
Tested:
jam, amani and linew
Description:
Previously, it was possible for a chunk to be flushed due to chunk operations in
the callback from H5B_iterate in H5D_chunk_prune_by_extent. Because flushing
the chunk can force it to be reallocated if it is filtered, this can change the
contents of the chunk b-tree in the middle of H5B_iterate. Because H5B_iterate
uses a locally cached copy of the b-tree, this causes subsequent operations
to be passed incorrect data. Rewrote H5D_chunk_prune_by_extent to avoid
H5B_iterate entirely.
Also fixed a bug in the dataset expand algorithm that could cause extra chunks
to be created.
Tested: jam, linew, smirom (h5committest)
Fixed a bug in the H5DSdetach_scale function when 0 bytes
were allocated after the last reference to a dim. scale
was removed from the list of references in a VL element of the
DIMENSION_LIST attribute; modified the function to comply
with the Spec: DIMENSION_LIST attribute is deleted now when no
dimension scales left attached.
Platforms tested: jam with electric fence, amani amd linew
Add bug fixes for Tools.
Description:
Added list:
1726 - NPOESS: h5repack loses attributes for datasets of type H5T_REFERENCE
1814 - NPOESS: h5repack doesn't handle references to the groups as an
element of a dataset
1817 - h5copy fail to copy dangling link by specifying link path directly
1793 - h5ls on a non-existent file gives 0 return code
Tested:
no, just text update.
Fix memory leaks
Description
Added a routine to free memory which addresses a memory leak
when variable length strings are used as fill values.
Tested:
h5committest and valgrind (on jam/amani) to confirm freed memory.
Added notes about
+ PropList::PropList(id) pclass id/plist id
+ new DataSet::getInMemDataSize
+ CommonFG::getLinkval and CommonFG::getComment take default values
+ bugzilla 1061
+ read/write methods of DataSet and Attribute
Improve configure's large-file support control.
Description:
Modified configure to now attempt to add defines necessary for
supporting largefiles on all systems, instead of solely on linux. This
is in response to user requests to enable largefile support on Solaris
by default, as well as to give extra control on AIX (instead of just
jamming the necessary flag into the config files).
The old --enable-linux-lfs flag has been removed in favor of the
--enable-largefile flag (enabled by default), which can be used on all
platforms.
On systems where large files cannot be supported in this manner,
configure will report as such.
Tested:
h5committest
AIX (NCSA's blue_print machine)
duty, liberty, and linew.
Autotool Upgrade / Libtool Bug Fix
Description:
Updated autotools referenced in bin/reconfigure as follows:
Autoconf 2.64 --> Autoconf 2.65
Automake 1.11 --> Automake 1.11.1
Libtool 2.2.6a --> Libtool 2.2.6b-mcg
The referenced libtool version is a custom version of 2.2.6b. It
has been tweaked to fix a bug in libtool that occurs
when using PGI 10.0 compilers. A check incorrectly categorizes
the C++ compiler as version 1.0 instead of 10.0, and the link
line is subsequently set up incorrectly and fails to compile.
A patch has been made available and will be included in the next
release of libtool, but in the meantime I've applied the patch to a custom
installation as indicated above. This bin/reconfigure now references
the custom installation, and the resulting configure script will correctly
categorize the PGI 10.0 C++ compiler.
Ran bin/reconfigure to regenerate configure and makefiles.
Tested:
jam and amani with PGI 10.0 compilers.
h5committest
Description:
Attempting to copy an object with NULL references (all bytes zero) with the
H5O_COPY_EXPAND_REFERENCE_FLAG flag set would cause a failure or an assertion
(depending on whether it was in debug mode). Changed copy routine to detect
NULL references (object and region) and avoid attempting to expand the reference
in this case.
Tested: jam, linew, amani (h5committest)
Description:
Modified algorithm for extending a dataset with early allocation so it only
deals with the new chunks. Formerly, it would loop over all chunks, checking
to see if each existed in cache and on disk, causing major performance issues
with large numbers of chunks.
Tested: jam, linew, amani (h5committest)
Description:
The core file driver previously matched files only based on the name used to
open them, even when a backing store is used. This caused problems in the
failing test when it traversed an external link to itself, and did not recognize
that it was the same file. Similar tests in links.c were skipped when the core
file driver was used. The failure was previously masked by inappropriate file
flushes caused by external link traversal. Added code to the core file driver
to keep track of low level file information of the backings store to allow for
accurate recognition of duplicate files. Also re-enabled all links.c tests for
the core file driver.
Tested: Fedora (core and sec2 drivers; no time for full commit test, will
monitor daily tests)
Description:
Support for expanding external links was not implemented in H5Ocopy, even though
a flag existed for H5Pset_copy_object to enable this. This patch implements
this feature.
Tested: jam, amani, linew (h5committest); Fedora
Description:
Previously, the library would reopen the source file when traversing an external
link if unable to find the target file otherwise. This has been corrected.
Also moved the call to H5F_flush from H5F_try_close to H5F_dest, so the file is
only flushed when the last identifier for the file is closed. This prevernts
situations where the library could attempt to flush a file with protected
metadata.
Tested: jam, amani, linew (h5committest); Fedora
bugzilla 1754: h5diff: support comparing through links.
Description:
Fix incorrect (or hanging) behavior in parallel mode when specifying
invalid combination of command options.
(ex: -v and -q , --no-dangling-links without --follow-links)
Add relate test case
Update h5diffgentest.c due to add test files this and previous time
Note:
svn #18266 (prior to this)
svn #18164 (original check-in)
Tested:
h5committest (jam, amani and linew)
bugzilla 1754: h5diff: support comparing through links.
(original check-in svn revision #18164)
Description:
fix the hang issue in parallel mode when compare external-link.
add --no-dangling-links option.
add test cases (#450-#459) relate to the new option.
improve test script to check exit code.
update --help relate to the new options.
correct some indentations.
Tested:
h5committest (jam, amani and linew)
Description:
In certain cases, removal of an object in a v1 b-tree would cause the leftmost
key in the right neighbor to be overwritten. While this did not pose a problem
for group b-trees, with chunked datasets it would overwrite the offset value
of the neighbor's leftmost child, causing corruption. Reworked the code to
differentiate between b-trees whose children are fundamentally associated with
their left key and those who are associated with their right key.
Tested: jam, linew, amani (h5committest)
Fix Bug1672 - Display garbage value on LE machine for BE data.
Description:
Casuing by calling diff_dataset twice when -r or no option was given.
Change to call it once which fix the problem.
It also improves the performance. (diffing time in half)
According further test, it also occurred on BE machine as well and it
seems occruing only with the BE data attached to Bugzilla.
Don't know how the file was created. Anyway this fix will prevent
from diffing with garbage values in similiar potential case.
Tested on:
linux32 (jam) , linux64 (almani), solaris (linew)
Description:
H5HG_load made improper assumptions about the ordering of object indices, namely
that they are in order. Not only is this not guaranteed by the file format
spec, but this condition can be violated if id's "wrap around" which can happen
when overwriting VL data. H5HG_load has been fixed to handle any order of
indices. Also fixed some other bugs involving allocation of global heaps in
memory.
Tested: jam, linew, amani (h5committest)
Description:
The H5TB API makes some improper assumptions about the order of compound
datatype members. Namely, it assumes that members remain in the order in which
they were inserted. Unfortunately, this assumption is inherent in the design of
the interface. The library has been patched so that this assumption holds in
situations relevant to H5TB.
Tested: jam, linew, amani (h5committest)
CFLAGS overhaul
Description:
Modified the way configure handles CFLAGS.
(note: all changes also apply to FCFLAGS, CXXFLAGS, CPPFLAGS, and LDFLAGS).
1. The configure process will now always preserve a user's CFLAGS
environment variable setup. Any additional flags necessary for compilation
added at configure time will be passed into the Makefiles as AM_CFLAGS,
which is an automake construct to be used in addition to CFLAGS.
This will allow a user to have the final say, as CFLAGS will always appear
later in the compile line than AM_CFLAGS. Additionally, setting CFLAGS
during make will no longer completely erase all flags set by configure,
since they're maintained in AM_CFLAGS.
2. Additionally, where possible, flags previously being assigned directly
into CFLAGS (and thus propagating into h5cc) have now been redirected into
H5_CFLAGS, so they're used ONLY for compiling hdf5, and not embedded into
the h5cc wrapper script as well.
*Note that H5_CFLAGS ultimately is assigned into AM_CFLAGS for use in the
Makefiles. Complete description of changes and build process will be
included in a Configure Document that Elena and I are working on.
3. Removed unsupported config files. This includes:
config/dec-osf*
config/hpux11.00
config/irix5.x
config/powerpc-ibm-aix4.x
config/rs6000-ibm-aix5.x
config/unicos*
4. Modified configure summary to display additional values. Specifically,
appropriate AM_* variables are being shown, as well as H5_FCFLAGS and
H5_CXXFLAGS, which were for some reason not already present.
Tested:
- H5committest
- Tested on all THG / NCSA machines, using several combinations of the more
prominent configure options (c++, fortran, szip, threadsafe, parallel,
et cetera). (Thanks to Quincey for rysnc testing setup!)
- With regards to new automated testing, anything *necessary* for
compilation will be caught by the daily tests as it stands now. (i.e.,
if LDFLAGS is not properly set when szip is used, linking will fail).
Additionally, with regards to which flags get into h5cc, if any
*necessary* flags have been improperly removed, then daily tests should
fail during make installcheck. Additional machine-specific desired
behaviors and/or checks may have to be set up separately within the
daily tests, so this is something to work on.
Description:
When copying a dataset using a vlen inside a compound, the various dataset
copying callbacks would allocate a background buffer but would not use it when
converting from disk to memory, only memory to disk. This caused an assertion
failure as compounds always need a background buffer. These callbacks have
been modified to use the background buffer for both conversions.
Tested: jam, linew, smirom (h5committest)
Makefile Bug Fix
Description:
Added $(EXEEXT) extension to H5detect when it's executed to generate
H5Tinit.c so it works correctly on platforms that require the full
extension when running executables.
Tested:
h5committest, mingw, and user-verified (via submitted patch).
Configure Fix
Description:
Modified configure to set FC and CXX to "no" when fortran and c++ are
not being compiled, respectively. This will prevent configure from running
some checks on these compilers when they are not being used.
This fixes a problem that was occuring when FC and/or CXX are set to
a nonexistant or invalid compiler. Even if the languages were disabled,
a libtool test would fail, thus preventing libtool from properly linking
any libraries, yet configure would not flag an error message due to this
because the languages were disabled.
Tested:
- h5committest (jam, smirom, linew)
- manually eyeballed configure output and logfile.
Added missing H5Tclose in region reference routines.
H5DS.c checked for other locations which need to account for platform dependent types in dealing with ds_list_t. Added generator program for test files of LE and BE dimension scales.
Tested: local linux, h5committest
Changed H5check_version() to suppress the warning message totally if
$HDF5_DISABLE_VERSION_CHECK is 2 or higher. (Old behavior treated 3 or higher
the same as 1, that is to print a warning and allows the program to continue.
Tested:
h5committested, extra jam serial, --disable-embedded-libinfo.
Description:
H5Ppublic.h listed H5Pget_filter_by_id1 and H5Pget_filter_by_id2 as returning
H5Z_filter_t, when in fact they return herr_t. The header file has been
corrected to indicate that they return herr_t.
Tested: jam
Updating autotools
Description:
Installed new autotools and used them to reconfigure HDF5.
- Automake upgraded to 1.11
- Autoconf upgraded to 2.64
- bin/reconfigure script edited to use new versions (on jam), and
run to generate new configure script and Makefile.in's.
- configure.in script edited to add "_cv_" to all AC_CACHE_VAL strings
(in order to comply with new autoconf standard).
Tested:
Tested on machines jam, smirom, liberty, linew.
Tested w/ features c++, fortran, parallel.
Tested w/ compilers gcc, pgcc, icc.
Further testing via Daily Tests should catch any other outliers. Upon
passing DT's, I'll propogate the new tools into 1.8, hdf4, et cetera.
Fix BZ #1583
Description:
The --enable-static-exec flag was broken. Configure was adding the
-all-static linker flag to an environment variable which was subsequently
never used. (looks like it was used in 1.6 Makefile.ins, but never
moved to 1.8's Makefile.ams when we added automake support).
To fix, I've added this environment varibale to the link lines of all the
tools. Now, when --enable-static-exec is invoked, it will correctly
generate statically linked executables in the installed bin directory.
Tested:
I've tested manually to ensure that the generated execs are static when
the flag is used and dynamic when not. I've also tested when szip is
used, as this was a use case mentioned in the bug report.
Also: ran full tests with make check on jam and smirom for good measure.
Description:
Previously, when H5Z_filter_info was called for a filter that was not present,
it would succeed and return the information from one position past the end of
the pipeline, possibly causing a segfault. This affected at least
H5Pget_fitler_by_id1/2, and possibly other API functions. Fixed to properly
return failure.
Tested: jam, linew, smirom (h5committest)
Description:
When using H5T_copy on committed datatypes that are already open, H5T_copy would
properly use the already existing shared struct, but would still deep copy all
of the fields in that struct. This would cause memory leaks, and in the case of
a compound containing a vlen (or reference), the change in size would cause the
size of the resulting type to be set to an incorrect value. Changed H5T_copy to
properly avoid deep copies when using a reopened shared struct.
Tested: jam, linew, smirom (h5committest), purify on jam
ISSUE: h5ltread_dataset_string_f causes library assertion with g95.
SOLUTION: convert the fortran string buffer to a C buffer with HD5f2cstring, and pass this string to the C function
TEST: added a test call in the fortran test lite program
DOCS: added the note in RELEASE.txt "- Lite: the h5ltread_dataset_string_f and h5ltget_attribute_string_f functions had memory problems with the g95 fortran compiler. (PVN - 5/13/2009) 1522
Description:
Previously, there was no versioning for H5Z_class_t. This prevented applications
written for 1.6 using custom filters from being able to use the 1.8 library.
There is now an H5Z_class1_t and H5Z_class2_t to enable compatibility. H5Zregister is
*not* versioned, it determines which version of the struct has been passed in by the
value of the first field (id or version, both are ints).
Tested: jam, linew, smirom (h5committest), jam (--with-default-api-version=v16)
Description:
When writing data to a dataset, the data transform was performed after type conversion.
This caused an error if the file type was non-native. This has been changed so data
transforms are always performed on the memory type.
Tested: jam, linew. smirom (h5committest)
Description:
h5repack previously would not take named datatypes into consideration when copying
datasets and attributes. This would cause extra anonymous datatypes in the target file
at best, and cause errors halfway through the repacking at worst. h5repack should now
always handle named datatypes correctly. Named datatypes are also now converted to the
native type when -n is given.
Tested: jam, linew, smirom (h5committest)
- Updated bin/reconfigure to use latest version of automake (1.10.2).
Re-generated Makefile.in's by running bin/reconfigure.
- Added libtool version numbers to c++, fortran, hl, hl c++, and hl fortran
libraries.
Tested:
jam, liberty, smirom
Added a Make target of check-all-install to test the correctness of
installing via the prefix= or $DESTDIR options.
Platforms tested:
h5committested and jam serial mode. Then test check-all-install by hand.
- h5diff new flag, -c, --compare, list objects that are not comparable.
PVN - 2009/4/2 - 1368
- h5diff new flag, -N, --nan, avoids NaNs detection. PVN - 2009/4/2
Description:
The H5L interface was not marked as uninitialized when H5L_term_interface was
called. This caused the interface to not initialize itself later on. This
prevented external links from working after calling H5close, because they were
never re-registered.
Tested: jam, smirom, linew (h5committest)
Description:
Versions of the library between 1.3.0 and 1.6.3 have a bug which prevents them
from opening any file that does not have the root group's symbol table
information cached in the root group's entry in the superblock. Prior to 1.8
this was not an issue as this information was always cached. However, 1.8.0
stopped writing this information (which is not required by the file format
specification), and these older versions can therefore not read files created or
last written by versions 1.8.0 to 1.8.2. This fix modifies the library to once
again add this information to the superblock (when using the old file format).
Tested: jam, linew, smirom (h5committest)
Description:
Changed Skip list package to use a deterministic skip list. This allows the
skip list package to avoid calling rand() and srand(), even on machines without
rand_r(). There is no longer a p-value or maximum level for skip lists.
Tested: jam, smirom, linew (h5committest)
Description:
Previously, H5Lcopy and H5Lmove would (through H5L_move) improperly apply the
"create intermediate groups" property to the source path traversal, and not the
destination. Fixed it to apply the property to the destination and not the
source. Also fixed H5Lcreate_ud to reject internal link classes without
throwing an assertion.
Tested: linew, jam, smirom (h5committtest)
Fortran High-Level APIs:
------
- Lite: The h5ltget_dataset_info_f function (gets information about a dataset)
was not correctly returning the dimension array (PVN - 2009/3/23)
When reading the compression parameter keyword, the compression type read flag was incorrectly set to read, removed this line of code
in->configOptionVector[COMPRESS] = 1;
Modified one configuration file to have the COMPRESSION-TYPE GZIP
Keyword.
Entered a bug description fix of
- h5import: By selecting a compression type, a big endian byte order was being
selected (PVN - 2009/11/3)
tested: linux
Bug Fix
Description:
Fixing BZ #1381. The --includedir=DIR configure option, which is used
to specify the installation location of C header files, did not work
correctly as the path was hard-coded in config/commence.am. I'm presuming
this is because an older version of automake didn't know where to put
c header files. In any case, removing this line now defaults the includedir
to the same directory that it is currently hard-coded to, and also fixes
the configure flag to allow customization of this value.
Tested:
jam, liberty
Bug Fix
Description:
Removing the code from configure which strips the '-g' flag from CFLAGS
when in production mode. The current default CFLAGS in production mode
does not include '-g', as intended, but we should allow users to
override this and enable '-g' by setting the CFLAGS environment variable
if desired. Note that this applies to FCFLAGS and CXXFLAGS as well.
Tested:
kagiso, linew, liberty
Description:
In some situations it was possible for the fill value to not be written to parts
of a chunked dataset, particularly when extending and/or shrinking. Prior to
the fix for the chunk cache (1015) these bugs would have been exceedingly rare.
Tested: jam, smirom, linew (h5committest)
Description:
The meaning of the "nbytes" field in H5D_rdcc_t was not clear, and some places
assumed it was the maximum size of the chunk cache, while some assumed it was
the current size of the chunk cache. The end result was that only 1 chunk could
be held in cache at a time. This field has been replaced by "nbytes_max" and
"nbytes_used". Performance of cached I/O should improve greatly.
Tested: jam, smirom (h5committest)
file handles.
Description:
An attribute's "oloc" field which specifies the file it resides in was located
in the attribute's "shared" structure. So when an attribute was opened multiple
times all of the handles for that attribute pointed to the same file id, even if
different file id's were used to open the different handles for the attribute.
The "oloc" has been moved to the top level H5A_t struct.
Tested: jam, smirom (h5committest)
Description:
Since the new object header format, it has been possible for a situation to be
created where none of the messages are large enough to hold a continuation
message and there are no null messages to merge with. This makes it impossible
to add a new object header chunk. This case will now be handled by moving every
message in the last chunk to the newly allocated one, except for null messages
which are deleted.
Tested: jam, smirom (h5committest)
Description:
When an attribute was created with a datatype or dataspace that was shared in
the same object header that the attribute was in, the attribute could not be
deleted. Changes made to ensure that the attribute can be deleted both when the
attribute is in the object header and when it is shared in the heap. Object
header message decode routines now take an "open_oh" parameter to enable them to
avoid opening the same object header twice.
Tested: jam, smirom (h5committest)
Description:
Attribute object copy routines have been moved from H5Oattr.c to H5Aint.c.
These routines are now shared between compact and densely stored attributes.
New routines written to support the copying of dense attributes. This patch wasmostly written by Peter Cao.
Tested: jam, smirom (h5committest)
Description: When attempting to copy an object with a message shared in its own
object header, the library attempts to protect the same object header twice.
Previously, it was possible for the object header to be protected with write
access in one or both of these protects, which would be illegal. The library
should now always protect with read only access in this case. The conditions
for fixing incorrect datatype versions have been made weaker to support this
change. The version will only be corrected if the object header the datatype
is in is modified.
Tested: jam, smirom (h5committest)
Description: H5Tset_order will now properly reject H5T_ORDER_NONE for most
datatypes. Previously this could cause major problems as the file could not be
flushed.
Tested: jam, smirom (h5committest)
Description:
If a compound type was packed except for some extra space at the end, H5Tpack
would not modify the type and the extra space would remain. Changed
H5T_is_packed to fix this behaviour.
Tested: jam, smirom (h5committest - linew down)
Adding code to maintain a min_clean_fraction in the cache in serial mode.
Description:
The metadata cache now has the ability to maintain a min_clean_fraction
when in serial mode. The default initial cache size has been changed
from 1MB to 2MB, and the default min_clean_fraction has been set at 30%.
This check-in includes modifications to H5C.c to support maintaining a
min_clean_size, including the addition of clean_index_size and
dirty_index_size trackers, modifications to the H5C_make_space_in_cache
algorithm, as well as associated test code and additional statistics
tracking variables.
Maintaining the min_clean_fraction addresses the possibility of
experiencing a "metadata blizzard" when the cache gets completely
full with dirty entries. Upon having to make space, the cache would
previously need to flush every single entry in the cache before coming
across a clean entry which could be evicted. This resulted in unnecessary
flushing of oftentimes hot entries in the cache. Maintaining the
min_clean_fraction ensures that, when space is needed, clean entries
are more readily available to evict.
Tested:
jam, smirom, linew (h5committest)
- h5repack: When user doesn't specify a chunk size, h5repack now defines a default
chunk size as the same size of the size of the hyperslab used to read the chunks.
The size of the hyperslabs are defined as the size of each dimension or a
predefined constant, whatever is smaller. This assures that the chunk
read fits in the chunk cache. (PVN - 2008/11/21)
- H5Dset_extent: when shrinking dimensions, some chunks were not deleted.
(PVN - 2009/01/8)
Description: Added H5Pset/get_elink_cb to allow the user to specify a callback
function to be called whenever an external link is traversed. Added
H5Pset/get_elink_acc_flags to allow the user to specify the file access flags
to use to open the target file of an external link. All these properties are set on a LAPL.
Tested: jam, linew, smirom (h5committest)
Bug Fix
Description:
Resolved issues with broken sanity checks that were failing when the 'b2'
debugging package was enabled (v2 b-tree debugging):
- Fixed outdated assertions that would not compile. They now compile
and function correctly.
- Removed too far outdated assertions that would require too much
unnecessary refactoring of code in order to satisfy their
initial purpose.
- Re-enabled b2 debugging package when "--enable-debug=all" is
specified on the configure line.
Tested:
jam, smirom, linew
Description: Fixes a problem in H5F_close_mounts where it wouldn't correctly
reshape the "child" array when unmounting files. Test added for this case.
Also fixed a potential bug in H5F_unmount where that routine reshapes the child
array.
Tested: kagiso linew smirom (h5committest)
Description: H5Iis_valid function added. This function checks if the id is
valid (i.e. it is currently registered and visible to the application), and
returns TRUE if it is valid, or FALSE otherwise. It should never return an
error.
Tested: kagiso, linew, smirom (h5committest)
Description: H5Ovisit and H5Ovisit by name will now check for a positive return
value from the first callback. Test added for this case.
Tested: kagiso
Description:
In June, we discovered a bug in Windows which only affected specific timezones with the mtime test. This bug was promptly fixed, but never documented. This checkin adds a note to the release document.
Description:
- Remove need to set LD_LIBRARY_PATH when using shared szip library.
- Libtool 2.2.6a is now used to generate libraries.
- 'make check install' dependency bug is fixed, and should no longer
break the build.
- removed hard coding of shell in config/commence.am, as this causes
problems on Solaris with the new version of libtool.
- RELEASE.txt with appropriate changes.
Tested:
- kagiso, smirom, linew (merged from 1.8, pretty quick tests)
Description: Added a new function, H5O_alloc_shrink_chunk, which removes all
null messages from a chunk and shrinks the chunk appropriately. Modified
H5O_merge_null to call this function when a null message is created with a size
>= 64k. Tests added for this functionality.
Tested: kagiso (h5committest on 1.8 version)
Description: Improved external link traversal of h5dump. h5dump will now
properly avoid all cycles, even those spanning multiple files. Improvement
to the output of committed datatypes. Committed datatypes are now checked
for uniqueness (like other objects). Tests added for these cases.
Tested: kagiso, linew, smirom (h5committest)
Various bug fixes & cleanups with VFDs and file free space changes,
along with cleaning up compiler warnings.
Enable more VFDs (for 'make check-vfd' testing) for many tests.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Bring file free space branch changes through r15795 into trunk, which
includes a fair bit of code cleanup & rearrangement along with a couple of
bug fixes also.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Description: Fixes a possible datatype id leak that could occur during compound
datatype conversion, or more precisely, when unregistering those conversions.
Datatype ids normally registered by the library are no longer visible to the
application via H5Fget_obj_ids and H5Fget_obj_count.
Tested: kagiso, linew, smirom (h5committest)
Description: The optimized "subset" compound conversion function would
improperly copy the "blank" space at the end of compound types. Modified
H5T_conv_struct_init to detect when the subset type has extra space at the end,
and calculate the size of the data that should be copied into the conversion
buffer for each element. Changes to the functions that implement these
conversions.
Tested: kagiso, linew, smirom (h5committest)
Description: Adds capability to h5ls to traverse external links when the -r
(recursive) option is given. Changes to the way absolute path names are patched
in h5trav.c. Changes to the way recursive traversal starting from a non-root
group is handled (which also fixes some preexisting issues). Tests added for
these cases.
Tested: kagiso, smirom, linew (h5committest)
Description: Added configure test to see if pointer alignment restrictions are enforced (as in dereferencing an unaligned pointer causes an error). Added code in H5Tvlen.c to avoid dereferencing unaligned pointers, conditionally compiled based on the configure test. Added test case in dtypes.c which would previously cause such machines to fail.
Tested: kagiso, smirom, linew (h5committest); linew64
> 1.
Description: Added a new field 'app_count' to H5I_id_info_t struct, to track
the reference count on an id due to the application. the old 'count' field
tracks the total. Generally any id visible to the application gets placed
in app_count. Added app_ref boolean parameter to H5I_inc_ref, H5I_dec_ref,
H5I_register, H5I_clear_type, and a few other functions, to specify whether
the operation(s) being performed on the id(s) are due to the application
(TRUE) or not (FALSE). Test added for this case.
Tested: kagiso, smirom, linew (h5committest)
datatype versions are encountered.
Description: The library now recognizes some problems with datatype versions in
H5O_decode_helper(), and, if not performing strict format checks, automatically
corrects them. Framework added for other message decode routines to
automatically correct file errors. Datatype version information added to
h5debug.
Tested: kagiso, smirom, linew (h5committest)
Added notes about:
- bugzilla 1068: revising class hierarchy to move Attribute out.
- replacing IdComponent::dereference with new H5Object member functions
and constructors in various other classes.
Description:
Moved mount table from top file structure to shared file structure. Moved
parent out of mount table and back into top file structure. Mounted files can
now be accessed from any handle of the parent file. Changes to how files are
closed. Stricter cycle checking on mounted files. Removed unused function
H5F_has_mount().
Tested:
committest in 1.8 branch. Committing now due to the urgency of the fix. No
changes here are specific to the trunk, but I will keep an eye on the daily
tests.
the file didn't have the data. It happened because each handle had its own
object structure, and the empty one overwrote the data with fill value. This is
fixed by making some attribute information like the data be shared in the
attribute structure.
Tested on smirom, kagiso, and linew.
Fixed bug in h5ls that prevented relative group listings (like
"h5ls foo.h5/bar") from working correctly.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.3 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Description: Removing make target "check-perform". The extra target is unneeded
because the perform tests don't take as long to run as initially
anticipated, and can be lumped in with 'make check'.
Tested: kagiso
Description: The configure option --disable-sharedlib-rpath will disable
embedding of the '-Wl,-rpath' information into executables
when shared libraries are produced.
Tested: kagiso
