Fix BZ #1583
Description:
The --enable-static-exec flag was broken. Configure was adding the
-all-static linker flag to an environment variable which was subsequently
never used. (looks like it was used in 1.6 Makefile.ins, but never
moved to 1.8's Makefile.ams when we added automake support).
To fix, I've added this environment varibale to the link lines of all the
tools. Now, when --enable-static-exec is invoked, it will correctly
generate statically linked executables in the installed bin directory.
Tested:
I've tested manually to ensure that the generated execs are static when
the flag is used and dynamic when not. I've also tested when szip is
used, as this was a use case mentioned in the bug report.
Also: ran full tests with make check on jam and smirom for good measure.
Description:
Previously, when H5Z_filter_info was called for a filter that was not present,
it would succeed and return the information from one position past the end of
the pipeline, possibly causing a segfault. This affected at least
H5Pget_fitler_by_id1/2, and possibly other API functions. Fixed to properly
return failure.
Tested: jam, linew, smirom (h5committest)
Description:
When using H5T_copy on committed datatypes that are already open, H5T_copy would
properly use the already existing shared struct, but would still deep copy all
of the fields in that struct. This would cause memory leaks, and in the case of
a compound containing a vlen (or reference), the change in size would cause the
size of the resulting type to be set to an incorrect value. Changed H5T_copy to
properly avoid deep copies when using a reopened shared struct.
Tested: jam, linew, smirom (h5committest), purify on jam
ISSUE: h5ltread_dataset_string_f causes library assertion with g95.
SOLUTION: convert the fortran string buffer to a C buffer with HD5f2cstring, and pass this string to the C function
TEST: added a test call in the fortran test lite program
DOCS: added the note in RELEASE.txt "- Lite: the h5ltread_dataset_string_f and h5ltget_attribute_string_f functions had memory problems with the g95 fortran compiler. (PVN - 5/13/2009) 1522
Description:
Previously, there was no versioning for H5Z_class_t. This prevented applications
written for 1.6 using custom filters from being able to use the 1.8 library.
There is now an H5Z_class1_t and H5Z_class2_t to enable compatibility. H5Zregister is
*not* versioned, it determines which version of the struct has been passed in by the
value of the first field (id or version, both are ints).
Tested: jam, linew, smirom (h5committest), jam (--with-default-api-version=v16)
Description:
When writing data to a dataset, the data transform was performed after type conversion.
This caused an error if the file type was non-native. This has been changed so data
transforms are always performed on the memory type.
Tested: jam, linew. smirom (h5committest)
Description:
h5repack previously would not take named datatypes into consideration when copying
datasets and attributes. This would cause extra anonymous datatypes in the target file
at best, and cause errors halfway through the repacking at worst. h5repack should now
always handle named datatypes correctly. Named datatypes are also now converted to the
native type when -n is given.
Tested: jam, linew, smirom (h5committest)
- Updated bin/reconfigure to use latest version of automake (1.10.2).
Re-generated Makefile.in's by running bin/reconfigure.
- Added libtool version numbers to c++, fortran, hl, hl c++, and hl fortran
libraries.
Tested:
jam, liberty, smirom
Added a Make target of check-all-install to test the correctness of
installing via the prefix= or $DESTDIR options.
Platforms tested:
h5committested and jam serial mode. Then test check-all-install by hand.
- h5diff new flag, -c, --compare, list objects that are not comparable.
PVN - 2009/4/2 - 1368
- h5diff new flag, -N, --nan, avoids NaNs detection. PVN - 2009/4/2
Description:
The H5L interface was not marked as uninitialized when H5L_term_interface was
called. This caused the interface to not initialize itself later on. This
prevented external links from working after calling H5close, because they were
never re-registered.
Tested: jam, smirom, linew (h5committest)
Description:
Versions of the library between 1.3.0 and 1.6.3 have a bug which prevents them
from opening any file that does not have the root group's symbol table
information cached in the root group's entry in the superblock. Prior to 1.8
this was not an issue as this information was always cached. However, 1.8.0
stopped writing this information (which is not required by the file format
specification), and these older versions can therefore not read files created or
last written by versions 1.8.0 to 1.8.2. This fix modifies the library to once
again add this information to the superblock (when using the old file format).
Tested: jam, linew, smirom (h5committest)
Description:
Changed Skip list package to use a deterministic skip list. This allows the
skip list package to avoid calling rand() and srand(), even on machines without
rand_r(). There is no longer a p-value or maximum level for skip lists.
Tested: jam, smirom, linew (h5committest)
Description:
Previously, H5Lcopy and H5Lmove would (through H5L_move) improperly apply the
"create intermediate groups" property to the source path traversal, and not the
destination. Fixed it to apply the property to the destination and not the
source. Also fixed H5Lcreate_ud to reject internal link classes without
throwing an assertion.
Tested: linew, jam, smirom (h5committtest)
Fortran High-Level APIs:
------
- Lite: The h5ltget_dataset_info_f function (gets information about a dataset)
was not correctly returning the dimension array (PVN - 2009/3/23)
When reading the compression parameter keyword, the compression type read flag was incorrectly set to read, removed this line of code
in->configOptionVector[COMPRESS] = 1;
Modified one configuration file to have the COMPRESSION-TYPE GZIP
Keyword.
Entered a bug description fix of
- h5import: By selecting a compression type, a big endian byte order was being
selected (PVN - 2009/11/3)
tested: linux
Bug Fix
Description:
Fixing BZ #1381. The --includedir=DIR configure option, which is used
to specify the installation location of C header files, did not work
correctly as the path was hard-coded in config/commence.am. I'm presuming
this is because an older version of automake didn't know where to put
c header files. In any case, removing this line now defaults the includedir
to the same directory that it is currently hard-coded to, and also fixes
the configure flag to allow customization of this value.
Tested:
jam, liberty
Bug Fix
Description:
Removing the code from configure which strips the '-g' flag from CFLAGS
when in production mode. The current default CFLAGS in production mode
does not include '-g', as intended, but we should allow users to
override this and enable '-g' by setting the CFLAGS environment variable
if desired. Note that this applies to FCFLAGS and CXXFLAGS as well.
Tested:
kagiso, linew, liberty
Description:
In some situations it was possible for the fill value to not be written to parts
of a chunked dataset, particularly when extending and/or shrinking. Prior to
the fix for the chunk cache (1015) these bugs would have been exceedingly rare.
Tested: jam, smirom, linew (h5committest)
Description:
The meaning of the "nbytes" field in H5D_rdcc_t was not clear, and some places
assumed it was the maximum size of the chunk cache, while some assumed it was
the current size of the chunk cache. The end result was that only 1 chunk could
be held in cache at a time. This field has been replaced by "nbytes_max" and
"nbytes_used". Performance of cached I/O should improve greatly.
Tested: jam, smirom (h5committest)
file handles.
Description:
An attribute's "oloc" field which specifies the file it resides in was located
in the attribute's "shared" structure. So when an attribute was opened multiple
times all of the handles for that attribute pointed to the same file id, even if
different file id's were used to open the different handles for the attribute.
The "oloc" has been moved to the top level H5A_t struct.
Tested: jam, smirom (h5committest)
Description:
Since the new object header format, it has been possible for a situation to be
created where none of the messages are large enough to hold a continuation
message and there are no null messages to merge with. This makes it impossible
to add a new object header chunk. This case will now be handled by moving every
message in the last chunk to the newly allocated one, except for null messages
which are deleted.
Tested: jam, smirom (h5committest)
Description:
When an attribute was created with a datatype or dataspace that was shared in
the same object header that the attribute was in, the attribute could not be
deleted. Changes made to ensure that the attribute can be deleted both when the
attribute is in the object header and when it is shared in the heap. Object
header message decode routines now take an "open_oh" parameter to enable them to
avoid opening the same object header twice.
Tested: jam, smirom (h5committest)
Description:
Attribute object copy routines have been moved from H5Oattr.c to H5Aint.c.
These routines are now shared between compact and densely stored attributes.
New routines written to support the copying of dense attributes. This patch wasmostly written by Peter Cao.
Tested: jam, smirom (h5committest)
Description: When attempting to copy an object with a message shared in its own
object header, the library attempts to protect the same object header twice.
Previously, it was possible for the object header to be protected with write
access in one or both of these protects, which would be illegal. The library
should now always protect with read only access in this case. The conditions
for fixing incorrect datatype versions have been made weaker to support this
change. The version will only be corrected if the object header the datatype
is in is modified.
Tested: jam, smirom (h5committest)
Description: H5Tset_order will now properly reject H5T_ORDER_NONE for most
datatypes. Previously this could cause major problems as the file could not be
flushed.
Tested: jam, smirom (h5committest)
Description:
If a compound type was packed except for some extra space at the end, H5Tpack
would not modify the type and the extra space would remain. Changed
H5T_is_packed to fix this behaviour.
Tested: jam, smirom (h5committest - linew down)
Adding code to maintain a min_clean_fraction in the cache in serial mode.
Description:
The metadata cache now has the ability to maintain a min_clean_fraction
when in serial mode. The default initial cache size has been changed
from 1MB to 2MB, and the default min_clean_fraction has been set at 30%.
This check-in includes modifications to H5C.c to support maintaining a
min_clean_size, including the addition of clean_index_size and
dirty_index_size trackers, modifications to the H5C_make_space_in_cache
algorithm, as well as associated test code and additional statistics
tracking variables.
Maintaining the min_clean_fraction addresses the possibility of
experiencing a "metadata blizzard" when the cache gets completely
full with dirty entries. Upon having to make space, the cache would
previously need to flush every single entry in the cache before coming
across a clean entry which could be evicted. This resulted in unnecessary
flushing of oftentimes hot entries in the cache. Maintaining the
min_clean_fraction ensures that, when space is needed, clean entries
are more readily available to evict.
Tested:
jam, smirom, linew (h5committest)
- h5repack: When user doesn't specify a chunk size, h5repack now defines a default
chunk size as the same size of the size of the hyperslab used to read the chunks.
The size of the hyperslabs are defined as the size of each dimension or a
predefined constant, whatever is smaller. This assures that the chunk
read fits in the chunk cache. (PVN - 2008/11/21)
- H5Dset_extent: when shrinking dimensions, some chunks were not deleted.
(PVN - 2009/01/8)
Description: Added H5Pset/get_elink_cb to allow the user to specify a callback
function to be called whenever an external link is traversed. Added
H5Pset/get_elink_acc_flags to allow the user to specify the file access flags
to use to open the target file of an external link. All these properties are set on a LAPL.
Tested: jam, linew, smirom (h5committest)
Bug Fix
Description:
Resolved issues with broken sanity checks that were failing when the 'b2'
debugging package was enabled (v2 b-tree debugging):
- Fixed outdated assertions that would not compile. They now compile
and function correctly.
- Removed too far outdated assertions that would require too much
unnecessary refactoring of code in order to satisfy their
initial purpose.
- Re-enabled b2 debugging package when "--enable-debug=all" is
specified on the configure line.
Tested:
jam, smirom, linew
Description: Fixes a problem in H5F_close_mounts where it wouldn't correctly
reshape the "child" array when unmounting files. Test added for this case.
Also fixed a potential bug in H5F_unmount where that routine reshapes the child
array.
Tested: kagiso linew smirom (h5committest)
Description: H5Iis_valid function added. This function checks if the id is
valid (i.e. it is currently registered and visible to the application), and
returns TRUE if it is valid, or FALSE otherwise. It should never return an
error.
Tested: kagiso, linew, smirom (h5committest)
Description: H5Ovisit and H5Ovisit by name will now check for a positive return
value from the first callback. Test added for this case.
Tested: kagiso
Description:
In June, we discovered a bug in Windows which only affected specific timezones with the mtime test. This bug was promptly fixed, but never documented. This checkin adds a note to the release document.
Description:
- Remove need to set LD_LIBRARY_PATH when using shared szip library.
- Libtool 2.2.6a is now used to generate libraries.
- 'make check install' dependency bug is fixed, and should no longer
break the build.
- removed hard coding of shell in config/commence.am, as this causes
problems on Solaris with the new version of libtool.
- RELEASE.txt with appropriate changes.
Tested:
- kagiso, smirom, linew (merged from 1.8, pretty quick tests)
Description: Added a new function, H5O_alloc_shrink_chunk, which removes all
null messages from a chunk and shrinks the chunk appropriately. Modified
H5O_merge_null to call this function when a null message is created with a size
>= 64k. Tests added for this functionality.
Tested: kagiso (h5committest on 1.8 version)
Description: Improved external link traversal of h5dump. h5dump will now
properly avoid all cycles, even those spanning multiple files. Improvement
to the output of committed datatypes. Committed datatypes are now checked
for uniqueness (like other objects). Tests added for these cases.
Tested: kagiso, linew, smirom (h5committest)
Various bug fixes & cleanups with VFDs and file free space changes,
along with cleaning up compiler warnings.
Enable more VFDs (for 'make check-vfd' testing) for many tests.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Bring file free space branch changes through r15795 into trunk, which
includes a fair bit of code cleanup & rearrangement along with a couple of
bug fixes also.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Description: Fixes a possible datatype id leak that could occur during compound
datatype conversion, or more precisely, when unregistering those conversions.
Datatype ids normally registered by the library are no longer visible to the
application via H5Fget_obj_ids and H5Fget_obj_count.
Tested: kagiso, linew, smirom (h5committest)
Description: The optimized "subset" compound conversion function would
improperly copy the "blank" space at the end of compound types. Modified
H5T_conv_struct_init to detect when the subset type has extra space at the end,
and calculate the size of the data that should be copied into the conversion
buffer for each element. Changes to the functions that implement these
conversions.
Tested: kagiso, linew, smirom (h5committest)
Description: Adds capability to h5ls to traverse external links when the -r
(recursive) option is given. Changes to the way absolute path names are patched
in h5trav.c. Changes to the way recursive traversal starting from a non-root
group is handled (which also fixes some preexisting issues). Tests added for
these cases.
Tested: kagiso, smirom, linew (h5committest)
Description: Added configure test to see if pointer alignment restrictions are enforced (as in dereferencing an unaligned pointer causes an error). Added code in H5Tvlen.c to avoid dereferencing unaligned pointers, conditionally compiled based on the configure test. Added test case in dtypes.c which would previously cause such machines to fail.
Tested: kagiso, smirom, linew (h5committest); linew64
> 1.
Description: Added a new field 'app_count' to H5I_id_info_t struct, to track
the reference count on an id due to the application. the old 'count' field
tracks the total. Generally any id visible to the application gets placed
in app_count. Added app_ref boolean parameter to H5I_inc_ref, H5I_dec_ref,
H5I_register, H5I_clear_type, and a few other functions, to specify whether
the operation(s) being performed on the id(s) are due to the application
(TRUE) or not (FALSE). Test added for this case.
Tested: kagiso, smirom, linew (h5committest)
datatype versions are encountered.
Description: The library now recognizes some problems with datatype versions in
H5O_decode_helper(), and, if not performing strict format checks, automatically
corrects them. Framework added for other message decode routines to
automatically correct file errors. Datatype version information added to
h5debug.
Tested: kagiso, smirom, linew (h5committest)
Added notes about:
- bugzilla 1068: revising class hierarchy to move Attribute out.
- replacing IdComponent::dereference with new H5Object member functions
and constructors in various other classes.
Description:
Moved mount table from top file structure to shared file structure. Moved
parent out of mount table and back into top file structure. Mounted files can
now be accessed from any handle of the parent file. Changes to how files are
closed. Stricter cycle checking on mounted files. Removed unused function
H5F_has_mount().
Tested:
committest in 1.8 branch. Committing now due to the urgency of the fix. No
changes here are specific to the trunk, but I will keep an eye on the daily
tests.
the file didn't have the data. It happened because each handle had its own
object structure, and the empty one overwrote the data with fill value. This is
fixed by making some attribute information like the data be shared in the
attribute structure.
Tested on smirom, kagiso, and linew.
Fixed bug in h5ls that prevented relative group listings (like
"h5ls foo.h5/bar") from working correctly.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.3 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Description: Removing make target "check-perform". The extra target is unneeded
because the perform tests don't take as long to run as initially
anticipated, and can be lumped in with 'make check'.
Tested: kagiso
Description: The configure option --disable-sharedlib-rpath will disable
embedding of the '-Wl,-rpath' information into executables
when shared libraries are produced.
Tested: kagiso
Description: new autotool version information was missing from 1.9
documentation. It was added to 1.8, but I forgot to put it in
the trunk when I did that update. It's there now.
New versions: Automake 1.10.1, Libtool 2.2.2
Tested: none needed, doc update only.
Description: Tests in perform directory were never getting run, and
h5perf* programs were not being installed.
Solution: Added another build option, 'make check-perform', which runs the
tests in the perform directory. Also modified the Makefiles in the
perform directory to install (with 'make install') h5perf when
parallel is enabled, and h5perf and h5perf_serial when parallel
is disabled.
Tested: kagiso, smirom, linew
Bring r14737 back from the 1.8 branch: Fix bug which would
incorrectly encode the member offsets for compound datatypes whose size was
between 256 & 511 bytes, when the "use the latest format" feature was enabled.
Tested on:
Mac OS X/32 10.5.2 (amazon) w/debug
FreeBSD/32 6.2 (duty) w/production
Tools
-----
- h5import bug on Windows w/binary datasets. fread in windows needs a binary file
to be open with rb instead of r otherwise it terminates execution if an end
of file character is found on the input file. Besides that the binary file
generated needs to be open with wb , otherwise an end of line character is
read twice. PVN - 2008/2/19
Removed the stream-vfd from the basic library code.
(The stream-vfd source files are not removed yet but the MANIFEST has been
updated to NOT release those stream-vfd source files.)
Platforms tested:
Kagiso and smirom.
Then test the release tar ball in kagiso.
Tools:
------
- h5dump lists groups and attributes in requested orders (by name and
creation order, both ascending and descending). PVN - 2007/10/5
- h5import imports string (text) data. PVN - 2007/10/5
Description:
Updated RELEASE.txt to reflect support for 64-bit Windows. Also updated the testing table, and removed some old Windows-related issues that are no longer present.
No testing needed.
Change back to clearing memory buffers by default, but give users option
to disable this, for performance reasons (--disable-clear-file-buffers).
Also, add configure option (--enable-using-memchecker) to conveniently
control making the library "play nice" with memory checking tools like valgrind
and purify.
Tested on:
Mac OS X/32 10.4.9 (amazon)
FreeBSD/32 6.2 (duty)
Linux/32 2.6 (kagiso)
added entries
- Fortran interfaces for the Image, Table and Lite APIs. PVN - 2007/5/1
- New HDF5 Dimension Scale API (H5DS) allows dimension scales to be
created in an HDF5 file and associated with specific datasets.
PVN - 2007/5/1
Updated for 1.8.0-beta1.
Updated URLs from hdf.ncsa.uiuc.edu
to www.hdfgroup.org or www.hdfgroup.uiuc.edu, as appropriate
and corrected otherwise-out-of-date URLs.
Updated HDF Help Desk address to 'help@hdfgroup.org'.
Added notes regarding beta documentation.
And here were a few minor edits.
Tested:
Visual inspection.
-This line, and those below, will be ignored--
M RELEASE.txt
Change H5[D|G|T]<foo>_expand() "temporary" API routines to
H5[D|G|T]<foo>2() "versioned" routines. Also added
H5[D|G|T](create|commit)_anon() routines to continue to allow "anonymous"
objects to be created in a file.
Tested on:
Mac OS X/32 10.4.9 (amazon)
FreeBSD/32 6.2 (duty)
FreeBSD/64 6.2 (liberty)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add mention of the h5copy & h5mkgrp tools to the release notes.
Update documentation for the "--private" flag in release script
Tested on:
None - just text/comment changes
Relax restriction on the "location ID" for attribute operations to allow
file IDs to be used as the location ID. If a file ID is used, the attribute
operations will occur on the file's root group.
Tested on:
FreeBSD/32 6.2 (duty)
Added the WARNING messages of unstable temporary formats.
For README.txt,
In addition, also updated website name to hdfgroup.org, updated email addresses
new names and new subscription instructions.
Platforms tested:
Eyeballed since these are just information files.
Add tests for H5Oopen_by_idx() (in a rare fit of superlative coding I
apparently got the implementation complete correct in my earlier checkin :-)
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add support for "delete by index" to "old-style" groups, finishing
implmentation of H5Ldelete_by_idx() routine.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Aix/32 5.? (copper)
Add new H5Lget_val_by_idx() routine & tests.
Also includes most of changes for H5Ldelete_by_idx() routine.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
AIX/32 5.? (copper)
"make check-vfd" will now run all tests in the test directory with different
file drivers (at least, all of those tests that use the testing framework's
FAPL). Tests that fail will be skipped.
This is not a perfect fix, but is better than nothing.
Along with this change, check-vfd should be added to the Daily Tests.
Description: Added support for HPUX11.23
Solution: Added configuration file for the system.
Both C, C++, F90, HL 32-bit and 64-bit are supported. Known problems:
C++ library will not build in 64-bit mode if shared is enabled.
Platforms tested: HPUX 11.23 at UMKC
Misc. update:
Configuration feature
Description:
'make install' now tests both static and shared libraries if both are installed.
Solution:
Previously, shared libraries were only tested when static libraries were not installed.
Also cleaned up line in commence.am that was including HL library in all Makefiles.
Platforms tested:
mir (Makefile change only)
