Description: There was a bug in the calculating code of the Fletcher32
checksum in the library before v1.6.3. The checksum value wasn't consistent
between big-endian and little-endian systems. This bug was fixed in
Release 1.6.3. However, after fixing the bug, the checksum value is no
longer the same as before on little-endian system.
Solution: Made the library compare both the correct checksum and incorrect
checksum generated from v1.6.2 or before. This makes the library be
backward compatible but not forward compatible.
Platforms tested: h5committest and fuss.
Misc. update: Documented the forward incompatibility problem in the
Known Problem section in RELEASE.txt
Modification on .Net and Intel part
Description:
In batch mode, there should be no options to choose build Only HDF5 C or both C and C++ library
with visual studio .net or intel compiler for the time being
Solution:
Platforms tested:
Misc. update:
Bug fix & code cleanup
Description:
Don't unmount child files from a parent file if the parent file is
being held open by an object.
Also, moved the mounting routines into their own source file and the
superblock routines into their own file also.
Solution:
Moved the code for unmounting child files down in the H5F_close routine
and also add checks to the file and group mounting code to unmount child
files when the last object holding open a file closes.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Bug fix
Description:
Fixed bugs that caused tests to be unable to find testhdf5.sh during
daily tests, and another that broke sol and shanti.
Solution:
When tests are run, their path is ./$testname .
When scripts are run, their path has no leading "./". Fixed.
Sol and shanti don't recognize the -e option for test, but they do know
-f. Fixed.
Also modified release notes.
Platforms tested:
mir, shanti, heping
Bug fix
Description:
When a series of files is mounted on one another and one of those files
is not unmounted, the library gets confused at shutdown and goes into an
infinite loop in the file interface.
Solution:
If there are still files left in the "closing" state when shutting down
the file API, iterate over those file IDs and unmount any child files that we
find mounted on those files.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Description: First, report a bug fix(#214) of family driver. Second
document PGI compiler bug for optimization on AMD processor.
Misc. update: First, report a bug fix(#214) of family driver. Second
document PGI compiler bug for optimization on AMD processor.
Description:
When compiler flag was used to set the size of Fortran integer
to 8 bytes, library would fail.
Solution:
Cleaned up the code; added detection of Fortran INTEGER type size
and appropriately defined int_f type for C-stubs routines.
Platforms tested:
Solaris 2.8 32 and 64-bit, AIX 5.1 64-bit parallel
PGI Fortran with -i8 flag on heping
Absoft Fortran with -i8 flag on heping
g95 on mir (Fortran integer is 8 bytes by default that cannot be
changed - compiler bug ;-)
AIX Fortran with -qintsize=8 32 and 64-bit modes on copper
Misc. update:
Bug fix
Description:
Hyperslab selections that had a selection offset and were applied to a
chunked dataset could get into an infinite loop or core dump if the same
selection was used multiple times, with different selection offsets.
Solution:
"Normalize" the selection with the selection offset, generate the
selections for the chunks overlapped and then "denormalize" the selection.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Add check for opaque tags that are too long for file format to handle
currently.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5commmitest
File was split into three parts.
Description:
The short version was split into three parts three compilers
MSVS 6.0 MSVS .NET and Intel Compiler
Solution:
Platforms tested:
Misc. update:
Add short version instruction for users using Intel Compiler
Description:
Add a short version instruction for building and testing HDF5
with Intel C and Fortan compiler on windows XP.
Solution:
Platforms tested:
Misc. update:
Add short version instruction for Visual C++ .net 2003
Description:
Short version instruction for building and testing HDF5
with Visual C++ .Net was added.
Solution:
Platforms tested:
Misc. update:
Separate short version instruction for different compilers.
Description:
Add short version instruction for MSVS 6.0
Solution:
Platforms tested:
Misc. update:
Bug fix
Description:
Fix several NULL pointer dereferences when tracking the name of open
objects during some unusual sequences of mount & unmount operations.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix/new feature
Description:
Setting "SEMI" or "STRONG" file close degrees causes problems when multiple
file IDs exist for the same file on disk.
Solution:
Make the "SEMI" and "STRONG" settings only apply to the file ID that is
being closed.
Also, add an "H5F_OBJ_LOCAL" flag for the H5Fget_obj_count() &
H5Fget_obj_ids() calls, so that applications can query about objects opened
with a particular file ID instead of all the objects opened in the file on
disk.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Description: See details from Bug #213. Family member file size wasn't saved
anywhere in file. When family file is opened, the first member size determine
the member size.
Solution: This is the third step of checkin. h5repart has been modified. If h5repart is used
to change the size of family member file, the new size(actual member size) is saved in the superblock.
In the second step of checkin, multi driver is checked against the driver
name saved in superblock. Wrong driver will result in a failure with an error message indicating
multi driver should be used. This change includes split driver because it's a special case for multi
driver.
In the first step of checkin. Family member size and name template(unused at this stage) are saved
in file superblock. When file is reopened,the size passed in thrin superblock. A different size
will trigger a failure with an error message indicating the right size. Wrong driver to open family
file will cause a failure, too.
Platforms tested: h5committest and fuss
Misc. update: RELEASE.txt
Feature
Description:
Added "support" for UTF-8 character encoding.
Solution:
Wrote tests to check that UTF-8 can be used in a number of places in
HDF5 (object names, data, etc.). These tests live in test/tunicode.c.
Added a new UTF-8 character encoding for datatypes.
Platforms tested:
mir, modi4, heping
Misc. update:
Description: h5pget_driver_f function returned information that could not
be interpreted by fortran application
Solution: Defined Fortran global variables that correspond to
the C H5FD_<driver_name> variables at the Fortran library
initialization time.
Platforms tested: heping with PGI compilers, sol 64-bit and copper parallel
Misc. update:
Description: Removed support for SRB driver
Solution: Removed or modified appropriate files; ran reconfigure
to regenerate Makefile.in and configure files.
Platforms tested: heping and shanti
Misc. update: ran bin/chkmanifest on heping
Description: Added missing h5get_member_class_f function
Solution:
Platforms tested: heping; it is a simple change. I will
watch daily tests and take care of failures.
Misc. update:
Update documents about installing hdf5 on windows xp
Description:
Modified files: INSTALL_Windows.txt INSTALL_Windows_From_Command_Line.txt
INSTALL_Windows_Short.txt
Solution:
Platforms tested:
Misc. update:
Purpose:
Added C++ wrapper for Packet Table API.
Description:
Added macro for high-level C++ library (LIBH5CPP_HL), which changes every
Makefile.in.
Added directories for high-level C++ library (though currently only Packet
Table API is supported).
Added both C++ source and tests.
Platforms tested:
sleipnir, mir, modi4
Misc. update:
Optimization
Description:
Speed up I/O on enumerated datatypes (including those nested in compound
datatypes, arrays, etc.) if the destination datatype is a proper superset of
the source datatype.
Solution:
Detect the situation and treat as no-op datatype conversion.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Automake update cleanup and minor changes
Description:
Removed macros in acsite.m4 that are no longer used.
Switched to using autoconf's AC_LANG_PUSH(Fortran) instead of old
AC_LANG_FORTRAN9X macro.
Switched to using AC_LANG_PUSH() and AC_LANG_POP() (from old AC_LANG_X).
Added ifort to list of Fortran compilers configure will look for.
Added a note about automake change to Release Notes.
Platforms tested:
copper, modi4 (parallel and serial), eirene
Description: Support of hardware conversion between "long double" and integers(mainly in
H5Tconv.c) and test cases for it(mainly in dtypes.c)
Platforms tested: h5committest and fuss
Misc. update: RELEASE.txt
Update.
Description:
Update installation file on how to link HDF5 with Zlib 1.2.2 and Szip 2.0 DLL libraries.
Solution:
Platforms tested:
Misc. update:
Description: H5Tget_member_value didn't return correct value if called after
H5Tenum_valueof. It's because there's a sorting on the members of enum type in
H5Tenum_valueof which changed the order of members.
Solution: Made a copy of original type and do sorting on it to protect the
original order.
Platforms tested: fuss; tested v1.6 with h5committest
Misc. update: RELEASE.txt
Description: For variable-length string, H5Tget_class returned H5T_STRING as
its class. But H5Tdetect_class and H5Tget_member_class considered it as
H5T_VLEN. This is fixed to let all these 3 functions treat it as H5T_STRING.
Some test cases have been added to dtypes.c
Platforms tested: heping - already tested for v1.6 with h5committest
Misc. update: RELEASE.txt
Description: The byte order for all 1-byte integer types was fixed as little-endian
even on a big-endian machine. This's corrected in h5detect.c. When types are only
1 byte long, a native int is used substitute the type to detect byte order. Some tools
like h5dump and h5repack are also corrected in this case.
Platforms tested: fuss, copper, sol.(There're some failures from the recent configure
change).
Misc. update: Information in the RELEASE.txt.
Description: Removed PABLO from the source
Solution:
Platforms tested: arabica with 64-bit, copper with parallel,
heping with GNU C and C++ and PGI fortran (but
I disabled hl, there is some weird problem only
on heping: F9XMODFLAG is not
propagated to the Makefile files
Misc. update:
Description: Start to support software conversion between long double and
all integers.
Solution: No major changes to the algorithm. Changes to configure is to
exclude SGI for long double to integers test because there're too many problems
in their compiler.
Platforms tested: h5committest, modi4, fuss, Teragrid, arabica
Misc. update: RELEASE.txt
Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-)
Description:
Generally speaking, this is the "signed->unsigned" change to selections.
However, in the process of merging code back, things got stickier and stickier
until I ended up doing a big "sync the two branches up" operation. So... I
brought back all the "infrastructure" fixes from the development branch to the
release branch (which I think were actually making some improvement in
performance) as well as fixed several bugs which had been fixed in one branch,
but not the other.
I've also tagged the repository before making this checkin with the label
"before_signed_unsigned_changes".
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & fphdf5
FreeBSD 4.10 (sleipnir) w/threadsafe
FreeBSD 4.10 (sleipnir) w/backward compatibility
Solaris 2.7 (arabica) w/"purify options"
Solaris 2.8 (sol) w/FORTRAN & C++
AIX 5.x (copper) w/parallel & FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN
Linux 2.4 (heping) w/FORTRAN & C++
Misc. update:
Description: INSTALL file in 1.7 branch had old information about
the minimum set of header files that should be installed
Solution: Fixed
Platforms tested: N/A
Misc. update:
Update.
Description:
Update HDF5 Windows installation documentation to include recently added HDF5 high level library and tests information.
Solution:
Platforms tested:
Not necessary.
Misc. update:
Bug fix
Description:
Correct values used in "space utilization" equation: the 'used' and
'total' values were backwards.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Fix core dump when flushing a file with a newly created attribute which
hasn't had a value written to it still open.
Solution:
Write the attribute fill value when appropriate.
Platforms tested:
FreeBSd 4.10 (sleipnir)
Linux 2.4 (verbena)
Solaris 2.7 (arabica)
Purpose: change feature
Description: Back up support bitfield and time datatypes in H5Tget_native_type.Leave it to future support. Let it return "not supported" error message for
now.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Bug fix
Description:
Relax restrictions on parallel I/O to allow compressed, chunked datasets
to be read in parallel (collective access will be degraded to independent
access, but will retrieve the information still).
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
IRIX64 6.5 (modi4)
h5committest
Description: enable size adjustment for compound datatype. The size
can be increased and decreased(as long as the last member isn't cut).
Solution: mainly check if the last member is being cut when decreasing
the size. Others are simply taking out assertion to enable this feature.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Bug fix.
Description:
Fix off-by-one error in Core VFL driver which would cause spurious address
or size overflow errors when an odd valued address or size was checked.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Linux 2.4 (heping)
Solaris 2.7 (arabica)
Update release_docs/RELEASE.txt to describe options for dealing
with installations which insert extra text in output, thereby
breaking tests which function by comparing output with an exemplar.
Description:
See above.
Solution:
Add discussion of this possible problem, and listed two possible
solutions:
1) "setenv HDF5_Make_Ignore yes" and then score "failed" tests
by hand.
2) Use the RUNSERIAL environment variable to use a filter script
to filter out the extra text.
Platforms tested:
No testing needed.
Misc. update:
Bug fix
Description:
Correct typedef for dataset region references to avoid struct alignment
issues on Crays.
Solution:
Change the typedef for hdset_reg_ref_t from a struct to an array of
unsigned char's of the correct size and propagate the appropriate adjustments
around the code.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Linux 2.4 (verbena) w/fortran
Cray T90 (subzero) w/fortran
Cray SV1 (wind) w/fortran & parallel
Cray T3E (cyclone) w/fortran & parallel
Description: Restore 6 old error API functions back to the library to be backward
compatible with v1.6. They are H5Epush, H5Eprint, H5Ewalk, H5Eclear, H5Eset_auto,
H5Eget_auto. These functions do not have error stack as parameter.
Solution: Internally, these functions use default error stack.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Bug fix (sorta)
Description:
Change reading of "missing" chunks from datasets with undefined fill
values to not overwrite the application buffer with random garbage from
memory. Note that this is almost the same, since whatever garbage the
application had in those locations will still be there...
Platforms tested:
FreeBSD 4.10 (sleipnir)
IRIX64 6.5 (modi4)
h5committested
Description: Added PGI Fortran support for Linux64 systems (x86_64)
Solution:
Platforms tested: AMD Opteron box at AMD Sunnyvale and verbena
with PGI and gcc compilers.
Misc. update:
Bug fix
Description:
Correct possible core dump when a datatype conversion function is
registered with the library after a compound datatype has been converted
(having it's type conversion information cached by the library). The compound
datatype must have been created by inserting the fields in non-increasing
offset order to see the bug.
Solution:
Re-sort the fields in the compound datatypes before recalculating the
cached information when performing the conversion on them.
Platforms tested:
FreeBSD 4.10 (sleipnir)
h5committested
Bug fix
Description:
Correct buffer overrun in "multi" VFL driver that was writing past the
end of the "driver name" buffer when encoding the driver info block for the
file's superblock.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Too minor to require h5committest
Bug fix.
Description:
Address two problems:
- The computation of the scanline in the szip filter was being
performed in the "can apply" callback routine instead of the
"set local" routine.
- The routine which allocated all the chunks for an entire dataset
(which is invoked when the allocation time is early or late,
rather than incremental) wasn't recording a failed filter in
the information for the chunk, causing the library to believe
that the chunk had the filter applied when it really hadn't.
Solution:
- Move the scanline computation to the "set local" callback.
- Record the filter mask with each chunk created when allocating them.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/szip
Too obscure to require h5committest
Description: Added support for Absoft Fortran compiler
Solution: Modified configuration file to check which Fortran compiler is used
and set appropriate flags.
Platforms tested: verbena with pgf90 and Absoft f95 compilers
Misc. update:
Bug fix
Description:
Allow I/O on extendible chunked datasets with (currently) zero-sized
dimensions to proceed harmlessly instead of dumping core on an assertion.
Solution:
Removed assertion and added checks to avoid problem situation in H5TB_end
Platforms tested:
FreeBSD 4.10 (sleipnir) w/ & w/o parallel
Too minor to require h5committest
Bug fix
Description:
Always write fill values to chunks when initializing entire B-tree and
any filters are defined.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
Too minor to require h5committest
Purpose: Maintenance for MAC OSX
Description: Added support for Absoft Fortran compiler f95;
Ddefault compiler is set to IBM xlf.
Solution:
Platforms tested: pommier with xlf and Absoft f95 compilers
Misc. update:
Code cleanup, sorta
Description:
Added ifdef sections for "H5_USING_PURIFY" in various places in the code,
which are designed to reduce the spurious "uninitialized memory read" warnings
from purify which are actually OK. Note that this macro will have to be
turned on by adding it to the CFLAGS for the build - I didn't think it was
important enough to add a configure flag for.
Also, the changes in H5HG.c optimize the walks through the objects in a
heap to only look at the 'used' entries instead of all the 'allocated' entries.
Platforms tested:
Solaris 2.7 (arabica) w/purify
Not tested by h5committest
Code cleanup
Description:
Fixed reserved.c test to use h5_fileaccess/h5_fixname/h5_cleanup.
Updated RELEASE.txt for previous bug fix
Platforms tested:
sleipnir, verbena
Description: This is the second step of checkin for encoding and decoding objects.
H5Tencode and H5Tdecode have been committed in the previous step. H5Sencode
and H5Sdecode are checked in this time.
Solution: Given object ID, these functions encode and decode object information
into and from binary buffer and return new object ID. They take advantage of the
existing codes of object header message and encode in the same format.
Platforms tested: fuss and h5committest.
Misc. update: RELEASE.txt
Bug fix
Description:
Allow buffer parameter to H5Dread & H5Dwrite to be NULL if there are no
elements to transfer.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Purpose:
Bug Fix
Description:
Trying to create the root group or the working group ("/" or ".") fakes out
HDF5 so that it neither creates a group nor returns an error.
Solution:
H5G_namei now throws an error if it was supposed to insert but didn't.
Platforms tested:
sleipnir, Visual Studio 7 (very minor change)
Misc. update:
Bug fix
Description:
Correct problems with "resurrecting" a dataset in a file. (This occurs
when a dataset which is open gets unlinked from the group hierarchy (making it
"dead" and marked for deletion in the file) and then is re-linked to the group
hierarchy). Note that the current solution applies only to datasets, further
work will fix this for groups and named datatypes also.
Also, fix the "debug" routines to be a little more helpful in certain
situations.
Additionally, fix a locking bug in the symbol table node splitting routine
which could be [one of] the cause[s] of the file corruption in flexible
parallel operation.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
h5committested
Bug fix
Description:
Fix error in chunked dataset I/O where data written out wasn't read
correctly from a chunked, extendible dataset after the dataset was extended.
Also, fix parallel I/O tests to gather error results from all processes,
in order to detect errors that only occur on one process.
Solution:
Bypass chunk cache for reads as well as writes, if parallel I/O driver is
used and file is opened for writing.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Purpose: Maintenance
Description: Added h5fget_name_f and h5fget_filesize_f subroutines and tests.
Solution: N/A
Platforms tested: arabica (32-bit), sol (64-bit)
parallle build on copper failed for the C library with the the
following error:
ld: 0711-317 ERROR: Undefined symbol: .H5FD_stdio_term
Since this change doesn't affect the C library, I am cheking it in
and will retest the fresh CVS copy after this check-in.
Misc. update:
Purpose:
HDF5 now supports SZIP with no encoder.
Description:
SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled.
Solution:
Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC.
Platforms tested:
Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++)
Misc. update:
Purpose:
Updated H5MPprivate.h for the new API function H5Itype_exists.
Description:
This is the last commit for these changes... I swear...
Solution:
Platforms tested:
Misc. update:
Description: Added new API H5Fget_name and new test program called filename.c. This function
returns the name of the file by object ID(file, group, dataset, named datatype, and attribute)
which belongs to the file.
Platforms tested: h5committest and fuss.
Misc. update: MANIFEST and RELEASE.txt
Description: If compound datatype has variable-length string as field, after it's committed to
the file, when it's reopened, the size of the datatype is miscalculated. It makes the data
unreadable. Dumper also fails to read data because of this problem.
Solution: Change the size of datatype when it's committed to file. Then modify the size again
for memory type when it's reopened.
Platforms tested: h5committest and RH 8.
Misc. update: RELEASE.txt
Purpose: Maintenance
Description: h5dwrite/read_f and h5awrite_read_f functions were overloaded
with dims argument being of type INTEGER and INTEGER(HSIZE_T).
We promised to remove overloading in 1.4 release. It was
removed from documentation but not from the source code.
Solution: Finish code cleanup.
Platforms tested: Solaris 2.7 32 and 64 bit, and copper with --enable-
parallel. Windows are broken, so Kent and Xuan
will test it with their changes later.
Misc. update:
Description: a new dataspace, null dataspace.
Solution: it's a dataspace without any element. This is a staged check-in.
After daily test succeeds, will add a test.
Platforms tested: h5committest
Misc. update: RELEASE.txt
Description: integer-float software conversion. There are some
print commands left in the source code. Will clean them up once
no problem is found in test.
Platforms tested: h5committest.
Misc. update: RELEASE.txt
bug fix
Description:
the fletcher filter used a temporary 2 byte word buffer to compute the checksum.
this is non portable between big-endian/little endian.
Solution:
replaced with a buffer of 1 byte type
Platforms tested:
linux
solaris
solaris 64 bit
AIX
windows
Misc. update:
Code cleanup/bug fix
Description:
Fix configuration files to allow 64-bit compilation of the library with
a 64-bit version of zlib.
Solution:
Moved important compiler flags settings from CC macro to CFLAGS macro.
Instead of setting CC (F9X, CXX) environment variable for compiling for 64-bit
code, users should just set the CFLAGS (FFLAGS, CXXFLAGS) to -q64.
Updated INSTALL file to reflect this change.
Platforms tested:
IBM p690 (copper) w/parallel, FORTRAN, C++, srcdir.
Platforms specific - h5committest does not apply.
Description: software conversion from float to integer(integer to float hasn't
be done yet).
Solution: This is the software conversion.
Platforms tested: h5committest
Misc. update: RELEASE.txt
Purpose:
replaced name of delete filter with remove filter for the new function H5Premove_filter
Description:
Solution:
Platforms tested:
linux
Misc. update:
New Feature. (Contributed by Robb)
Description:
Allow h5ls to specify a VFL with the '--vfl=' command line flag.
Add MPI-I/O & MPI-POSIX VFL drivers to list of drivers available for
h5ls and h5dump.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committested
Bug fix.
Description:
Allow H5Tget_native_type() to handle opaque fields in compound datatypes.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix/optimization
Description:
Address slowdown in MPI-I/O file metadata operations that was introduced
mid-stream. We now _require_ a POSIX compliant parallel file system for the
MPI-I/O file driver (as well as for the MPI-POSIX file driver).
Also optimized file open operation when the file is being created by
reducing the number of collective & syncronizing calls.
Additionally, refactor the MPI routines into a common place, eliminating
duplicated code.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committest
Code cleanup
Description:
Add C++ and FORTRAN wrappers for new H5Pdelete_filter routine, along with
documentation and a note in the release notes.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Linux 2.4 (verbena) w/ C++ and FORTRAN
Too minor for full h5committest
Add tests & docs
Description:
Added tests for the recent B-tree bug fixes & document their fix in the
release notes.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Correct two problems with variable-length datatypes in datasets:
- When overwriting an entire dataset, writing the fill value to the
file would be skipped, causing problems for VL datatypes when
objects in the file had been unlinked (and thus the space in the
file was not all zeros)
- When an application has set a fill-value for a dataset and the
dataset's datatype contained a VL datatype, the library was filling
space on disk with the memory form of the VL information, instead
of the disk form.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
When two property lists are compared, the H5Pequal routine was just
comparing the raw information for the property values. This causes problems
when the raw information contains pointers to other information.
Solution:
Allow a 'compare' callback to be registered for properties, so that a user
application get perform the comparison itself, allowing for "deep" compares of
the property value.
This was exported to the H5Pregister & H5Pinsert routines in the development
branch, but not the release branch.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Purpose: Maintenance
Description: Fortran APIs MAC OS X port for IBM XL Fortran compiler
Solution: Brought back changes from 1.6 branch
Platforms tested: pommier, h5comittested; this time h5committest
complained about copperpp directory and didn't run;
tests on verbena and sol passed.
Misc. update:
Bug fix
Description:
Fix small memory leak of fill-value information inside the library.
Solution:
"reset" the fill-value object header message, so that the internal buffer
gets freed.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix
Description:
Range check the szip 'pixels per block' against the chunk size of the
dataset when attempting to create a new dataset, since the szip library
requires the PPB to be at least the size of the fastest changing dimension
in the chunk.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor for h5committest
Bug fix
Description:
Fixed bug in hyperslab iteration where certain combinations of flattened
and non-flattened dimensions would cause incorrect locations to be iterated
over.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code cleanup, bug fixes
Description:
Wrap up rest of changes necessary for fixing the "short" MPI-I/O read
problem that Robb reported.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix
Description:
Attributes which were created with scalar dataspaces were reporting their
dataspace as a simple dataspace when queried later.
Solution:
Fix the dataspace handling code when reading in the attribute message from
the file to set the extent type correctly.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix.
Description:
Using a selection offset with hyperslab selections in chunked datasets
was getting into an infinite loop and hanging the application.
Solution:
Apply the selection offset to the hyperslab selection properly.
Platforms tested:
FreeBSD 4.9 (sleipnir) w & w/o parallel
h5committest
Bug fix
Description:
Make a copy of the index value for H5Giterate and use that instead of
dereferencing the index pointer.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to need h5committest
Bug fix
Description:
Variable length strings and sequences with NULL pointers were not handled
by library, causing problems access the data. This also affected fill values
for variable-length datatypes.
Solution:
Address the issues in the library by detecting NULL sequences/strings
and avoid trying to convert them.
Patched up dumper to display NULL sequences/strings.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code optimization
Description:
Improved integer & floating-point datatype conversions by removing some
corner cases. Got rid of algorithm which walked the buffer of elements to
convert backwards, switching to algorithm which computes the non-overlapping
space in the buffer and repeatedly converts it in the forward direction.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Cray SV1 & T3E (T90 is not accessible)
Improvement
Description:
If the library is built in parallel mode, then name the "h5cc" and
"h5fc" programs "h5pcc" and "h5pfc" resp. This saves the user from
worrying about mixing serial and parallel builds of the library and
having to specify the pathnames and so-forth.
Platforms tested:
Linux (w/ parallel & Fortran)
Modi4 (w/ parallel & Fortran)
Sol (Serial & Fortran & C++)
Misc. update:
Updated release/docs/RELEASE
Bug fix & code cleanups
Description:
Change our use of MPI derived datatypes to not create datatypes with
"0-sized" lengths, which causes the LANL Q machine to hang.
Also, get rid of "prefer MPI derived datatypes" environment variable since
it has no advantage.
Platforms tested:
FreeBSD 4.9 (sleipnir) w & w/o parallel
h5committest
Description: new function H5Iget_file_id. It returns file ID given an obeject
(group, dataset, or attribute) ID.
Platforms tested: h5committest
Misc. update: RELEASE.txt
Bug fix
Description:
Single hyperslab selections (which were set with only one call to
H5Sselect_hyperslab) that had dimensions that could be "flattened" but were
interspersed with dimensions that could not be flattened were not correctly
handled, causing core dumps.
Solution:
Re-work "flattening" code to handle this case properly.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committest
bug fix, update documentation
Description:
version 8 of Code Warrior has a bug on the open file function
that causes one of our tests to fail
documented this in the install instructions
removed a CW specific header file include , that was left from version 6
from the file testhdf5.c
Platforms tested:
Code Warrior
linux
Misc. update:
Purpose: Bug fix
Description: Parameter rdcc_nelmts of the h5pget_cache_f subroutine
had wrong INTEGER(SIZE_T) instead of INTEGER type.
Solution: Fixed the type
Platforms tested: arabica in 64-bit mode (where INTEGER(SIZE_T) is not
the same as INTEGER)
Misc. update:
Bug fix
Description:
The 'char *' type is one of the "strongly" aligned types on Crays, but
a 'void *' is "weakly" aligned. So, assigning a 'void *' (pointing to a
location to place a 'char *') to a 'char **' can change the pointer value
during the assignment.
Solution:
Don't alias the 'void *' where the variable-length information ('char *'
or 'hvl_t') will go. Use a temporary variable on the stack to build up the
information about the VL string or sequence and then memcpy() the temporary
variable directly to the location pointed to with the 'void *'
Platforms tested:
FreeBSD 4.9 (sleipnir)
Cray SV1 (wind)
specific to Cray problems, h5committest not necessary.
Bug fix
Description:
When too many messages were inserted into an object header, the library
had an internal pointer to the "new message" that was pointing to the incorrect
location when the array of messages was re-allocated.
In the worst case, this could cause a file to be corrupted.
Solution:
Update the internal pointer when the array is re-allocated.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Too small to require h5committest
Feature add
Description:
Add a few new fields to the H5G_stat_t structure, to allow more information
about the object header to be retrieved.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Add feature
Description:
Add H5Fget_freespace() routine, to check the amount of free space in a
file. This information is only valid until the file is closed currently,
however (until we start recording the free space information in the file
itself).
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Improved algorithm (bug fix, sorta)
Description:
The internal algorithm for adding new IDs in the ID manager code (H5I) was
adding new IDs to the front of the linked list and never adjusting the order
of the items on the list (unless an ID was deleted). If many new objects were
created, they would push earlier ones _way_ down the list (especially if the
objects were being leaked in the application, as they appear to be in the
current HDF-EOS5 library) and would cause O(n) search time for items on the
list.
The ID caching code in the ID manager was avoiding this behavior sometimes,
but it was adding IDs that were looked up to the very tail of the cache and
they would frequently leave the cache before helping.
Solution:
Implemented a "move to front" scheme for the linked list of IDs, which
improves the lookup situation for frequently accessed objects.
Removed ID caching code now, as the "move to front" algorithm actually
works better.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug/feature fix.
Description:
Relax restriction on parallel writing to compact datasets to allow partial
I/O.
Updates to reference manual mentioning the issues involved are delayed until
reference manual 'lock' is removed later this week.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fixes and code cleanup
Description:
Lots of changes here:
- Fixed bug #691 - when shared datatypes are used in attributes they
are incorrectly copied into the attribute instead of referring
the the named datatype in the file. This required bumping the
version of the attribute message. The new version of the attribute
message is only written out when a shared datatype is used in
the attribute. [Also, this format change made the size of the
attribute smaller.]
- Added information to attribute debugging routine so that shared
datatypes are displayed correctly with the h5debug tool.
- Refactored the H5O* routines to extract code that was common to
several routines into subroutines to call.
- Added 'link' method for H5O message sub-classes, which increments
the link count on shared objects when a message is created which
shares them.
- Corrected [unreported] bug where the link count was not being
decremented on the shared object when a object header message
with a reference to that object was deleted from the file.
- Reduced size of shared message from 49 bytes (which was incorrect
anyway and should have been 48 bytes) to 10 bytes, which required
bumping the version of "shared" messages.
- Refactored some of the shared datatype routines to allow for easier
queries of "committedness" internally to the library and also
added routine to easily increment/decrement the reference count of
a shared datatype.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
The MPI_File_set_size() routine on ASCI Red is not able to extend files
so that they are larger than 2GB.
Solution:
Add an extra macro which controls whether MPI_File_set_size() can handle
>2GB offsets or if our "older" way of reading a byte, then writing a byte at
the appropriate offset should be used.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code cleanup, etc.
Description:
Generalize Ray's datatype fixes to handle packing compound datatypes which
are the base type of an array or variable-length type, etc.
Also track "packedness" of a compound datatype from it's creation, instead
of only setting the 'packed' flag after the datatype was explicitly packed.
Updated docs to reflect that a compound datatype is allowed to grow (but
not shrink).
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix.
Description:
Correct bug where a file opened twice, once with read-write permission
and once with read-only permission would cause closing the file with the
read-only file ID to fail because it was trying to flush information out
of the file.
Solution:
Check the permissions on file IDs that are being closed and only flush
when the particular file ID was opened with write permission.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too small to need h5committest
Misc. update:
Bug fix
Description:
The VFL driver ID in a file's access proprty list wasn't being reference
counted correctly, causing the VFL driver to get prematurely closed after
several calls to "H5Pget_access_plist->H5Pclose".
Solution:
Increment VFL driver ID reference count when copy of file's access property
list is made in H5Pget_access_plist()
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug Fix and code cleanup
Description:
Correct error in H5T_detect_class that was causing nested compound datatypes
with to not detect the datatype class of fields correctly, which caused errors
with fill-values, variable-length datatypes and chunks later on.
Return the rank of the array datatype from H5Tget_array_dims(), like
H5Sget_dims().
Lots of cleanups to datatype code, to make the handling of arrays, compound
types, variable-length strings and sequences and enumerated types more
consistent and robust.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
When datasets are deleted from a file, they are removed from the sieve
buffer, but instead of invalidating only the part of the sieve buffer affected,
the sieve buffer code would throw away the entire sieve buffer, potentially
including other raw data in the buffer that hadn't been written to disk yet.
Solution:
Improve the sieve buffer clearing code to handle partial invalidations.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
H5Gget_objname_by_idx should allow NULL for the 'name' parameter, to allow
for querying for the name's length.
Solution:
Allow NULL for the 'name' parameter, the internal functions were already
ready to handle that case.
Clean up RM information about H5Gget_objname_by_idx and
H5Gget_objtype_by_idx also.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
H5Gget_num_objs, H5Gget_objname_by_idx and H5Gget_objtype_by_idx were
only accepting a group ID, instead of a location ID, as our documentation for
them stated.
Solution:
Allow them to accept a location ID.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
I/O on chunked datasets with a scalar dataspace for the memory dataspace
was not working correctly.
Solution:
Translate the scalar dataspace into a n-dimensional (where n is the number
of dimensions of the file's dataspace) dataspace of dimensions 1x1x1...
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest (modulo modi4 which is not working correctly)
Bug fix
Description:
Linking and unlinking objects with multiple internal and/or trailing '/'s
was not handled well in the library.
Solution:
"Normalize" strings by removing trailing '/'s and collapsing multiple
internal '/'s down into just one '/' before operating on the name.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committested
Bug fix
Description:
Allow a user block to be "inserted" in front of a file (probably by
writing a validly-sized userblock to a new file and then appending another
HDF5 file to the new file).
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committested
Update readme
Description:
Noted that hobj_ref_t's and the objno field in the H5G_stat_t structure
can now be directly compared for equality by applications.
Add more contents related to szip
Fix other bugs
Description:
Describe where to find szip, how to turn off szip compression.
Solution:
Platforms tested:
Misc. update:
Bug fix
Description:
I/O on chunked datasets with point selections was not working correctly.
Solution:
Re-wrote some parts of raw data I/O routines that build the selections for
each chunk to correctly handle point selections.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
When a non-default indexed storage B-tree internal 'K' value is set by the
user, the chunked datasets created in that file (until it is closed) use the
user's 'K' value and the data can be accessed correctly, but the 'K' value is
not stored in the file.
However, once the file is closed and re-opened, the non-default 'K' value
is lost and the data in the chunked datasets will not be able to be accessed
correctly.
Solution:
Store the indexed storage B-tree internal 'K' value in the superblock.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Update install_windows.txt.
Description:
the description of using the allexample.dsw needs to be modified with
the addition of szip to HDF5 library.
Some typos are corrected.
Solution:
Platforms tested:
Misc. update:
Bug fix
Description:
H5T_BKG_TEMP was accidentally removed from library code, but is used by
application's datatype conversion routines.
Solution:
Revert removal of H5T_BKG_TEMP.
Platforms tested:
h5committest
Purpose: Maintenance after 1.6.0 release
Description: I cleaned the file. Please note that information about known
problems, config features, and tested platforms is there.
Please do not forget to modify those sections too!
Solution:
Platforms tested:
Misc. update:
Update configuration
Description:
Now that we've split the tree (Yea!), change the main trunk back to debug
on and production off.
Also, change the version # to 1.7.0
Description: When make install prefix=NEW_DIR is used to specify
new location for the installed HDF5 library, h5redeploy
should be used to fix h5cc, etc. scripts. INSTALL file
suggested to fix scripts by hand.
Solution: Modified instructions to use h5redeploy.
Platforms tested: verbena
Misc. update:
Description: After talking to Barbara we decided to use
ftp://ftp.ncsa.uiuc.edu/HDF/HDF5/current/ location for
Intel and Cray Fortran source code patches.
Solution: Edited the file to point to new location.
Platforms tested:
Misc. update:
update for release
Description:
Set the version information to 1.6.0-pre1 to get ready for
v1.6 release.
Platforms tested:
h5committested.
Misc. update:
Purpose: Maintenance for the 1.6.0 release.
Description: I updated compilers information for Crays and added
h5fc and h5c++ to the new tools sections in REELASE.txt.
I added instructions for how to build HDF5 using Intel compilers
on Windows.
Solution:
Platforms tested:
Misc. update:
Purpose: Maintenance
Description: Bumped the version number to 1.5.62 after creating
a tar ball for the fourth round of testing.
Solution: run bin/h5vers -s scipt on arabica
Platforms tested: N/A
Misc. update:
Purpose: Maintenance for the third round of testing
Description: Increased the version number to 1.5.59 after creating
a tar ball for testing.
Solution:
Platforms tested:
Misc. update:
Bug fix
Description:
The dataset's modification time was getting set whenever raw data was
written with H5Dwrite. Unfortunately, this is a metadata change (which are
required to be performed collectively) and H5Dwrite may be called independently
from a parallel program, resulting in metadata cache corruption and/or program
hangs.
Solution:
Don't update the modification time when raw data it written. :-(
Platforms tested:
FreeBSD 4.8 (sleipnir) w/parallel
h5committest
Misc. update:
Noted in release notes and also sent to Frank for updating the docs.
Code cleanup/new features.
Description:
Clean up whitespace & formatting.
Make the -S (simple) output display a dataset's modification time in UTC
rather than local timezone.
Display variable-length datatype information.
Simplified & enhanced bitfield datatype information displayed.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
API tweak.
Description:
The H5Sget_select_bounds() API call was using hsize_t arrays for retrieving
the 'start' and 'end' coordinates, which is counter to the rest of the dataspace
API.
Solution:
Change the arrays to be hssize_t instead.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
FreeBSD 4.8 (sleipnir) w/parallel
h5committested
Misc. update:
Updated all docs for this change.
Added 1.4 compatibility #ifdef's
Purpose: Maintenance for 1.6 release
Description: I did more cleanup and restructuring of the file.
Please take a look at the file when you have a chance.
Thank you!
Solution:
Platforms tested:
Misc. update:
Purpose: Maintenance for 1.6 release
Description: HISTORY.txt was not updated in 1.5 branch since 1.2.0 release.
RELEASE.txt had a lot of redundant info that is now in HISTORY.txt
INSTALL_Windows_withF90.txt was lost in 1.5 branch.
Solution: I updated HISTORY.txt to include release history up to 1.4.5 release.
After that I removed all information related to the 1.4.* releases
from RELEASE.txt file. "Platforms Tested" section was brought up
to date based on the information provided by developers for 1.6.0 first
round of testing. RELEASE.txt still needs a lot of work.
INSTALL_Windows_withF90.txt is also a first draft, more work is needed.
Platforms tested: N/A
Misc. update:
Purpose: Maintenance for the second round of testing
Description: Used bin/h5vers to change version number to 1.5.55
Solution:
Platforms tested: arabica
Misc. update:
Add decriptions to use libtestD.dll.
Description:
libtestD.dll should be placed to a place that the user can find in order to
test HDF5 DLL within HDF5 workspace.
Solution:
Explained in document.
Platforms tested:
No need
Misc. update:
code warrior update
Description:
changed the project file to include the new H5Dio.c and remove the old H5Farray.c files
updated install instructions
Solution:
Platforms tested:
code warrior
Misc. update:
Purpose: Maintenance
Description: I changed release version to 1.5.54.
Email about tetsing of 1.5.53 will be send after this checkins.
Solution: used bin/h5vers -s to change the version number
Platforms tested: Done on arabica
Misc. update:
New feature/enhancement
Description:
Chunked datasets are handled poorly in several circumstances involving
certain selections and chunks that are too large for the chunk cache and/or
chunks with filters, causing the chunk to be read from disk multiple times.
Solution:
Rearrange raw data I/O infrastructure to handle chunked datasets in a much
more friendly way by creating a selection in memory and on disk for each chunk
in a chunked dataset and performing all of the I/O on that chunk at one time.
There are still some scalability (the current code attempts to
create a selection for all the chunks in the dataset, instead of just the
chunks that are accessed, requiring portions of the istore.c and fillval.c
tests to be commented out) and performance issues, but checking this in will
allow the changes to be tested by a much wider audience while I address the
remaining issues.
Platforms tested:
h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
Feature Add
Description:
Added knob so that the programmer can enable or disable GPFS
hints during runtime instead of having it only enabled at
configure/compile time. Some of the public APIs were changed
to add an extra parameter for this option...
Platforms tested:
Blue (LLNL). It only affects the MPI/POSIX driver, so no need
to test it on non-GPFS platforms.
Misc. update:
Add a little more comments on where to put DLL.
Description:
User can put HDF5DLL in the place where the application can find.
It doesn't have to be at system directory.
Solution:
Add comments to "only suggest the user to use system directory".
Platforms tested:
text file, no need to be tested.
Misc. update:
Update stdio driver supported on windows
Description:
To tell users how to set and use stdio driver.
Solution:
Platforms tested:
Text file doesn't have to be tested.
Misc. update:
Update
Description:
The H5FC was already in this doc...but under "bug fixes". Removed
that since it's really a new feature.
Platforms tested:
IBM/PC Jr.
Misc. update:
Bug fix
Description:
This fixes a bug in the low-level metadata caching code in the library
which could possibly lose metadata during file I/O when a lot of objects are
inserted into a group.
This also fixes a couple of (similar) fencepost bugs in the B-tree
deletion code.
Solution:
For the metadata bug - call the low-level driver's 'write' routine instead
of H5FD_write.
For the B-tree bug - include the correct number of keys.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
(h5committest is still not working for me on burrwhite)
Misc. update:
Purpose:
Maintenance
Description:
* Added support for generic properties.
* Added support for time allocation properties.
* Added support for variable length datatypes (only datatypes based
on INTEGER, REAL and CHARACTER Fortran types are supported).
* added some missing functions
Solution:
I am checking in new Fortran APIs and their man pages to support
1.5 features listed above.
Not all APIs have tests yet. APIs were written in Fall 2002, and I am afraid
that I will loose the code or totally forget what I did if I wait longer. ;-)
Platforms tested:
arabica (fortran), eirene (fortran), modi4 (parallel, fortran)
Purpose:
1.update windows release doc related to the addition of semi-automatic generated H5Tinit.c project and workspace files.
2. Make the doc more clear.
Description:
1. H5Tinit.c was manually maintained before and it might affect the users who wants to build and test from HDF5 snapshot release.
2. Make sure that we are supporting .Net only on XP and also emphasize to DLL users to the DLL critical section of the doc.
Solution:
1. Add a section for those users to semi-auto generate H5Tinit.c.
Platforms tested:
Doc files, no need to test.
Misc. update:
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
New feature for developers.
Description:
Added "function stack" tracing to library. This allows developers (there
is no public API) to call H5FS_print within the library and get a listing
of the functions traversed to reach that point in the library. Eventually,
I may add support for reporting the parameters to each function also...
Mainly for debugging parallel I/O programs, but I think it will come in
handy in other cases also.
The function stack tracking is controlled with a configure switch:
--enable-funcstack, which defaults to enabled currently. When we branch
for 1.6, we should change the default setting on the branch to be disabled.
Also, added a destructor to the thread-specific keys when thread-safety is
turned on in the library. Otherwise, they were leaking memory and causing
difficult to debug errors in threaded programs (like the test/ttsafe test).
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) w/thread-safety enabled.
Misc. update:
Updated MANIFEST with new files added (src/H5FS.c & src/H5FDprivate.h)
Update release_docs/RELEASE with thread-safety bug fix.
Bug Fix
Description:
When calling H5Fopen with the core VFL driver, but without the
H5F_ACC_CREAT flag goes ahead and creates a memory file.
Solution:
Check for the H5F_ACC_CREAT flag before allowing the memory file to be
created.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Update
Description:
Added description of the configure bugfix for people specifying
--with-*=/usr/include,/usr/lib in their scripts during configure...
Bug fix
Description:
Currently, when the library encounters an object header message that isn't
know, it fails to open that object in the file.
Solution:
Allow the library to skip over the unknown object header message and
continue to process the remaining messages, in the hope that the skipped
message isn't important later. If it is important, it will be caught at
a higher level of the library.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
Purpose:
add "shuffle" filter note
Description:
Solution:
Platforms tested:
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Purpose:
New function.
Description:
H5Dget_offset returns the offset of a dataset's data relative to the
beginning of a file.
Platforms tested:
arabica(simple function, one test should be enough.)
Misc. update:
RELEASE.txt
New feature
Description:
Add support for scalar dataspaces in parallel I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Also, FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Update release_docs/RELEASE for bug fixes, new features, etc.
Purpose:
Change release note for new functions(H5Gget_comment, H5Arename,
H5Aget_storage_size).
Misc. update:
Update release_docs/RELEASE for bug fixes, new features, etc.
Purpose:
Added missing fortran functions h5set(get)_buffer_f.
Also added docs and tests for them.
Solution:
Currently functions do not accept conversion and background buffers.
This corresponds to H5set(get)_buffer call with buffer pointers
set to NULL. If there is a demand, I can overload the functions
to have new parameters and go through all trouble creating functions
for all supported datatypes.
Platforms tested:
Solaris 2.7, Linux 2.2., IRIX64-6.5
Purpose:
Added new F90 APIs
Description:
I added new F90 APIs, tests, and documentation for the following
functions:
h5fget_obj_count_f h5pequal_f h5tget_member_index_f
h5fget_obj_ids_f h5pget_fclose_degree_f
h5pset_fclose_degree_f
Documentation for exisiting functions was missing:
h5freopen_f, h5fflush_f, h5fmount_f, h5unmount_f, h5fget_create_plist_f,
h5fget_access_plist_f.
Platforms tested:
Solaris 2.7, Linux 2.2 and IRIX64-6.5
Purpose:
New API functions
Description:
Added API functions to return pointer to low-level file handle
(H5Fget_vfd_handle and H5FDget_vfd_handle) and related property list
setting functions(H5Pset_family_offset and H5Pset_multi_type).
Platforms tested:
Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
Purpose:
Added new fortran functions
Description:
Source code, tests and documentation for h5glink2_f and h5gmove2_f
functions were added. I also added missing man page for h5glink_f
Platforms tested:
Solaris 2.7, IRIX64-6.5 and Liniux 2.2
Purpose:
Added missing fortran functions.
Description:
Four Library Fortran API functions have been added:
h5get_libversion_f, h5_check_version_f, h5garbage_collect_f and
h5dont_atexit_f. Only first two functions were tested.
Documentation file and RELEASE.txt were updated.
Platforms tested:
Solaris 2.7, IRIX64-6.5 and Linux 2.2
API name change
Description:
Change all "space time" references to "alloc time", including API functions
and macro definitions, etc.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
Code cleanup/New Feature
Description:
Improve the space allocation in the file by re-using freed space more
effectively.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel
Purpose:
Merged from 1.4 branch
Description:
API tracing improvements
Platforms tested:
Linux (--disable-hsizet didn't work before my changes and still doesn't
work, but --enable-hsizet is fine).
Note: This checkin includes temporary code in H5FDmpiposix.c to turn
off GPFS byte range token prefetches on ASCI/Blue. Once the HDF5 API
supports the necessary prerequisite functionality this temporary code
can be migraged up above HDF5.
Purpose:
update windows support.
Description:
1. Since we don't support w98 anymore, delete the description about w98.
Actually DLL may work on w98.
2. Release dll work for new HDF5 release.
Solution:
Platforms tested:
Bug Fix
Description:
It was possible to create corrupted metadata information (either in memory
or in the file or both) with a parallel I/O program because of the way
metadata writes were being handled for writes out of the metadata cache.
Solution:
Added a dataset transfer property called "block before metadata write"
which is used by the MPI-I/O and MPI-posix drivers to sync up all the
processes before attempting a metadata write. This property is currently
only for metadata writes from the metadata cache.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
Bug Fix
Description:
When parallel I/O is used, the MPI-I/O VFL driver uses a "lazy" model to
call MPI_File_set_view() in order to reduce the number of calls to this
function. However, this is unsafe, because if a collective I/O which uses
MPI derived types (and thus uses MPI_File_set_view()) is immediately
followed by an independent I/O, the code will attempt to call
MPI_File_set_view() in order to switch back to the default view of the
file. MPI_File_set_view() is a collective call however, and this causes
the application to hang.
Solution:
Removed "lazy" MPI_File_set_view() code, instead set the file view when it
is needed (with MPI derived types) and immediately set the file view back to
the default view before leaving the I/O routine.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel. Also, tested with the latest development
and release code for the SAF library, which now works correctly with this
change. (Although the release branch of the SAF library seems to have a
bug, this 1.4.4 release candidate code gets as far as the version the SAF
library is released on top of (1.4.2-patch1, I believe)).
New feature.
Description:
There is some discussion among the SAF team as to whether it is better
to use MPI derived types for raw data transfers (thus needing a
MPI_File_set_view() call), or whether it is better to use a sequence of
low-level MPI types (i.e. MPI_BYTE) for the raw data transfer.
Solution:
Added an in internal flag to determine whether derived types are preferred
(the default), or whether they should be avoided. An environment variable
("HDF5_MPI_PREFER_DERIVED_TYPES") can be set by users to control whether MPI
types should be used or not. Set the environment variable to "0" (i.e.:
'setenv HDF5_MPI_PREFER_DERIVED_TYPES 0' to avoid using MPI derived types.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
Bug fix.
Description:
The chunking code was using internal allocation routines to put blocks on
a free list for reuse, instead of using the system allocation routines (ie.
malloc, free, etc.). This causes problems when user filters attempt to
allocate/free chunks for their algorithm's use.
Solution:
Switched the chunking code back to using the system allocation routines,
we can address performance issues with them if it becomes a real problem.
Platforms tested:
Linux 2.2.x (eirene) && IRIX64 6.5 (modi4)
Code optimization
Description:
Avoid creating MPI types (and thus requiring a MPI_File_set_view() call)
when contiguous selections are used for dataset I/O. This should be a
performance improvement for those sorts of selections.
Platforms tested:
Linux 2.2.x (eirene) w/parallel && IRIX64 6.5 (modi4) w/parallel & FORTRAN
Bug fix
Description:
I/O on "Regular" hyperslab selections could fail to transfer correctly
if the number of elements in the selection's row did now fit "evenly"
into the buffer being used for the transfer.
Solution:
Correct the calculation of the block & count offsets within the optimized
"regular" hyperslab routines.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Update
Description:
Explained that, if the user uses the "make install prefix=NEW_DIR"
option, they'll need to modify the installed h5cc file to reflect the
change.
Bug Fix
Description:
H5Dcreate and H5Tcommit allow "empty" compound and enumerated types (i.e.
ones with no members) to be stored in the file, but this causes an assertion
failure and is somewhat vapid.
Solution:
Check the datatype "makes sense" before using it for H5Dcreate and
H5Tcommit.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Bug Fix (#709)/Code improvement.
Description:
Allow chunks for chunked datasets to be cached when file is open for
read-only access.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
Bug fix (bug #777)
Description:
Current code allows a compound datatype to be inserted into itself.
Solution:
Check if the ID for the member is the same as the ID for the compound
datatype and reject it if so.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Bug Fix for bug #789
Description:
Creating a 1-D dataset region reference caused the library to hang (go into
an infinite loop).
Solution:
Corrected algorithm for serializing hyperslab regions.
Platforms tested:
FreeBSD 4.5 (sleipnir)
New feature.
Description:
Added a "small data" block allocation mechanism to the library, similar to
the mechanism used for allocating metadata currently.
See the RFC for more details:
http://hdf.ncsa.uiuc.edu/RFC/SmallData/SmallData.html
This reduces the number of I/O operations which hit the disk for my test
program from 19 to 15 (i.e. from 393 to 15, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
Purpose:
Bug fix (#699), fix provided by a user, approved by Quincey
Description:
When a scalar dataspace was written to the file and then
subsequently queried with the H5Sget_simple_extent_type function,
type was reported H5S_SIMPLE instead of H5S_SCALAR.
Solution:
Applied a fix (see bug report 699)
Platforms tested:
Solaris 2.7 and Linux 2.2.18
Code improvement
Description:
The metadata aggregation code in the library was not terribly smart about
extending contiguous regions of metadata in the file and would not extend
them as far as possible. This causes space in the file to be wasted, also.
Solution:
Be smarter about extending the space used in the file for metadata by
checking whether new metadata blocks allocated in the file are at the end
of the current metadata aggregation region and append them to the metadata
region if so. This has the nice side benefit of reducing the number of
bytes we waste in the file and reducing the size of the file by a small
amount in some cases.
This reduces the number of I/O operations which hit the disk for my test
program from 53 to 19 (i.e. from 393 to 19, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
Bug Fix
Description:
The "dirty" flag for symbol table entries and symbol table nodes was not
being cleared when they were flushed to the file, causing lots of extra
metadata I/O.
Solution:
Reset the symbol table entry & nodes' flags when thy are flushed to disk.
This reduces the number of I/O operations which hit the disk for my test
program from 83 to 53 (i.e. from 393 to 53, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
Code cleanup/bug fix
Description:
The "metadata accumulator" cache in the library (which is designed to catch
small metadata writes/reads and bundle them together into larger I/O
buffers) was incorrectly detecting the important case of metadata pieces
being written sequentially to the file, adjoining but not overlapping.
Additionally, the metadata accumulator was not being used to cache data
read in from disk, only caching writes.
Solution:
Fix accumulator to correctly cache adjoining metadata writes and also to
cache metadata read from disk.
Between these two fixes, the number of I/O requests which resulted in actual
reads/writes to the filesystem dropped from 393 requests to 82 for the
particular test I was using. :-)
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
Document Bug Fix
Description:
Under certain [obscure] circumstances, an object header would get paged out
of the metadata cache, and when it was accessed again and brought back into
the cache, and immediately had additional metadata added to it (an
attribute, usually, or perhaps adding an object to a group), and needed to
be extended with a continuation message, but there was no room in any
existing object header chunks for the continuation message and an existing
object header message needed to be moved to the new object header chunk (I
told you it was obscure :-), the object header message moved to the new
chunk (not the new metadata being added) would get corrupted. *whew* :-)
Solution:
Actually copy the "raw" object header message information of the object
header message being moved to the new chunk, instead of relying on the
"native" object header message information being re-encoded when the object
header is flushed. This is because when an object header is paged out of
the metadata cache and subsequently brought back in, the "native"
information pointer in memory is reset to NULL and only the "raw"
information exists.
Platforms tested:
Solaris 2.7 (arabica) & FreeBSD 4.5 (sleipnir)
Purpose:
update, remove hdf4-related stuff.
Description:
hdf4 related tools have been moved out of HDF5 CVS tree, The install doc should reflect this.
Solution:
Platforms tested:
Document Code improvement below:
Description:
Propagated the "fill time" property into the parallel chunk allocation
routine, allowing it to avoid writing fill values to each new chunk
allocated. This improves the performance of chunked datasets in parallel
I/O to be on par with contiguous datasets again (on modi4).
Document Bug fix/Code improvement below:
Description:
Currently, the chunk data allocation routine invoked to allocate space for
the entire dataset is inefficient. It writes out each chunk in the dataset,
whether it is already allocated or not. Additionally, this happens not
only when it is created, but also anytime it is opened for writing, or the
dataset is extended. Worse, there's too much parallel I/O syncronization,
which slows things down even more.
Solution:
Only attempt to write out chunks that don't already exist. Additionally,
share the I/O writing between all the nodes, instead of writing everything
with process 0. Then, only block with MPI_Barrier if chunks were actually
created.
