Bug fix
Description:
Retrieving an object's name could fail (in various ways) under certain
circumstances (mostly having to do with mounted files).
Solution:
Re-write & simplify "get object name" code to fix error in a better way
than adding yet another hack to the previous pile of hacks... :-)
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Unstable format version marker.
Description:
This marks the Check in baseline for compact group revisions,
which radically revises the source code for managing groups and object headers.
Earlier versions (1.7.52 or earlier) have the original format. This version
and later have the unstable format until further notice.
Platforms tested:
h5committested.
New feature
Description:
Check in baseline for compact group revisions, which radically revises the
source code for managing groups and object headers.
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
This initiates the "unstable" phase of the 1.7.x branch, leading up
to the 1.8.0 release. Please test this code, but do _NOT_ keep files created
with it - the format will change again before the release and you will not
be able to read your old files!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
Solution:
There's too many changes to really describe them all, but some of them
include:
- Stop abusing the H5G_entry_t structure and split it into two separate
structures for non-symbol table node use within the library: H5O_loc_t
for object locations in a file and H5G_name_t to store the path to
an opened object. H5G_entry_t is now only used for storing symbol
table entries on disk.
- Retire H5G_namei() in favor of a more general mechanism for traversing
group paths and issuing callbacks on objects located. This gets us out
of the business of hacking H5G_namei() for new features, generally.
- Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t
- Lots more...
Platforms tested:
h5committested and maybe another dozen configurations.... :-)
New feature
Description:
Add in baseline "object copy" code from Peter [in the form of a new API
routine: H5Gcopy()]. There's still some work to do (like handling variable-
length datatypes and possibly support for references) and it hasn't been tested
on mounted files yet, but the core functionality is there and working
correctly.
I've also got a set of patches to update the 1.6 branch with tweaks to
keep the branches mostly in sync, but Elena will kill me if I import them
before the 1.6.5 release is out... :-)
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committested
Updated RELEASE.txt
Description:
Mentioned that h5c++ and h5fc now support linking against shared libraries.
Platforms tested:
documentation change only
Bug fix
Description:
Fix "double root open" bug and apply same fix to datasets & named datatypes.
Also fix bug in symbol table entry lookup code that would "transfer" an
object from one file handle to another file handle when the two file handles
were opened on the same actual file.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Updated release notes
Description:
Mentioned that shared C++ and Fortran libraries are supported in
INSTALL and RELEASE.txt.
Platforms tested:
documentation change only
Description: Added support for "big" REAL and DOUBLE PRECISION types.
Solution:
Platforms tested: heping with g95 (-r8, -d8)
mir with ifc (-r16, -r8)
Misc. update:
Purpose:
Add the known problem for IRIX6.5 C version >7.4.
Description:
Somehow SGI's MPI implementation doesn't free MPI_TYPE_entries,
So for large application to use MPI derived datatype, environment
variable MPI_TYPE_MAX needs to be reset.
Solution:
Platforms tested:
Misc. update:
Code cleanup & speedup
Description:
Refactor h5dump & related code to be considerably cleaner & faster for
files with large #'s of objects.
Platforms tested:
FreeBS 4.11 (sleipnir)
Linux 2.4
Bug fix
Description:
Fix core dump when checking whether to invalidate the name of an object
when unlinking an object in a group opened through an object reference.
Solution:
Check if names of various objects are valid before comparing them, etc.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Add more comments for collective chunk IO supports.
Description:
On some platforms that use mpich library internally, we have no way to check mpich version number. We have to manually turn off the collective IO supports.
One example is Altix(cobalt). Just put a comment there so that we won't forget about this.
Solution:
Platforms tested:
Misc. update:
Description:
Added missing member functions:
H5::CompType::getMemberArrayType
H5::CompType::getMemberVarLenType
H5::AbstractDs::getArrayType
H5::AbstractDs::getVarLenType
H5::CommonFG::openArrayType
H5::CommonFG::openVarLenType
H5::PropList::copyProp -- this will replace the current
H5::PropList::copyProp in later releases due
to incorrect prototype.
H5::IdComponent::getHDFObjType
Bug fix
Description:
Correct problems when querying information about a group that was opened
by dereferencing an object reference.
Solution:
Read in symbol table information instead of rely on it being cached.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
If a named datatype is copied and the copy is used to create a dataset,
the dataset would inadvertantly refer to the original named datatype instead
of a local (possibly modified) copy of the named datatype.
Solution:
Fixed datatype copying routine.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Further progress on fixing file mounting to work properly.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Too minor to require h5committest
Description: There was a bug in the calculating code of the Fletcher32
checksum in the library before v1.6.3. The checksum value wasn't consistent
between big-endian and little-endian systems. This bug was fixed in
Release 1.6.3. However, after fixing the bug, the checksum value is no
longer the same as before on little-endian system.
Solution: Made the library compare both the correct checksum and incorrect
checksum generated from v1.6.2 or before. This makes the library be
backward compatible but not forward compatible.
Platforms tested: h5committest and fuss.
Misc. update: Documented the forward incompatibility problem in the
Known Problem section in RELEASE.txt
Bug fix & code cleanup
Description:
Don't unmount child files from a parent file if the parent file is
being held open by an object.
Also, moved the mounting routines into their own source file and the
superblock routines into their own file also.
Solution:
Moved the code for unmounting child files down in the H5F_close routine
and also add checks to the file and group mounting code to unmount child
files when the last object holding open a file closes.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Bug fix
Description:
Fixed bugs that caused tests to be unable to find testhdf5.sh during
daily tests, and another that broke sol and shanti.
Solution:
When tests are run, their path is ./$testname .
When scripts are run, their path has no leading "./". Fixed.
Sol and shanti don't recognize the -e option for test, but they do know
-f. Fixed.
Also modified release notes.
Platforms tested:
mir, shanti, heping
Bug fix
Description:
When a series of files is mounted on one another and one of those files
is not unmounted, the library gets confused at shutdown and goes into an
infinite loop in the file interface.
Solution:
If there are still files left in the "closing" state when shutting down
the file API, iterate over those file IDs and unmount any child files that we
find mounted on those files.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Description: First, report a bug fix(#214) of family driver. Second
document PGI compiler bug for optimization on AMD processor.
Misc. update: First, report a bug fix(#214) of family driver. Second
document PGI compiler bug for optimization on AMD processor.
Description:
When compiler flag was used to set the size of Fortran integer
to 8 bytes, library would fail.
Solution:
Cleaned up the code; added detection of Fortran INTEGER type size
and appropriately defined int_f type for C-stubs routines.
Platforms tested:
Solaris 2.8 32 and 64-bit, AIX 5.1 64-bit parallel
PGI Fortran with -i8 flag on heping
Absoft Fortran with -i8 flag on heping
g95 on mir (Fortran integer is 8 bytes by default that cannot be
changed - compiler bug ;-)
AIX Fortran with -qintsize=8 32 and 64-bit modes on copper
Misc. update:
Bug fix
Description:
Hyperslab selections that had a selection offset and were applied to a
chunked dataset could get into an infinite loop or core dump if the same
selection was used multiple times, with different selection offsets.
Solution:
"Normalize" the selection with the selection offset, generate the
selections for the chunks overlapped and then "denormalize" the selection.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Add check for opaque tags that are too long for file format to handle
currently.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5commmitest
Bug fix
Description:
Fix several NULL pointer dereferences when tracking the name of open
objects during some unusual sequences of mount & unmount operations.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix/new feature
Description:
Setting "SEMI" or "STRONG" file close degrees causes problems when multiple
file IDs exist for the same file on disk.
Solution:
Make the "SEMI" and "STRONG" settings only apply to the file ID that is
being closed.
Also, add an "H5F_OBJ_LOCAL" flag for the H5Fget_obj_count() &
H5Fget_obj_ids() calls, so that applications can query about objects opened
with a particular file ID instead of all the objects opened in the file on
disk.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Description: See details from Bug #213. Family member file size wasn't saved
anywhere in file. When family file is opened, the first member size determine
the member size.
Solution: This is the third step of checkin. h5repart has been modified. If h5repart is used
to change the size of family member file, the new size(actual member size) is saved in the superblock.
In the second step of checkin, multi driver is checked against the driver
name saved in superblock. Wrong driver will result in a failure with an error message indicating
multi driver should be used. This change includes split driver because it's a special case for multi
driver.
In the first step of checkin. Family member size and name template(unused at this stage) are saved
in file superblock. When file is reopened,the size passed in thrin superblock. A different size
will trigger a failure with an error message indicating the right size. Wrong driver to open family
file will cause a failure, too.
Platforms tested: h5committest and fuss
Misc. update: RELEASE.txt
Feature
Description:
Added "support" for UTF-8 character encoding.
Solution:
Wrote tests to check that UTF-8 can be used in a number of places in
HDF5 (object names, data, etc.). These tests live in test/tunicode.c.
Added a new UTF-8 character encoding for datatypes.
Platforms tested:
mir, modi4, heping
Misc. update:
Description: h5pget_driver_f function returned information that could not
be interpreted by fortran application
Solution: Defined Fortran global variables that correspond to
the C H5FD_<driver_name> variables at the Fortran library
initialization time.
Platforms tested: heping with PGI compilers, sol 64-bit and copper parallel
Misc. update:
Description: Removed support for SRB driver
Solution: Removed or modified appropriate files; ran reconfigure
to regenerate Makefile.in and configure files.
Platforms tested: heping and shanti
Misc. update: ran bin/chkmanifest on heping
Description: Added missing h5get_member_class_f function
Solution:
Platforms tested: heping; it is a simple change. I will
watch daily tests and take care of failures.
Misc. update:
Purpose:
Added C++ wrapper for Packet Table API.
Description:
Added macro for high-level C++ library (LIBH5CPP_HL), which changes every
Makefile.in.
Added directories for high-level C++ library (though currently only Packet
Table API is supported).
Added both C++ source and tests.
Platforms tested:
sleipnir, mir, modi4
Misc. update:
Optimization
Description:
Speed up I/O on enumerated datatypes (including those nested in compound
datatypes, arrays, etc.) if the destination datatype is a proper superset of
the source datatype.
Solution:
Detect the situation and treat as no-op datatype conversion.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Automake update cleanup and minor changes
Description:
Removed macros in acsite.m4 that are no longer used.
Switched to using autoconf's AC_LANG_PUSH(Fortran) instead of old
AC_LANG_FORTRAN9X macro.
Switched to using AC_LANG_PUSH() and AC_LANG_POP() (from old AC_LANG_X).
Added ifort to list of Fortran compilers configure will look for.
Added a note about automake change to Release Notes.
Platforms tested:
copper, modi4 (parallel and serial), eirene
Description: Support of hardware conversion between "long double" and integers(mainly in
H5Tconv.c) and test cases for it(mainly in dtypes.c)
Platforms tested: h5committest and fuss
Misc. update: RELEASE.txt
Description: H5Tget_member_value didn't return correct value if called after
H5Tenum_valueof. It's because there's a sorting on the members of enum type in
H5Tenum_valueof which changed the order of members.
Solution: Made a copy of original type and do sorting on it to protect the
original order.
Platforms tested: fuss; tested v1.6 with h5committest
Misc. update: RELEASE.txt
Description: For variable-length string, H5Tget_class returned H5T_STRING as
its class. But H5Tdetect_class and H5Tget_member_class considered it as
H5T_VLEN. This is fixed to let all these 3 functions treat it as H5T_STRING.
Some test cases have been added to dtypes.c
Platforms tested: heping - already tested for v1.6 with h5committest
Misc. update: RELEASE.txt
Description: The byte order for all 1-byte integer types was fixed as little-endian
even on a big-endian machine. This's corrected in h5detect.c. When types are only
1 byte long, a native int is used substitute the type to detect byte order. Some tools
like h5dump and h5repack are also corrected in this case.
Platforms tested: fuss, copper, sol.(There're some failures from the recent configure
change).
Misc. update: Information in the RELEASE.txt.
Description: Removed PABLO from the source
Solution:
Platforms tested: arabica with 64-bit, copper with parallel,
heping with GNU C and C++ and PGI fortran (but
I disabled hl, there is some weird problem only
on heping: F9XMODFLAG is not
propagated to the Makefile files
Misc. update:
Description: Start to support software conversion between long double and
all integers.
Solution: No major changes to the algorithm. Changes to configure is to
exclude SGI for long double to integers test because there're too many problems
in their compiler.
Platforms tested: h5committest, modi4, fuss, Teragrid, arabica
Misc. update: RELEASE.txt
Bug fix
Description:
Correct values used in "space utilization" equation: the 'used' and
'total' values were backwards.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Fix core dump when flushing a file with a newly created attribute which
hasn't had a value written to it still open.
Solution:
Write the attribute fill value when appropriate.
Platforms tested:
FreeBSd 4.10 (sleipnir)
Linux 2.4 (verbena)
Solaris 2.7 (arabica)
Purpose: change feature
Description: Back up support bitfield and time datatypes in H5Tget_native_type.Leave it to future support. Let it return "not supported" error message for
now.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Bug fix
Description:
Relax restrictions on parallel I/O to allow compressed, chunked datasets
to be read in parallel (collective access will be degraded to independent
access, but will retrieve the information still).
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
IRIX64 6.5 (modi4)
h5committest
Description: enable size adjustment for compound datatype. The size
can be increased and decreased(as long as the last member isn't cut).
Solution: mainly check if the last member is being cut when decreasing
the size. Others are simply taking out assertion to enable this feature.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Bug fix.
Description:
Fix off-by-one error in Core VFL driver which would cause spurious address
or size overflow errors when an odd valued address or size was checked.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Linux 2.4 (heping)
Solaris 2.7 (arabica)
Update release_docs/RELEASE.txt to describe options for dealing
with installations which insert extra text in output, thereby
breaking tests which function by comparing output with an exemplar.
Description:
See above.
Solution:
Add discussion of this possible problem, and listed two possible
solutions:
1) "setenv HDF5_Make_Ignore yes" and then score "failed" tests
by hand.
2) Use the RUNSERIAL environment variable to use a filter script
to filter out the extra text.
Platforms tested:
No testing needed.
Misc. update:
Bug fix
Description:
Correct typedef for dataset region references to avoid struct alignment
issues on Crays.
Solution:
Change the typedef for hdset_reg_ref_t from a struct to an array of
unsigned char's of the correct size and propagate the appropriate adjustments
around the code.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Linux 2.4 (verbena) w/fortran
Cray T90 (subzero) w/fortran
Cray SV1 (wind) w/fortran & parallel
Cray T3E (cyclone) w/fortran & parallel
Description: Restore 6 old error API functions back to the library to be backward
compatible with v1.6. They are H5Epush, H5Eprint, H5Ewalk, H5Eclear, H5Eset_auto,
H5Eget_auto. These functions do not have error stack as parameter.
Solution: Internally, these functions use default error stack.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Bug fix (sorta)
Description:
Change reading of "missing" chunks from datasets with undefined fill
values to not overwrite the application buffer with random garbage from
memory. Note that this is almost the same, since whatever garbage the
application had in those locations will still be there...
Platforms tested:
FreeBSD 4.10 (sleipnir)
IRIX64 6.5 (modi4)
h5committested
Description: Added PGI Fortran support for Linux64 systems (x86_64)
Solution:
Platforms tested: AMD Opteron box at AMD Sunnyvale and verbena
with PGI and gcc compilers.
Misc. update:
Bug fix
Description:
Correct possible core dump when a datatype conversion function is
registered with the library after a compound datatype has been converted
(having it's type conversion information cached by the library). The compound
datatype must have been created by inserting the fields in non-increasing
offset order to see the bug.
Solution:
Re-sort the fields in the compound datatypes before recalculating the
cached information when performing the conversion on them.
Platforms tested:
FreeBSD 4.10 (sleipnir)
h5committested
Bug fix
Description:
Correct buffer overrun in "multi" VFL driver that was writing past the
end of the "driver name" buffer when encoding the driver info block for the
file's superblock.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Too minor to require h5committest
Bug fix.
Description:
Address two problems:
- The computation of the scanline in the szip filter was being
performed in the "can apply" callback routine instead of the
"set local" routine.
- The routine which allocated all the chunks for an entire dataset
(which is invoked when the allocation time is early or late,
rather than incremental) wasn't recording a failed filter in
the information for the chunk, causing the library to believe
that the chunk had the filter applied when it really hadn't.
Solution:
- Move the scanline computation to the "set local" callback.
- Record the filter mask with each chunk created when allocating them.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/szip
Too obscure to require h5committest
Description: Added support for Absoft Fortran compiler
Solution: Modified configuration file to check which Fortran compiler is used
and set appropriate flags.
Platforms tested: verbena with pgf90 and Absoft f95 compilers
Misc. update:
Bug fix
Description:
Allow I/O on extendible chunked datasets with (currently) zero-sized
dimensions to proceed harmlessly instead of dumping core on an assertion.
Solution:
Removed assertion and added checks to avoid problem situation in H5TB_end
Platforms tested:
FreeBSD 4.10 (sleipnir) w/ & w/o parallel
Too minor to require h5committest
Bug fix
Description:
Always write fill values to chunks when initializing entire B-tree and
any filters are defined.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
Too minor to require h5committest
Purpose: Maintenance for MAC OSX
Description: Added support for Absoft Fortran compiler f95;
Ddefault compiler is set to IBM xlf.
Solution:
Platforms tested: pommier with xlf and Absoft f95 compilers
Misc. update:
Code cleanup, sorta
Description:
Added ifdef sections for "H5_USING_PURIFY" in various places in the code,
which are designed to reduce the spurious "uninitialized memory read" warnings
from purify which are actually OK. Note that this macro will have to be
turned on by adding it to the CFLAGS for the build - I didn't think it was
important enough to add a configure flag for.
Also, the changes in H5HG.c optimize the walks through the objects in a
heap to only look at the 'used' entries instead of all the 'allocated' entries.
Platforms tested:
Solaris 2.7 (arabica) w/purify
Not tested by h5committest
Code cleanup
Description:
Fixed reserved.c test to use h5_fileaccess/h5_fixname/h5_cleanup.
Updated RELEASE.txt for previous bug fix
Platforms tested:
sleipnir, verbena
Description: This is the second step of checkin for encoding and decoding objects.
H5Tencode and H5Tdecode have been committed in the previous step. H5Sencode
and H5Sdecode are checked in this time.
Solution: Given object ID, these functions encode and decode object information
into and from binary buffer and return new object ID. They take advantage of the
existing codes of object header message and encode in the same format.
Platforms tested: fuss and h5committest.
Misc. update: RELEASE.txt
Bug fix
Description:
Allow buffer parameter to H5Dread & H5Dwrite to be NULL if there are no
elements to transfer.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Purpose:
Bug Fix
Description:
Trying to create the root group or the working group ("/" or ".") fakes out
HDF5 so that it neither creates a group nor returns an error.
Solution:
H5G_namei now throws an error if it was supposed to insert but didn't.
Platforms tested:
sleipnir, Visual Studio 7 (very minor change)
Misc. update:
Bug fix
Description:
Correct problems with "resurrecting" a dataset in a file. (This occurs
when a dataset which is open gets unlinked from the group hierarchy (making it
"dead" and marked for deletion in the file) and then is re-linked to the group
hierarchy). Note that the current solution applies only to datasets, further
work will fix this for groups and named datatypes also.
Also, fix the "debug" routines to be a little more helpful in certain
situations.
Additionally, fix a locking bug in the symbol table node splitting routine
which could be [one of] the cause[s] of the file corruption in flexible
parallel operation.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
h5committested
Bug fix
Description:
Fix error in chunked dataset I/O where data written out wasn't read
correctly from a chunked, extendible dataset after the dataset was extended.
Also, fix parallel I/O tests to gather error results from all processes,
in order to detect errors that only occur on one process.
Solution:
Bypass chunk cache for reads as well as writes, if parallel I/O driver is
used and file is opened for writing.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Purpose: Maintenance
Description: Added h5fget_name_f and h5fget_filesize_f subroutines and tests.
Solution: N/A
Platforms tested: arabica (32-bit), sol (64-bit)
parallle build on copper failed for the C library with the the
following error:
ld: 0711-317 ERROR: Undefined symbol: .H5FD_stdio_term
Since this change doesn't affect the C library, I am cheking it in
and will retest the fresh CVS copy after this check-in.
Misc. update:
Purpose:
HDF5 now supports SZIP with no encoder.
Description:
SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled.
Solution:
Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC.
Platforms tested:
Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++)
Misc. update:
Purpose:
Updated H5MPprivate.h for the new API function H5Itype_exists.
Description:
This is the last commit for these changes... I swear...
Solution:
Platforms tested:
Misc. update:
Description: Added new API H5Fget_name and new test program called filename.c. This function
returns the name of the file by object ID(file, group, dataset, named datatype, and attribute)
which belongs to the file.
Platforms tested: h5committest and fuss.
Misc. update: MANIFEST and RELEASE.txt
Description: If compound datatype has variable-length string as field, after it's committed to
the file, when it's reopened, the size of the datatype is miscalculated. It makes the data
unreadable. Dumper also fails to read data because of this problem.
Solution: Change the size of datatype when it's committed to file. Then modify the size again
for memory type when it's reopened.
Platforms tested: h5committest and RH 8.
Misc. update: RELEASE.txt
Purpose: Maintenance
Description: h5dwrite/read_f and h5awrite_read_f functions were overloaded
with dims argument being of type INTEGER and INTEGER(HSIZE_T).
We promised to remove overloading in 1.4 release. It was
removed from documentation but not from the source code.
Solution: Finish code cleanup.
Platforms tested: Solaris 2.7 32 and 64 bit, and copper with --enable-
parallel. Windows are broken, so Kent and Xuan
will test it with their changes later.
Misc. update:
Description: a new dataspace, null dataspace.
Solution: it's a dataspace without any element. This is a staged check-in.
After daily test succeeds, will add a test.
Platforms tested: h5committest
Misc. update: RELEASE.txt
Description: integer-float software conversion. There are some
print commands left in the source code. Will clean them up once
no problem is found in test.
Platforms tested: h5committest.
Misc. update: RELEASE.txt
bug fix
Description:
the fletcher filter used a temporary 2 byte word buffer to compute the checksum.
this is non portable between big-endian/little endian.
Solution:
replaced with a buffer of 1 byte type
Platforms tested:
linux
solaris
solaris 64 bit
AIX
windows
Misc. update:
Description: software conversion from float to integer(integer to float hasn't
be done yet).
Solution: This is the software conversion.
Platforms tested: h5committest
Misc. update: RELEASE.txt
Purpose:
replaced name of delete filter with remove filter for the new function H5Premove_filter
Description:
Solution:
Platforms tested:
linux
Misc. update:
New Feature. (Contributed by Robb)
Description:
Allow h5ls to specify a VFL with the '--vfl=' command line flag.
Add MPI-I/O & MPI-POSIX VFL drivers to list of drivers available for
h5ls and h5dump.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committested
Bug fix.
Description:
Allow H5Tget_native_type() to handle opaque fields in compound datatypes.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix/optimization
Description:
Address slowdown in MPI-I/O file metadata operations that was introduced
mid-stream. We now _require_ a POSIX compliant parallel file system for the
MPI-I/O file driver (as well as for the MPI-POSIX file driver).
Also optimized file open operation when the file is being created by
reducing the number of collective & syncronizing calls.
Additionally, refactor the MPI routines into a common place, eliminating
duplicated code.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committest
Code cleanup
Description:
Add C++ and FORTRAN wrappers for new H5Pdelete_filter routine, along with
documentation and a note in the release notes.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Linux 2.4 (verbena) w/ C++ and FORTRAN
Too minor for full h5committest
Add tests & docs
Description:
Added tests for the recent B-tree bug fixes & document their fix in the
release notes.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Correct two problems with variable-length datatypes in datasets:
- When overwriting an entire dataset, writing the fill value to the
file would be skipped, causing problems for VL datatypes when
objects in the file had been unlinked (and thus the space in the
file was not all zeros)
- When an application has set a fill-value for a dataset and the
dataset's datatype contained a VL datatype, the library was filling
space on disk with the memory form of the VL information, instead
of the disk form.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
When two property lists are compared, the H5Pequal routine was just
comparing the raw information for the property values. This causes problems
when the raw information contains pointers to other information.
Solution:
Allow a 'compare' callback to be registered for properties, so that a user
application get perform the comparison itself, allowing for "deep" compares of
the property value.
This was exported to the H5Pregister & H5Pinsert routines in the development
branch, but not the release branch.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Purpose: Maintenance
Description: Fortran APIs MAC OS X port for IBM XL Fortran compiler
Solution: Brought back changes from 1.6 branch
Platforms tested: pommier, h5comittested; this time h5committest
complained about copperpp directory and didn't run;
tests on verbena and sol passed.
Misc. update:
Bug fix
Description:
Fix small memory leak of fill-value information inside the library.
Solution:
"reset" the fill-value object header message, so that the internal buffer
gets freed.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix
Description:
Range check the szip 'pixels per block' against the chunk size of the
dataset when attempting to create a new dataset, since the szip library
requires the PPB to be at least the size of the fastest changing dimension
in the chunk.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor for h5committest
Bug fix
Description:
Fixed bug in hyperslab iteration where certain combinations of flattened
and non-flattened dimensions would cause incorrect locations to be iterated
over.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Code cleanup, bug fixes
Description:
Wrap up rest of changes necessary for fixing the "short" MPI-I/O read
problem that Robb reported.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix
Description:
Attributes which were created with scalar dataspaces were reporting their
dataspace as a simple dataspace when queried later.
Solution:
Fix the dataspace handling code when reading in the attribute message from
the file to set the extent type correctly.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fix.
Description:
Using a selection offset with hyperslab selections in chunked datasets
was getting into an infinite loop and hanging the application.
Solution:
Apply the selection offset to the hyperslab selection properly.
Platforms tested:
FreeBSD 4.9 (sleipnir) w & w/o parallel
h5committest
Bug fix
Description:
Make a copy of the index value for H5Giterate and use that instead of
dereferencing the index pointer.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to need h5committest
Bug fix
Description:
Variable length strings and sequences with NULL pointers were not handled
by library, causing problems access the data. This also affected fill values
for variable-length datatypes.
Solution:
Address the issues in the library by detecting NULL sequences/strings
and avoid trying to convert them.
Patched up dumper to display NULL sequences/strings.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code optimization
Description:
Improved integer & floating-point datatype conversions by removing some
corner cases. Got rid of algorithm which walked the buffer of elements to
convert backwards, switching to algorithm which computes the non-overlapping
space in the buffer and repeatedly converts it in the forward direction.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Cray SV1 & T3E (T90 is not accessible)
Improvement
Description:
If the library is built in parallel mode, then name the "h5cc" and
"h5fc" programs "h5pcc" and "h5pfc" resp. This saves the user from
worrying about mixing serial and parallel builds of the library and
having to specify the pathnames and so-forth.
Platforms tested:
Linux (w/ parallel & Fortran)
Modi4 (w/ parallel & Fortran)
Sol (Serial & Fortran & C++)
Misc. update:
Updated release/docs/RELEASE
Bug fix & code cleanups
Description:
Change our use of MPI derived datatypes to not create datatypes with
"0-sized" lengths, which causes the LANL Q machine to hang.
Also, get rid of "prefer MPI derived datatypes" environment variable since
it has no advantage.
Platforms tested:
FreeBSD 4.9 (sleipnir) w & w/o parallel
h5committest
Description: new function H5Iget_file_id. It returns file ID given an obeject
(group, dataset, or attribute) ID.
Platforms tested: h5committest
Misc. update: RELEASE.txt
Bug fix
Description:
Single hyperslab selections (which were set with only one call to
H5Sselect_hyperslab) that had dimensions that could be "flattened" but were
interspersed with dimensions that could not be flattened were not correctly
handled, causing core dumps.
Solution:
Re-work "flattening" code to handle this case properly.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committest
Purpose: Bug fix
Description: Parameter rdcc_nelmts of the h5pget_cache_f subroutine
had wrong INTEGER(SIZE_T) instead of INTEGER type.
Solution: Fixed the type
Platforms tested: arabica in 64-bit mode (where INTEGER(SIZE_T) is not
the same as INTEGER)
Misc. update:
Bug fix
Description:
The 'char *' type is one of the "strongly" aligned types on Crays, but
a 'void *' is "weakly" aligned. So, assigning a 'void *' (pointing to a
location to place a 'char *') to a 'char **' can change the pointer value
during the assignment.
Solution:
Don't alias the 'void *' where the variable-length information ('char *'
or 'hvl_t') will go. Use a temporary variable on the stack to build up the
information about the VL string or sequence and then memcpy() the temporary
variable directly to the location pointed to with the 'void *'
Platforms tested:
FreeBSD 4.9 (sleipnir)
Cray SV1 (wind)
specific to Cray problems, h5committest not necessary.
Bug fix
Description:
When too many messages were inserted into an object header, the library
had an internal pointer to the "new message" that was pointing to the incorrect
location when the array of messages was re-allocated.
In the worst case, this could cause a file to be corrupted.
Solution:
Update the internal pointer when the array is re-allocated.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Too small to require h5committest
Feature add
Description:
Add a few new fields to the H5G_stat_t structure, to allow more information
about the object header to be retrieved.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Add feature
Description:
Add H5Fget_freespace() routine, to check the amount of free space in a
file. This information is only valid until the file is closed currently,
however (until we start recording the free space information in the file
itself).
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Improved algorithm (bug fix, sorta)
Description:
The internal algorithm for adding new IDs in the ID manager code (H5I) was
adding new IDs to the front of the linked list and never adjusting the order
of the items on the list (unless an ID was deleted). If many new objects were
created, they would push earlier ones _way_ down the list (especially if the
objects were being leaked in the application, as they appear to be in the
current HDF-EOS5 library) and would cause O(n) search time for items on the
list.
The ID caching code in the ID manager was avoiding this behavior sometimes,
but it was adding IDs that were looked up to the very tail of the cache and
they would frequently leave the cache before helping.
Solution:
Implemented a "move to front" scheme for the linked list of IDs, which
improves the lookup situation for frequently accessed objects.
Removed ID caching code now, as the "move to front" algorithm actually
works better.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug/feature fix.
Description:
Relax restriction on parallel writing to compact datasets to allow partial
I/O.
Updates to reference manual mentioning the issues involved are delayed until
reference manual 'lock' is removed later this week.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Bug fixes and code cleanup
Description:
Lots of changes here:
- Fixed bug #691 - when shared datatypes are used in attributes they
are incorrectly copied into the attribute instead of referring
the the named datatype in the file. This required bumping the
version of the attribute message. The new version of the attribute
message is only written out when a shared datatype is used in
the attribute. [Also, this format change made the size of the
attribute smaller.]
- Added information to attribute debugging routine so that shared
datatypes are displayed correctly with the h5debug tool.
- Refactored the H5O* routines to extract code that was common to
several routines into subroutines to call.
- Added 'link' method for H5O message sub-classes, which increments
the link count on shared objects when a message is created which
shares them.
- Corrected [unreported] bug where the link count was not being
decremented on the shared object when a object header message
with a reference to that object was deleted from the file.
- Reduced size of shared message from 49 bytes (which was incorrect
anyway and should have been 48 bytes) to 10 bytes, which required
bumping the version of "shared" messages.
- Refactored some of the shared datatype routines to allow for easier
queries of "committedness" internally to the library and also
added routine to easily increment/decrement the reference count of
a shared datatype.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
The MPI_File_set_size() routine on ASCI Red is not able to extend files
so that they are larger than 2GB.
Solution:
Add an extra macro which controls whether MPI_File_set_size() can handle
>2GB offsets or if our "older" way of reading a byte, then writing a byte at
the appropriate offset should be used.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code cleanup, etc.
Description:
Generalize Ray's datatype fixes to handle packing compound datatypes which
are the base type of an array or variable-length type, etc.
Also track "packedness" of a compound datatype from it's creation, instead
of only setting the 'packed' flag after the datatype was explicitly packed.
Updated docs to reflect that a compound datatype is allowed to grow (but
not shrink).
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix.
Description:
Correct bug where a file opened twice, once with read-write permission
and once with read-only permission would cause closing the file with the
read-only file ID to fail because it was trying to flush information out
of the file.
Solution:
Check the permissions on file IDs that are being closed and only flush
when the particular file ID was opened with write permission.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too small to need h5committest
Misc. update:
Bug fix
Description:
The VFL driver ID in a file's access proprty list wasn't being reference
counted correctly, causing the VFL driver to get prematurely closed after
several calls to "H5Pget_access_plist->H5Pclose".
Solution:
Increment VFL driver ID reference count when copy of file's access property
list is made in H5Pget_access_plist()
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug Fix and code cleanup
Description:
Correct error in H5T_detect_class that was causing nested compound datatypes
with to not detect the datatype class of fields correctly, which caused errors
with fill-values, variable-length datatypes and chunks later on.
Return the rank of the array datatype from H5Tget_array_dims(), like
H5Sget_dims().
Lots of cleanups to datatype code, to make the handling of arrays, compound
types, variable-length strings and sequences and enumerated types more
consistent and robust.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
When datasets are deleted from a file, they are removed from the sieve
buffer, but instead of invalidating only the part of the sieve buffer affected,
the sieve buffer code would throw away the entire sieve buffer, potentially
including other raw data in the buffer that hadn't been written to disk yet.
Solution:
Improve the sieve buffer clearing code to handle partial invalidations.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
H5Gget_objname_by_idx should allow NULL for the 'name' parameter, to allow
for querying for the name's length.
Solution:
Allow NULL for the 'name' parameter, the internal functions were already
ready to handle that case.
Clean up RM information about H5Gget_objname_by_idx and
H5Gget_objtype_by_idx also.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
H5Gget_num_objs, H5Gget_objname_by_idx and H5Gget_objtype_by_idx were
only accepting a group ID, instead of a location ID, as our documentation for
them stated.
Solution:
Allow them to accept a location ID.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
I/O on chunked datasets with a scalar dataspace for the memory dataspace
was not working correctly.
Solution:
Translate the scalar dataspace into a n-dimensional (where n is the number
of dimensions of the file's dataspace) dataspace of dimensions 1x1x1...
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest (modulo modi4 which is not working correctly)
Bug fix
Description:
Linking and unlinking objects with multiple internal and/or trailing '/'s
was not handled well in the library.
Solution:
"Normalize" strings by removing trailing '/'s and collapsing multiple
internal '/'s down into just one '/' before operating on the name.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committested
Bug fix
Description:
Allow a user block to be "inserted" in front of a file (probably by
writing a validly-sized userblock to a new file and then appending another
HDF5 file to the new file).
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committested
Update readme
Description:
Noted that hobj_ref_t's and the objno field in the H5G_stat_t structure
can now be directly compared for equality by applications.
Bug fix
Description:
I/O on chunked datasets with point selections was not working correctly.
Solution:
Re-wrote some parts of raw data I/O routines that build the selections for
each chunk to correctly handle point selections.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
When a non-default indexed storage B-tree internal 'K' value is set by the
user, the chunked datasets created in that file (until it is closed) use the
user's 'K' value and the data can be accessed correctly, but the 'K' value is
not stored in the file.
However, once the file is closed and re-opened, the non-default 'K' value
is lost and the data in the chunked datasets will not be able to be accessed
correctly.
Solution:
Store the indexed storage B-tree internal 'K' value in the superblock.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Bug fix
Description:
H5T_BKG_TEMP was accidentally removed from library code, but is used by
application's datatype conversion routines.
Solution:
Revert removal of H5T_BKG_TEMP.
Platforms tested:
h5committest
Purpose: Maintenance after 1.6.0 release
Description: I cleaned the file. Please note that information about known
problems, config features, and tested platforms is there.
Please do not forget to modify those sections too!
Solution:
Platforms tested:
Misc. update:
Update configuration
Description:
Now that we've split the tree (Yea!), change the main trunk back to debug
on and production off.
Also, change the version # to 1.7.0
update for release
Description:
Set the version information to 1.6.0-pre1 to get ready for
v1.6 release.
Platforms tested:
h5committested.
Misc. update:
Purpose: Maintenance for the 1.6.0 release.
Description: I updated compilers information for Crays and added
h5fc and h5c++ to the new tools sections in REELASE.txt.
I added instructions for how to build HDF5 using Intel compilers
on Windows.
Solution:
Platforms tested:
Misc. update:
Purpose: Maintenance
Description: Bumped the version number to 1.5.62 after creating
a tar ball for the fourth round of testing.
Solution: run bin/h5vers -s scipt on arabica
Platforms tested: N/A
Misc. update:
Purpose: Maintenance for the third round of testing
Description: Increased the version number to 1.5.59 after creating
a tar ball for testing.
Solution:
Platforms tested:
Misc. update:
Bug fix
Description:
The dataset's modification time was getting set whenever raw data was
written with H5Dwrite. Unfortunately, this is a metadata change (which are
required to be performed collectively) and H5Dwrite may be called independently
from a parallel program, resulting in metadata cache corruption and/or program
hangs.
Solution:
Don't update the modification time when raw data it written. :-(
Platforms tested:
FreeBSD 4.8 (sleipnir) w/parallel
h5committest
Misc. update:
Noted in release notes and also sent to Frank for updating the docs.
Code cleanup/new features.
Description:
Clean up whitespace & formatting.
Make the -S (simple) output display a dataset's modification time in UTC
rather than local timezone.
Display variable-length datatype information.
Simplified & enhanced bitfield datatype information displayed.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
API tweak.
Description:
The H5Sget_select_bounds() API call was using hsize_t arrays for retrieving
the 'start' and 'end' coordinates, which is counter to the rest of the dataspace
API.
Solution:
Change the arrays to be hssize_t instead.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
FreeBSD 4.8 (sleipnir) w/parallel
h5committested
Misc. update:
Updated all docs for this change.
Added 1.4 compatibility #ifdef's
Purpose: Maintenance for 1.6 release
Description: I did more cleanup and restructuring of the file.
Please take a look at the file when you have a chance.
Thank you!
Solution:
Platforms tested:
Misc. update:
Purpose: Maintenance for 1.6 release
Description: HISTORY.txt was not updated in 1.5 branch since 1.2.0 release.
RELEASE.txt had a lot of redundant info that is now in HISTORY.txt
INSTALL_Windows_withF90.txt was lost in 1.5 branch.
Solution: I updated HISTORY.txt to include release history up to 1.4.5 release.
After that I removed all information related to the 1.4.* releases
from RELEASE.txt file. "Platforms Tested" section was brought up
to date based on the information provided by developers for 1.6.0 first
round of testing. RELEASE.txt still needs a lot of work.
INSTALL_Windows_withF90.txt is also a first draft, more work is needed.
Platforms tested: N/A
Misc. update:
Purpose: Maintenance for the second round of testing
Description: Used bin/h5vers to change version number to 1.5.55
Solution:
Platforms tested: arabica
Misc. update:
Purpose: Maintenance
Description: I changed release version to 1.5.54.
Email about tetsing of 1.5.53 will be send after this checkins.
Solution: used bin/h5vers -s to change the version number
Platforms tested: Done on arabica
Misc. update:
New feature/enhancement
Description:
Chunked datasets are handled poorly in several circumstances involving
certain selections and chunks that are too large for the chunk cache and/or
chunks with filters, causing the chunk to be read from disk multiple times.
Solution:
Rearrange raw data I/O infrastructure to handle chunked datasets in a much
more friendly way by creating a selection in memory and on disk for each chunk
in a chunked dataset and performing all of the I/O on that chunk at one time.
There are still some scalability (the current code attempts to
create a selection for all the chunks in the dataset, instead of just the
chunks that are accessed, requiring portions of the istore.c and fillval.c
tests to be commented out) and performance issues, but checking this in will
allow the changes to be tested by a much wider audience while I address the
remaining issues.
Platforms tested:
h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
Feature Add
Description:
Added knob so that the programmer can enable or disable GPFS
hints during runtime instead of having it only enabled at
configure/compile time. Some of the public APIs were changed
to add an extra parameter for this option...
Platforms tested:
Blue (LLNL). It only affects the MPI/POSIX driver, so no need
to test it on non-GPFS platforms.
Misc. update:
Update
Description:
The H5FC was already in this doc...but under "bug fixes". Removed
that since it's really a new feature.
Platforms tested:
IBM/PC Jr.
Misc. update:
Bug fix
Description:
This fixes a bug in the low-level metadata caching code in the library
which could possibly lose metadata during file I/O when a lot of objects are
inserted into a group.
This also fixes a couple of (similar) fencepost bugs in the B-tree
deletion code.
Solution:
For the metadata bug - call the low-level driver's 'write' routine instead
of H5FD_write.
For the B-tree bug - include the correct number of keys.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
(h5committest is still not working for me on burrwhite)
Misc. update:
Purpose:
Maintenance
Description:
* Added support for generic properties.
* Added support for time allocation properties.
* Added support for variable length datatypes (only datatypes based
on INTEGER, REAL and CHARACTER Fortran types are supported).
* added some missing functions
Solution:
I am checking in new Fortran APIs and their man pages to support
1.5 features listed above.
Not all APIs have tests yet. APIs were written in Fall 2002, and I am afraid
that I will loose the code or totally forget what I did if I wait longer. ;-)
Platforms tested:
arabica (fortran), eirene (fortran), modi4 (parallel, fortran)
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
New feature for developers.
Description:
Added "function stack" tracing to library. This allows developers (there
is no public API) to call H5FS_print within the library and get a listing
of the functions traversed to reach that point in the library. Eventually,
I may add support for reporting the parameters to each function also...
Mainly for debugging parallel I/O programs, but I think it will come in
handy in other cases also.
The function stack tracking is controlled with a configure switch:
--enable-funcstack, which defaults to enabled currently. When we branch
for 1.6, we should change the default setting on the branch to be disabled.
Also, added a destructor to the thread-specific keys when thread-safety is
turned on in the library. Otherwise, they were leaking memory and causing
difficult to debug errors in threaded programs (like the test/ttsafe test).
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) w/thread-safety enabled.
Misc. update:
Updated MANIFEST with new files added (src/H5FS.c & src/H5FDprivate.h)
Update release_docs/RELEASE with thread-safety bug fix.
Bug Fix
Description:
When calling H5Fopen with the core VFL driver, but without the
H5F_ACC_CREAT flag goes ahead and creates a memory file.
Solution:
Check for the H5F_ACC_CREAT flag before allowing the memory file to be
created.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Update
Description:
Added description of the configure bugfix for people specifying
--with-*=/usr/include,/usr/lib in their scripts during configure...
Bug fix
Description:
Currently, when the library encounters an object header message that isn't
know, it fails to open that object in the file.
Solution:
Allow the library to skip over the unknown object header message and
continue to process the remaining messages, in the hope that the skipped
message isn't important later. If it is important, it will be caught at
a higher level of the library.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
Purpose:
add "shuffle" filter note
Description:
Solution:
Platforms tested:
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Purpose:
New function.
Description:
H5Dget_offset returns the offset of a dataset's data relative to the
beginning of a file.
Platforms tested:
arabica(simple function, one test should be enough.)
Misc. update:
RELEASE.txt
New feature
Description:
Add support for scalar dataspaces in parallel I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Also, FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Update release_docs/RELEASE for bug fixes, new features, etc.
Purpose:
Change release note for new functions(H5Gget_comment, H5Arename,
H5Aget_storage_size).
Misc. update:
Update release_docs/RELEASE for bug fixes, new features, etc.
Purpose:
Added missing fortran functions h5set(get)_buffer_f.
Also added docs and tests for them.
Solution:
Currently functions do not accept conversion and background buffers.
This corresponds to H5set(get)_buffer call with buffer pointers
set to NULL. If there is a demand, I can overload the functions
to have new parameters and go through all trouble creating functions
for all supported datatypes.
Platforms tested:
Solaris 2.7, Linux 2.2., IRIX64-6.5
Purpose:
Added new F90 APIs
Description:
I added new F90 APIs, tests, and documentation for the following
functions:
h5fget_obj_count_f h5pequal_f h5tget_member_index_f
h5fget_obj_ids_f h5pget_fclose_degree_f
h5pset_fclose_degree_f
Documentation for exisiting functions was missing:
h5freopen_f, h5fflush_f, h5fmount_f, h5unmount_f, h5fget_create_plist_f,
h5fget_access_plist_f.
Platforms tested:
Solaris 2.7, Linux 2.2 and IRIX64-6.5
Purpose:
New API functions
Description:
Added API functions to return pointer to low-level file handle
(H5Fget_vfd_handle and H5FDget_vfd_handle) and related property list
setting functions(H5Pset_family_offset and H5Pset_multi_type).
Platforms tested:
Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
Purpose:
Added new fortran functions
Description:
Source code, tests and documentation for h5glink2_f and h5gmove2_f
functions were added. I also added missing man page for h5glink_f
Platforms tested:
Solaris 2.7, IRIX64-6.5 and Liniux 2.2
Purpose:
Added missing fortran functions.
Description:
Four Library Fortran API functions have been added:
h5get_libversion_f, h5_check_version_f, h5garbage_collect_f and
h5dont_atexit_f. Only first two functions were tested.
Documentation file and RELEASE.txt were updated.
Platforms tested:
Solaris 2.7, IRIX64-6.5 and Linux 2.2
API name change
Description:
Change all "space time" references to "alloc time", including API functions
and macro definitions, etc.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
Code cleanup/New Feature
Description:
Improve the space allocation in the file by re-using freed space more
effectively.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel
