New feature.
Description:
Added tests to verify correct operation of re-using file space.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup
Description:
Added macro for reporting line #'s of test failures more easily.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Bug fix
Description:
Return correct value (1 instead of -1) on test failure.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Code cleanup
Description:
Switched some system/library calls to use wrappers.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Bug fix
Description:
This fixes a bug in the low-level metadata caching code in the library
which could possibly lose metadata during file I/O when a lot of objects are
inserted into a group.
This also fixes a couple of (similar) fencepost bugs in the B-tree
deletion code.
Solution:
For the metadata bug - call the low-level driver's 'write' routine instead
of H5FD_write.
For the B-tree bug - include the correct number of keys.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
(h5committest is still not working for me on burrwhite)
Misc. update:
Code cleanup & new tests
Description:
Updated szip and shuffle testing to reflect changes to API functions.
Added testing for new API functions: H5Pmodify_filter, H5Pget_filter_by_id
and H5Pall_filters_avail.
Added testing for "can apply" and "set local callbacks.
Cleaned up various things, also.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
Purpose: This a fix for bug #921
Description: On platforms where big test runs, it creates a set of
"solid" 1GB files. Test will take forever (modi4) or will fail
if quotas are exceeded.
Solution: Default library behavior is to allocate dataset space earlier and
to write fill values. I changed the test to use late space allocation
and never write fill values.
Platforms tested: True64 (PSC), modi4 (sequential and parallel), arabica with 64-bit
option(test was skipped there). I also enabled szip and it passed :-)
Misc. update:
Code cleanup
Description:
Updated copyright.
Removed szip-specific testing in favor of standard testing, now that bugs
in the szip filter are fixed.
Updated to latest version of H5Pset_szip API
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
To add tests for szip compression in HDF5
Description:
Three tests were added:
1) szip filter itself
3) szip + shuffle + fletch32
2) fletch32 + shuffle + szip
Solution:
Since H5Dget_storagesize returns 0(Quincey can fix it or it may be gone),
I create another function called test_internal_filter_szip, which is the almost the same
as test_internal_filter but comments out H5Dget_storage_size function.
Platforms tested:
Since there are changes of configure.in and configure,I didn't use h5committest.
I tested with four platforms.
1) Linux 2.4 (eirene) 2) Solaris 2.7(arabic
3) windows 2000(VS 6.0) 4) SGI IRIX6.5-64(modi4)
For test 1)-3), only basic C tests were done For modi4 test, I tested 64-bit C,parallel and fortran.
All tests passed, except a warning message from szip library when checksum is used in some order, which doesn't cause any real problems.
Misc. update:
Update
Description:
Updated the Copyright statement
Platforms tested:
Linux (This change is only in the comments, so I just check that the
modules still compile)
Misc. update:
New test feature
Description:
Added the VFD_LIST definition to support the "make check-vfd" tests.
Platforms tested:
h5committested.
Since check-vfd is not invoked by default, I went to each machine
to invoke them by hand. The target worked fine but there were
errors reported from those driver tests. (Those errors were beyond
the scope of this make target test.)
Misc. update:
New internal feature
Description:
Add internal API for building and working with heaps (H5HP). This will be
used for the LRU algorithm in the new metadata cache code.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Code cleanup
Description:
Added more information to information printed for test failure.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Code/comment cleanup
Description:
Made more information about reference-counted strings private to the
H5RS package by trimming the typedef used in H5RSprivate.h and switching
from using macros for a couple of operations to using functions.
Updated copyright information for a few of these files.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Code cleanup
Description:
Clean up miscellaneous warnings which have crept into the code.
Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD.
Adjust gcc compiler flags to be more concise for production mode.
Refactor the H5O code so that there is a stronger boundary between code
in the H5O package and code in the library which just calls H5O routines.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2
Misc. update:
Update MANIFEST if you add or remove any file.
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
Bug fix
Description:
The "system scope" for threads isn't supported on all platforms.
Solution:
Add detection of this feature to the configure script and check for
"H5_HAVE_SYSTEM_SCOPE_THREADS" in the appropriate places.
Platforms tested:
modi4 w/threadsafe
New feature for developers.
Description:
Added "function stack" tracing to library. This allows developers (there
is no public API) to call H5FS_print within the library and get a listing
of the functions traversed to reach that point in the library. Eventually,
I may add support for reporting the parameters to each function also...
Mainly for debugging parallel I/O programs, but I think it will come in
handy in other cases also.
The function stack tracking is controlled with a configure switch:
--enable-funcstack, which defaults to enabled currently. When we branch
for 1.6, we should change the default setting on the branch to be disabled.
Also, added a destructor to the thread-specific keys when thread-safety is
turned on in the library. Otherwise, they were leaking memory and causing
difficult to debug errors in threaded programs (like the test/ttsafe test).
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) w/thread-safety enabled.
Misc. update:
Updated MANIFEST with new files added (src/H5FS.c & src/H5FDprivate.h)
Update release_docs/RELEASE with thread-safety bug fix.
Bug Fix
Description:
When calling H5Fopen with the core VFL driver, but without the
H5F_ACC_CREAT flag goes ahead and creates a memory file.
Solution:
Check for the H5F_ACC_CREAT flag before allowing the memory file to be
created.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Bug fix
Description:
Currently, when the library encounters an object header message that isn't
know, it fails to open that object in the file.
Solution:
Allow the library to skip over the unknown object header message and
continue to process the remaining messages, in the hope that the skipped
message isn't important later. If it is important, it will be caught at
a higher level of the library.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Code cleanup
Description:
Various code cleanups to allow the development branch to be compiled with
a C++ compiler (i.e. CC=g++ )
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) C++
Update test
Description:
Some of the performance improvements changed the errors that are reported
for creating a duplicated dataset in a file, causing this test's hard-coded
errors to check for to fail.
Solution:
Updated error stack reported for dataset creation failure.
Platforms tested:
FreeBSD 4.7 (sleipnir) w/threads
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
Purpose:
bug fix
Description:
some arrays were too big, running out of memory limit for some machines.
Solution:
change to dynamic memory allocation.
Platforms tested:
arabica, sleipnir
Purpose:
bug fixed for windows
Description:
windows fopen functions by default opens a text mode file. It will fail for HDF5 file.
So to use HDopen HDread HDclose for test.
Solution:
Change from fopen, fread, fclose to HDopen, HDread, HDclose.
Platforms tested:
linux2.2.18smp, windows 2000
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Purpose:
Bug fix(kinda)
Description:
Windows handle "fopen, fseek, fread, fclose" C functions slightly different.
Solution:
change to HDopen, HDlseek, HDread, HDclose to cover both Unix and Windows.
Platforms tested:
arabica
Bug fix
Description:
Much of the VL-string testing code was recently disabled accidentally.
Solution:
Re-enable it and add test for writing/reading large string attributes.
Platforms tested:
FreeBSD 4.7 (sleipnir) - small enough to not need triple-check
Purpose:
New feature to H5Dget_offset
Description:
If user block is set, H5Dget_offset should be able to return the absolute
offset from the beginning of file.
Platforms tested:
eirene, arabica
Bug fix
Description:
Array testing routine is creatint huge arrays on the function stack
which causes a segmentation fault on Linux & FreeBSD when threadsafe
support is enabled.
Solution:
Allocate data for test dynamically instead of automatically.
In general, this should be the preferred method for all data arrays.
Platforms tested:
FreeBSD 4.7 (sleipnir) w/threadsafe enabled.
Code Cleanup & New Tests
Description:
tmisc.c:
Switched from using H5_HAVE_COMPRESSION flag in favor of
H5_HAVE_FILTER_DEFLATE.
dsets.c:
Switched from using H5_HAVE_COMPRESSION flag in favor of
H5_HAVE_FILTER_DEFLATE.
Refactored I/O filter tests to allow new filters to be [more] easily
added.
Added tests for shuffle & deflate+shuffle I/O filters (if the filter(s)
are enabled).
Added test for creating a new dataset with a filter that is not
available.
Added test for attempting to read a dataset created with a filter that
is not available.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Purpose:
Add a testing case for shuffling algorithm.
Description:
A control test:
Since shuffling 8-bit array should not change the original result; so we use a
8-bit array to test whether after shuffling the array keeps the same values.
Solution:
Platforms tested:
eirene,arabica,modi4
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Purpose:
New function.
Description:
H5Dget_offset returns the offset of a dataset's data relative to the
beginning of a file.
Platforms tested:
arabica(simple function, one test should be enough.)
Misc. update:
RELEASE.txt
New feature
Description:
Add support for scalar dataspaces in parallel I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Also, FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Update release_docs/RELEASE for bug fixes, new features, etc.
Improvement
Description:
fillval will fail to find the pre-exist data file if it is run
in --srcdir mode without setting $srcdir properly. This is
setup properly in the Makefile but unsuspecting users trying just
./fillval were puzzled by the failure.
Solution:
put in a more descriptive error message with a possible remedy.
Platforms tested:
Modi4 only since this is just adding a printf statment.
Code cleanup/Bug Fix
Description:
Added 10-20 more test cases to the ID->name code. Things seem more stable
now.
Platforms tested:
FreeBSD 4.6 (sleipnir) w and w/o parallel
Linux 2.2.x (eirene) w/FORTRAN & C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
New test
Description:
Add test to check for correct generic property class reference counting.
Platforms tested:
FreeBSD 4.6 (sleipnir) w and w/o parallel
Linux 2.2.x (eirene) w/FORTRAN & C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
New feature
Description:
If backing store for a core VFD file is turned on, remove the file created.
Platforms tested:
FreeBSD 4.6 (sleipnir) w and w/o parallel
Linux 2.2.x (eirene) w/FORTRAN & C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
Code cleanup
Description:
Clean up some compile warnings and have these tests create their data files
with the correct file drivers and directories, etc.
Platforms tested:
FreeBSD 4.6 (sleipnir) w and w/o parallel
Linux 2.2.x (eirene) w/FORTRAN & C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
Code cleanup
Description:
Added more test datafiles to clean directive.
Platforms tested:
FreeBSD 4.6 (sleipnir) w and w/o parallel
Linux 2.2.x (eirene) w/FORTRAN & C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN & parallel
Internal feature.
Description:
To complement the "get the class's path" functionality, there needs to be
a way to open a generic property class with a path.
Solution:
Implement the "open a class by path" functionality. Also, add internal
testing routines.
Platforms tested:
FreeBSD 4.6 (sleipnir) (too small for triple testing)
New internal feature.
Description:
Need some way to determine the "full path" for a generic property class,
i.e. where is this class in the class hierarchy, in relation to its parent
class, etc.
Solution:
Added an internal function "H5P_get_class_path" and a testing function
"H5Pget_class_path_test" that builds the full path of a generic property
class back to the top of its class hierarchy.
This implementation uses '/' characters to delimit the components of the
class path, but no special cases are currently supported for having a '/'
character as part of the actual class name. Should this become an issue,
code to support (and test) it will need to be added.
Platforms tested:
FreeBSD 4.6 (sleipnir) (not major enough to justify triple-test)
Bug fix
Description:
H5Pequal's algorithm for comparing property lists wasn't allowing property
lists that are equal but had the properties inserted in a different order
in the list's hash table.
Solution:
Re-did the algorithm a bit to compare the properties better.
Platforms tested:
FreeBSD 4.6 (sleipnir) (minor enough to not need triple testing)
Purpose:
a bug fix
Description:
modify the condition check for the file close degree.
Platforms tested:
Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
Purpose:
a bug fix
Description:
When a file is opened for a second time, file close degree is supposed to
be either the same as the first open, or default as the first open is also
default.
Platforms tested:
Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
Purpose:
Testing program for new APIs.
Description:
Added API functions to return pointer to low-level file handle
(H5Fget_vfd_handle and H5FDget_vfd_handle) and related property list
setting functions(H5Pset_family_offset and H5Pset_multi_type).
Platforms tested:
Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
Purpose:
New API functions
Description:
Added API functions to return pointer to low-level file handle
(H5Fget_vfd_handle and H5FDget_vfd_handle) and related property list
setting functions(H5Pset_family_offset and H5Pset_multi_type).
Platforms tested:
Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
Code cleanup
Description:
Clean up ID->name code:
- Reformat to better match library coding standard
- Changed several algorithms to be more efficient
- Integrated into library more smoothly
Platforms tested:
eirene w/FORTRAN & C++
arabica w/FORTRAN
modi4 w/FORTRAN & parallel
sleipnir
Purpose:
bug fix
Description:
memory leak regarding the ID to name buffer
Solution:
added a new function H5G_free_ent_name that is called on several places of the library
Platforms tested:
windows 2000
linux, with cpp
solaris, with fortran, cpp
irix64, with parallel, fortran
API name change
Description:
Change all "space time" references to "alloc time", including API functions
and macro definitions, etc.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
Purpose:
Added 'ID to name' support
Description:
There is a new API function H5Iget_name
Most of the changes are on H5G.c , regarding the symbol table entry struct H5G_entry_t
which has 2 new fields 'name' and 'old_name'
A new private function was introduced H5G_ent_copy, that does a deep copy
between 2 symbol table entries
The test file is getname.c
Platforms tested:
windows 2000, Linux, Solaris
Code cleanup/More tests
Description:
Cleaned up some compiler warnings and wrote additional tests for space
allocation and storage size routines.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel. Will be testing on IRIX64
6.5 (modi4) in serial & parallel shortly.
Regression test for bug fix
Description:
Adjust selection so chunked data needs to be read from pre-allocated chunks
w/filters, to verify that filter is applied correctly.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/parallel
Purpose:
Design for compact dataset
Description:
Compact dataset is stored in the header message for dataset layout.
Platforms tested:
arabica, eirene.
Additional regression tests & bug fixes
Description:
There was no testing for the H5Dget_storage_size function and it seemed to
be having problems with compressed, chunked datasets, so write some tests
to verify that its working correctly.
Also, fix case for allocating storage early for chunked datasets
Platforms tested:
FreeBSD 4.6 (sleipnir) serial & parallel
Code cleanup
Description:
Clean up a few warnings which were showing up with --enable-production
turned on.
Platforms tested:
FreeBSD 4.6 (sleipnir) serial & parallel
Update, Bug Fix, and Feature Add
Description:
- Updated how AC_{ENABLE,WITH} help messages were being generated.
Autoconf now gives you an AC_HELP_STRING macro to use to create
them.
- Fixed the problem with Linux LFS on RedHat 7.3 machines. It wasn't
finding getdents64(), so we can't rely on that being present
anymore...
- Added GPFS detection and setting.
- Updated how compression is specified. It's no longer necessary to
test for HAVE_ZLIB_H, HAVE_LIBZ, and HAVE_COMPRESS2. The one macro
{H5_}HAVE_COMPRESSION takes care of all of these.
Solution:
Changed the check for Linux LFS from looking for getdents() to
looking at the version number of the kernel (using the uname -r
command). You can still override with the --enable-linux-lfs switch
if you really believe that your <2.4 kernel has LFS support.
Platforms tested:
Linux (2.2 and 2.4)
Purpose:
New Test VL memory leak.
Description:
Test to see if VL type memory leaks when data is overwritten.
Platforms tested:
Linux 2.2(eirene), IRIX 6.5(paz).
Purpose:
Bug fix.
Description:
This test fails on TRUE64 system because a compound variable(fill_ctype
in test_rdwr) wasn't initialized.
Solution:
Initialize to zero.
Platforms tested:
Pittsburg's True64(lemieux) system.
Bug fix
Description:
I/O on "Regular" hyperslab selections could fail to transfer correctly
if the number of elements in the selection's row did now fit "evenly"
into the buffer being used for the transfer.
Solution:
Correct the calculation of the block & count offsets within the optimized
"regular" hyperslab routines.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Code cleanup
Description:
The test for inserting a compound datatype into itself (which should fail)
is issuing warnings about this failure.
Solution:
Turn the warnings off during this test.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Purpose:
Bug #774 fix
Description:
Added a test for the H5Tenum_nameof and H5Tenum_valueof functions.
Values that do not exists in the enumeration type are supplied to
the functions; functions should fail instead of succeeding.
Platforms tested:
Solaris 2.7 and Linux 2.2.18
Bug Fix
Description:
H5Dcreate and H5Tcommit allow "empty" compound and enumerated types (i.e.
ones with no members) to be stored in the file, but this causes an assertion
failure and is somewhat vapid.
Solution:
Check the datatype "makes sense" before using it for H5Dcreate and
H5Tcommit.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Bug fix (bug #777)
Description:
Current code allows a compound datatype to be inserted into itself.
Solution:
Check if the ID for the member is the same as the ID for the compound
datatype and reject it if so.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Bug Fix for bug #789
Description:
Creating a 1-D dataset region reference caused the library to hang (go into
an infinite loop).
Solution:
Corrected algorithm for serializing hyperslab regions.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Purpose:
Bug fix (#699), fix provided by a user, approved by Quincey
Description:
When a scalar dataspace was written to the file and then
subsequently queried with the H5Sget_simple_extent_type function,
type was reported H5S_SIMPLE instead of H5S_SCALAR.
Solution:
Applied a fix (see bug report 699) and modified out test program
to exercise the case.
Platforms tested:
Solaris 2.7 and Linux 2.2.18
Test Bug Fix
Description:
Under certain [obscure] circumstances, an object header would get paged out
of the metadata cache, and when it was accessed again and brought back into
the cache, and immediately had additional metadata added to it (an
attribute, usually, or perhaps adding an object to a group), and needed to
be extended with a continuation message, but there was no room in any
existing object header chunks for the continuation message and an existing
object header message needed to be moved to the new object header chunk (I
told you it was obscure :-), the object header message moved to the new
chunk (not the new metadata being added) would get corrupted. *whew* :-)
Solution:
Actually copy the "raw" object header message information of the object
header message being moved to the new chunk, instead of relying on the
"native" object header message information being re-encoded when the object
header is flushed. This is because when an object header is paged out of
the metadata cache and subsequently brought back in, the "native"
information pointer in memory is reset to NULL and only the "raw"
information exists.
[Actually, this additional testing doesn't trigger the bug, which needs
_lots_ of objects to be created and accessed, but it does execise the
object header continuation code more than other tests in the library.]
Platforms tested:
Solaris 2.7 (arabica) & FreeBSD 4.5 (sleipnir)
Feature Enhancement
Description:
We can now set the MPI_Info object to more than one key/value
pairing. By setting the HDF5_MPI_INFO environment variable to a
semicolon separated list of "key=value" pairs, we can set the
MPI_Info variable to them.
Platforms tested:
Linux
Feature
Description:
moved the routines of setting up and dumping MPI-info object to
test library so that it is avaiable for all tests too.
Platforms tested:
modi4(pp), eirene (serial)
Purpose:
added more tests to the H5set_extent function
Description:
there was a bug in the H5S_select_fill call when the fill value was not defined
added 2 more set of tests that call H5set_extent without the fill value being defined
there are now 4 sets of tests:
Test H5Dset_extent with chunks on the raw data cache, with and without fill value
Test H5Dset_extent with chunks written to file (b-tree routines), with and without fill value
Platforms tested:
w2000
linux
Bug Fix
Description:
Corrected int vs. unsigned short error that only showed up on big-endian
machines.
Also, add more testing to verify that the coordinates reported for each
element iterated through with H5Diterate are correct.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Bug Fix
Description:
Selection offsets were not being used correctly when iterating through
all hyperslabs selections and point selections.
Solution:
Use the selection offset appropriately.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Migrate from configure macros of XYZ_ABC to H5_XYZ_ABC
Description:
configure generates many macros definitions on the fly and
were stored in src/H5config.h which is included by H5public.h.
But other software that uses hdf5 may also run their own configure.
There can be a clash in macro name space. We decided awhile ago
to prepend all generated macros with "H5_" to avoid conflicts.
The process has started and this commit completes it (at least attempt
to).
Solution:
Many macros symbols (e.g. SIZEOF_xxx and HAVE_xxx were changed to
H5_SIZEOF_xxx and H5_HAVE_xxx). Then H5private.h no longer includes
H5config.h. This cuts H5config.h away from HDF5 source code.
Pending issues:
The module of fortran and pablo are to be resolved in a different
commit.
Platforms tested:
eirene (parallel), arabica (solaris 7 --enable-fortran, --enable-cxx)
Purpose:
New feature
Description:
Allow H5Glink and H5Gmove to handle links across different locations.
Solution:
Added H5Glink2 and H5Gmove2 functions with new parameter of destination
location.
Platforms tested:
Linux 2.2(eirene)
Bug Fix.
Description:
When the fill value for a dataset is not set, the size is set to zero,
causing problems with filling unused areas in previously defined chunks
(i.e. when the dataset shrinks in size)
Solution:
Use the size of the elements in the dataset directly, instead of relying on
the size of the fill value (which must be the same size).
Platforms tested:
FreeBSD 4.5 (sleipnir)
Code cleanup
Description:
Previously, the I/O pipeline (pline), external file list (efl) and fill-
value (fill) structs were passed down the raw data function call chain,
even into and/or through functions which didn't use them. Since all three
of these pieces of information are available from the dataset creation
property list, just pass the dataset creation property list down the
function call chain and query for the information needed in a particular
function.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Bug fix
Description:
$(LT_RUN) was used to execute ./H5detect in src/Makefile.in.
$RUNTEST was set to $(LT_RUN) which was used to execute sequential
executable. $(LT_RUN) is "../libtool --mode=execute". But libtool
invokes some commands that are not supported in the Tflops machine.
That caused failures during the build and check processes.
Solution:
Upon investigation, there does not seem to be need to use $(LT_RUN)
any more. The "libtool --mode=link" now generates a "fake" executable
that is actually a command-script file that can regenerate the real
executable with dynamic libraries hooks setup properly.
Undo all those $LT_RUN substitute and let $RUNSERIAL execute those
sequential executables.
Platforms tested:
Eirene(serial), modi4(parallel)
Purpose:
windows support of socket function
Description:
gethostname is treated as socket function in windows and
it is defined at winsock.h.
for every windows socket function to be called, it must start
with WSAStartup and end with WSACleanup
Solution:
Add WSAstartup and WSACleanup with WIN32 macro.
Platforms tested:
windows 2000, confirmed at linux 2.2.18
New feature
Description:
Added h5_show_hostname to display the hostname of the host in
which the process runs. It can help identify location of process
in multiple processes or batch launching environments.
Platforms tested:
Eirene (pp)
Feature
Description:
Added option -c to allow skipping file system Checking. This allows
users to run the test even if the program thinks it may fill up the
file system.
Platforms tested:
eirene
Code cleanup
Description:
Platform dependent code related to the struct stat and fstat
calls polluted source codes. Hard to maintain.
Solution:
Platform dependent code are moved to H5private.h and then internal
code can #include H5private.h. Repeat those macro definition for
the stdio and multi drivers since they area examples for writing
a virtual file driver. They must not use any internal code.
Platforms tested:
eirene (parallel), modi4 (serial including gass driver.)
Bug fix
Description:
When several level deep nested compound & VL datatypes are used, the data
in the nested compound datatypes is incorrectly sharing the same "background
buffer", causing data corruption when the data is written to the file.
Solution:
Allocate a separate background buffer for each level of the nested types
to convert. (Also allocate temporary background buffers for array
datatypes, where this sort of problem could occur also)
Added more regression tests to check for these errors.
Platforms tested:
FreeBSD 4.5 (sleipnir) & Solaris 2.6 (baldric)
Purpose:
added a new file
Description:
new file that contains the framework for a new test, regarding a new ID to name
function. for the moment the program does nothing
Platforms tested:
w2000
Purpose:
Bug Fix
Description:
Reading fill_old.h5 from fillval.c has problem to find from building
directory.
Solution:
prepend source directory into file name.
Platforms tested:
Linux 2.2
Purpose:
New feature
Description:
Fill-value's behaviors for contiguous dataset have been redefined.
Basicly, dataset won't allocate space until it's necessary. Full details
are available at http://hdf.ncsa.uiuc.edu/RFC/Fill_Value, at this moment.
These two file test backward compatibility with 1.4.
Platforms tested:
Linux 2.2.
Purpose:
test for the H5Dset_extent API function
Description:
there are 2 types of tests:
Test H5Dset_extent with chunks on the raw data cache
Test H5Dset_extent with chunks written to file
Platforms tested:
Windows 2000
SUN( arabica)
LINUX (eirene)
IRIX64 (modi4)
New Feature
Description:
Added new H5Dfill() routine to fill the elements in a selection for a
memory buffer with a fill value. This is a user API wrapper around some
internal routines which were needed for the fill-value modifications
from Raymond as well as Pedro's code for reducing the size of a chunked
dataset.
Platforms tested:
FreeBSD 4.5 (sleipnir) [and IRIX64 6.5 (modi4) in parallel, in a few
minutes]
Purpose:
New feature
Description:
Added a query function H5Tget_member_index for compound and enumeration
data types, to retrieve member's index by its name.
Platforms tested:
Linux 2.2
Bug Fix & Feature
Description:
The selection offset was being ignored for optimized hyperslab selection
I/O operations.
Additionally, I've found that the restrictions on optimized selection
I/O operations were too strict and found a way to allow more hyperslabs
to use the optimized I/O routines.
Solution:
Incorporate the selection offset into the selection location when performing
optimized I/O operations.
Allow optimized I/O on any single hyperslab selection and also allow
hyperslab operations on chunked datasets.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Purpose:
Bug fix(#697)
Description:
Variable-length string wasn't treated as string.
Solution:
Added character set and padding into VL string type.
Platforms tested:
FreeBSD
feature
Description:
Added -fsize <fsize> option which controls the family file size
used. This can be used to test the file size limits of a machine
or file system.
Platforms tested:
modi4 (passed even 20GB file size).
Eirene (started to fail when 2GB is used. Expected since linux 2.2
does not support large than 2GB file size.)
burrwhite (passed up to 4GB. Expected since Linux 2.4 supports larger
than 2GB file size.) Then it failed at 4GB and beyond. This is
probably a bug in the family file driver.
Bug Fix
Description:
Regression test for following bug:
The H5Gget_objinfo() function was not setting the 'fileno' field in the
H5G_stat_t struct passed in.
Solution:
Added a "file serial number" to each file currently open in the library
and put that in the 'fileno' field. If a file is opened twice (with
H5Fopen) and the VFL driver detects that it is the same file (i.e. the
two file structures have the same "shared file info" in the library's
memory structures), they will have the same serial number.
This serial number has two drawbacks:
- If a VFL driver doesn't/can't detect that two calls to H5Fopen with
the same file actually _are_ the same file, each will get a
different serial number
- If the same file is closed and re-opened, the serial number will be
different.
It is be possible to fix the second drawback for many VFL drivers, but it
would be a lot of effort and probably isn't worth it until we've got a
good reason to do it. Dunno if we'll ever be able to fix the first
drawback...
Platforms tested:
FreeBSD 4.5 (sleipnir)
VS: ----------------------------------------------------------------------
Bug Fix
Description:
Regression test for following bug:
When reading a contiguous hyperslab that spanned the entire dataset and
was larger that the type conversion buffer, the hyperslab routines need
to fill the type conversion buffer and then return to the I/O routines.
When the I/O routines resume the hyperslab operation, it was possible to
have a combination of coordinates which caused the hyperslab iterator
to incorrectly advance in the file, causing some data to be re-read or
re-written.
Platforms tested:
Linux (eirene)
Bug Fix
Description:
If a non-zero fill-value is used for a chunked dataset, any non-existent
chunked read with an "all" selection (or a contiguous hyperslab selection)
will return zero for those instead of the user's fill-value.
Solution:
Fixed I/O code to pass down fill-value to "optimized" I/O routines, so it
will be available to fill the user's buffer with.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Purpose:
fixed a bug that made a failure on H5Screate_simple, a 1D dimemsion array was declared instead of a 2D array
Platforms tested:
w2000 octopus
linux eirene
sun arabica
IRIX64 modi4
Cleanup
Description:
Added set_extend.h5 file to the CLEANUP macro in the Makefile so that
it'll be removed when the other temp files are removed.
Platforms tested:
Linux
Bug fix & feature add
Description:
Added new API function H5Sget_select_type to determine type of selection in
a dataspace. Return values are defined by the H5S_sel_type enumerated type
in H5Spublic.h
Also, hyperslab operations involving a "all" or "none" selection are not
generating the correct resulting selections.
Solution:
Added regression tests to verify correction operation of hyperslab
operations against "all" and "none" selections. New tests also verify
H5Sget_select_type() API call.
Platforms tested:
FreeBSD 4.5 (sleipnir)
