Modify the cache code (H5C) to support automatic cache resizing to
adapt to the work load at run time.
Description:
Different applications require different sized caches to maintain
an acceptable hit rate. This set of changes attempts to provide the
ability to adjust to circumstances automatically.
Solution:
Added highly configurable code to allow the user to either set a
fixed cache size, or allow the cache to grow and shrink according to
conditions.
If enabled, cache size increases are triggered when the hit rate
drops below a user specified threshold in a user specified interval.
Cache size reductions (if enabled) are triggered when either the
hit rate exceeds some user specified threshold over a user specified
interval, when the cache contains "enough" entries that haven't been
accessed for a user specified interval, or some mix of the above.
See the header comments on the H5C_auto_size_ctl_t structure in
H5Cprivate.h for further details.
At present, the cache resize configuration options are not
accessible via the user API. Must add this.
Platforms tested:
h5committested, heping (serial), and copper (parallel)
Misc. update:
New feature
Description:
Add callback iteration routine to skip lists.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Bug fix
Description:
Correct integer overflow situation for computing the size of a dataset.
Solution:
Rachet one of the dimensions down far enough that the total size of the
dataset's raw data doesn't overflow a 64-bit int.
Platforms tested:
FreeBSD 4.10 (sleipnir)
IRIX 6.? (tesla) w/-n32
Too minor to require h5committest
New feature
Description:
Allow skip list keys to be strings (needed for property list support)
Platforms tested:
FreeBSD 4.10 (sleipnir)
Too minor to require h5committest
New internal feature
Description:
Add a "release" routine to the skip lists, which releases all the nodes
in the skip list, but keeps the skip list active.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Linux 2.4 (verbena)
Too minor to require h5committest
Code optimization
Description:
Retarget metadata cache code from using TBBT routines to using new skiplist
routines, for a reasonable speedup (when dealing with dirty metadata)
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
Too minor to require h5committest
Add new internal data structure
Description:
Add an implementation of skip lists to the library (see comment in
src/H5SL.c for references to the papers describing them) as a potential
replacement for our current threaded, balanced binary tree container.
Skip lists are much simpler to implement and should be faster to use.
Also, added new error codes to release branch, so bump the minor version
number to indicate that the library is no longer perfectly compatible with
the 1.6.3 release.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
Too minor to require further testing (the skip lists aren't actually
used by any library code yet)
Cleanup & add test
Description:
Clean up test makefile a bit and add metadata cache test to 1.6 branch.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Too minor to require h5committest
Purpose: Correction to previous bug fix. This checkin is mainly to let user
test his program.
Description: For family driver bug(#213), the previous fix was imperfect. The problem
was when user create family file, the member size information wasn't saved in file. When
the file was re-opened, the library simply use the size of 1st member file as member size.
Solution:
When file is re-opened, member size passed in from access property
is checked to see if it's reasonable. If there is only 1 member
file, member size can't be smaller than current member size.
If there are at least 2 member files, member size can only be equal
to the 1st member size.
Platforms tested: h5committest and fuss
Description: H5Pset_fapl_family sets family member size only for creating
new file. The file doesn't keep this size information. When the file is
re-opened, the size of first member file is used as the member size.
Solution: Assume user knows the original member size and sets it through
H5Pset_fapl_family. That will be the member size. User can pass in value 0
as member size if he doesn't know the original member size. Library will
choose the size of current first member size as the member file size.
Platforms tested: h5committest and fuss.
Bug fix, sorta.
Description:
Revert change to H5Pget_data_transform() which changed len of buffer
returned to be inconsistent with H5Iget_name(), etc. We should discuss this
change and apply it to all the similar functions if we decide to change the
return value.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Too minor to require h5committest
Code clean-up
Description:
Rewrote code to make it simpler.
Added more data type tests
Solution:
Simplified code a lot in preparation for addition of new test cases.
Platforms tested:
heping + sol + copper
Code cleanup
Description:
Change loops for comparing results to use 2-D array lookups, since that's
what we've really got and the older 1-D array lookups was making the PGI C
compiler whine about array bounds issues.
Platforms tested:
Linux 2.4 (heping) w/PGI C
Not relevent to other platforms, no h5committest
Bug fix.
Description:
dtransform test still failed on windows because it was attempting to create a
ullong dataset.
Solution:
Removed creation of ullong dataset as well if H5_ULLONG_TO_FP_CAST_WORKS was
not defined.
Platforms tested:
windows
Bug fix.
Description:
unsigned long long test fails on windows because windows cannot convert from
ullong to double
Solution:
Skip unsigned long long test if H5_ULLONG_TO_FP_CAST_WORKS is not defined.
Platforms tested:
eirene and will do windows test asap.
Added tests
Code cleanup
Description:
New tests added for new version of H5Pget_data_transform and additional tests
to make sure that error conditions are handled properly by both
H5Pset_data_transform and H5Pget_data_transform.
Also cleaned up code to make it more compact.
Platforms tested:
sol + eirene + copper
Misc. update:
Code cleanup
Description:
Clean up minor warnings and align with release branch.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & FPH5
Solaris 2.7 (arabica) w/production mode
Linux 2.4 (heping) w/C++ and FORTRAN
"Improvement"?
Description:
Long double is a standard C89 type. Promoted it from
a conditional tested type to an assumed supported type.
Platforms tested:
H5committested.
Misc. update:
Description: "char" was considered as always "signed char" in data type conversion. However, ISO C leaves
the definition of "char" to individual implementation. i.e. for IBM AIX C compiler, it's treated as "unsigned
char".
Solution: Changed all "char" to "signed char". Don't even do "char" anymore because its definition is up
to each vendor.
Platforms tested: h5committest
Add code more code to test H5Pget_data_transform and H5Pset_data_transform
Description:
Added a test to verify that H5Pset_data_transform will correctly replace a
data transform with a new one and properly use the new transform on the data.
Added a test to verify that H5Pget_data_transform properly returns the
transform string.
Platforms tested:
sol + eirene + copper
Misc. update:
Bug fix
Description:
Fix core dump when flushing a file with a newly created attribute which
hasn't had a value written to it still open.
Solution:
Write the attribute fill value when appropriate.
Platforms tested:
FreeBSd 4.10 (sleipnir)
Linux 2.4 (verbena)
Solaris 2.7 (arabica)
Skip test.
Description:
Known failure in char and schar types for dtransform test on AIX and SGI.
Solution:
Temporary skip this test until bug is fixed.
Platforms tested:
copper (minor change)
Bug fix.
Description:
If a subtest failed, the error would not propogate properly to the main
function, thereby resulting in a succesful run, even though the test failed.
Solution:
Fixed error return values. The dtransform test will now fail for real.
Platforms tested:
sol + eirene + copper
Bug fix.
Description:
AIX defines different sizes of the signed and unsigned int_fast8_t.
Changed the code to check each type individually and do not verify
they must be the same size.
Platforms tested:
Tested in copper only since the change is trivial.
Misc. update:
tconfig.c:
Verified only the unsigned int types and did not detect inconsistencies
on signed type such as int_fast16_t.
Changed code to verify both signed and unsigned int types wherever
applicable. It also depends on signed and unsigned forms of an
int type must be of the same sizes.
testhdf5.c:
Shorten the configure test name to 'config'--easier to type.
Tested on LLNL Frost and Snow. Will test in Copper after commit.
tconfig.c:
Verified only the unsigned int types and did not detect inconsistencies
on signed type such as int_fast16_t.
Changed code to verify both signed and unsigned int types wherever
applicable. It also depends on signed and unsigned forms of an
int type must be of the same sizes.
testhdf5.c:
Shorten the configure test name to 'config'--easier to type.
Tested on LLNL Frost and Snow. Will test in Copper after commit.
Bug fix
Description:
Correct a couple of array bounds issues exposed by the PGI compiler
Platforms tested:
Linux 2.4 (verbena) w/PGI compilers
Too minor too requie h5committest
Code cleanup
Description:
Clean up formatting and some mis-casts, etc.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Linux 2.4 (verbena)
Solaris 2.7 (arabica)
Added tests for most of the types that were added to the data transform code.
Further tests will also be added shortly.
Platforms tested:
sol + eirene
Misc. update:
Purpose: change feature
Description: Back up support bitfield and time datatypes in H5Tget_native_type.Leave it to future support. Let it return "not supported" error message for
now.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Purpose:
Feature
Description:
Datatypes and groups now use H5FO "file object" code that was previously
only used by datasets. These objects will hold a file open if the file
is closed but they have not yet been closed. If these objects are unlinked
then relinked, they will not be destroyed. If they are opened twice (even
by two different names), both IDs will "see" changes made to the object
using the other ID.
When an object is opened using two different names (e.g., if a dataset was
opened under one name, then mounted and opened under its new name), calling
H5Iget_name() on a given hid_t will return the name used to open that hid_t,
not the current name of the object (this is a feature, and a change from the
previous behavior of datasets).
Solution:
Used H5FO code that was already in place for datasets. Broke H5D_t's, H5T_t's,
and H5G_t's into a "shared" struct and a private struct. The shared structs
(H5D_shared_t, etc.) hold the object's information and are used by all IDs
that point to a given object in the file. The private structs are pointed
to by the hid_t and contain the object's group entry information (including its
name) and a pointer to the shared struct for that object.
This changed the naming of structs throughout the library (e.g., datatype->size
is now datatype->shared->size). I added an updated H5Tinit.c to windows.zip.
Platforms tested:
Visual Studio 7, sleipnir, arabica, verbena
Misc. update:
Bug fix
Description:
Fix situation where deleting a chunked datasets with B-tree nodes that
weren't in the metadata cache would die with a core dump.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Linux 2.4 (heping)
Solaris 2.7 (arabica)
Bug fix.
Description:
the Data transformation code that failed when a data transform property
for simple expression is fixed. Turned the tests back on.
Platforms tested:
Tested in copper (pp) where the failure appeared. Also in eirene
as double check. No h5committest as the change is trivial.
Bug fix.
Description:
THe trivial transform property list creation triggered a code
error. Skip these two tests temporary. Will fix it later.
Platforms tested:
Copper where it exposed the failures.
Misc. update:
Code cleanup
Description:
Tweak recent "forward compatibility" changes to the H5E* API (which allowed
for the old H5E API functions to remain unchanged) by allowing for the error
stack callback function (H5E_auto_t) to also remain unchanged from the 1.6
branch. This required changing the H5E{get|set}_auto routines to have the
old style H5E_auto_t type (which didn't have a stack ID parameter) and the new
H5E{get|set}_auto_stack routines to have a newer "H5E_auto_stack_t" type (which
has a stack ID parameter). This should make the H5E API changes as forwardly
compatible as possible.
One side-affect of this change was that it was impossible to determine if
the current auto error callback was the old style (H5E_auto_t) or the new style
(H5E_auto_stack_t) of callback, so a new API function (H5Eauto_is_stack) was
adde to query this.
Platforms tested:
FreeBSD 4.10 (sleipnir)
IRIX64 6.5 (modi4)
h5committest
Description: One place was left out for update after the Error API was modified.
Changed it from H5Eset_auto to H5Eset_auto_stack.
Platforms tested: sleipnir(with threadsafe) - simple change.
new test for the native types test
Description:
on the Cray SV1 an INT type was wrongly converted to a SHORT type
by the get_native_integer function
Choose the type based on the precision; this is to support cases
like the Cray SV1, where the size of short is 8 but precision is 32
(e.g an INT (size 8, prec 64) would be converted to a SHORT
(size 8, prec 32) if the size was the deciding factor)
Solution:
Platforms tested:
linux
solaris
aix
Misc. update:
Description: A few items were left out when tried to restore the old Error API.
There are also a few minor bug fixes.
Platforms tested: arabica fuss h5committest.
Description: Restore 6 old error API functions back to the library to be backward
compatible with v1.6. They are H5Epush, H5Eprint, H5Ewalk, H5Eclear, H5Eset_auto,
H5Eget_auto. These functions do not have error stack as parameter.
Solution: Internally, these functions use default error stack.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Bug fix (sorta)
Description:
Change reading of "missing" chunks from datasets with undefined fill
values to not overwrite the application buffer with random garbage from
memory. Note that this is almost the same, since whatever garbage the
application had in those locations will still be there...
Platforms tested:
FreeBSD 4.10 (sleipnir)
IRIX64 6.5 (modi4)
h5committested
