Correct a minor error in file free space allocation which was affecting
the 'multi' VFD and preventing some tests from fully working with it.
Wholesale revisitation of all the places where tests were disabled
with various VFDs and remove or correct all these so that _only_ the tests
which _really_ can't work with particular VFDs are skipped during a
'make check-vfd' test.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Fix various problems with a the core & sec2 VFDs.
Improve the h5_get_file_size() routine to handle files created with
VFDs that use multiple files.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
Update extensible array code with function to open an existing earray,
add more tests and avoid running the test when core/split/family/multi VFDs
are used.
Clean up fractal heap test code a bit and expand some of the tests a
little bit also.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Initial checkin of extensible array data structure prototype code and
regression tests.
Initial definitions for revised FUNC_ENTER/LEAVE and error reporting
macros, which are being vetted in the extensible array code.
Minor warning and formatting cleanups in other sections of code.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
- Add hash value for skip list string types, to reduce # of string
comparisons.
- Fixed bug with metadata/small data block aggregator adding size == 0
block into file free space list.
- Refactored metadata/small data block aggregator code into single set of
common routines.
- Changed block aggregator code to be smarter about releasing space in the
'other' block when the 'other' block has aggregated enough data.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Refactor H5HF_size() size statistics code for fractal heap to conform to
how the rest of the fractal heap routines work.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.10 (amazon)
Correct problem with fractal heap's free space size usage gathering
routine, which was "poisoning the cache" by loading an incorrectly initialized
piece of metadata from the file.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 6.2 (amazon)
Rename new error handling API routines from H5E<foo>_stack() to
H5E<foo>2().
Tested on:
Mac OS X/32 10.4.9 (amazon)
FreeBSD/32 6.2 (duty)
FreeBSD/64 6.2 (liberty)
Add version # and flags to external link format (as fields in a single
byte), in order to accomodate future changes/expansions.
Tested on:
Mac OS X/32 10.4.9 (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
FreeBSD/32 6.2 (duty)
FreeBSD/64 6.2 (liberty)
Allow "big" attributes to push attribute storage into "dense" form
immediately, to accomodate storing the attribute. (This is only allowed
in the "latest" version of the format).
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
Add [quite] limited ability to update (ie. write) data for objects in
fractal heap. Limited to just updating objects in managed heap blocks (i.e.
not 'tiny' or 'huge' objects) and must be updated with data of the same length
as the object in the heap. Updating objects in compressed heaps does work
though [as long as the data isn't 'tiny' or 'huge'].
Needed for changing the data value or the name of an attribute that is
stored in dense or shared storage.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Migrate more object header routines to use the H5O_msg_ prefix and put
them into the src/H5Omessage.c code module.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
AIX/32 5.? (copper)
Add support for compress heap blocks in "managed" heaps.
Also, fix bug when the first direct block moves between having a parent
and not having one (and vice versa), which occurs when the heap moves between
having a direct root block and having an indirect root block.
Tested on:
FreeBSD 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/C++ & FORTRAN
Linux/64 2.4 (mir) w/build-all & 1.6 compat
Bug fix (related to 544)
Description:
h5_get_file_size() was coded to return 0 if failed but
file size can be 0. Changed the failure return value to -1
which is allowed by the returned type of off_t which is a signed
type.
Also changed the checking code of the stat call to just == 0
since that is how it is defined.
Test:
Could test it in heping only. Both Sol and Copper failed to
compiled due to error in the Direct IO VFD code.
Add test to fractal heaps to exercise issues with opening the same heap
through two different file handles.
Fix issues with file handle contexts in metadata cache callbacks for heap
components.
Fix bug in file close handling where cached information was being
invalidated even when another file handle was open to the file.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/64 2.4 (mir) w/1.6 compat
Linux/32 2.4 (heping) w/FORTRAN & C++
Mac OSX/32 10.4.8 (amazon)
Add "op" routine to perform operation on heap object "in situ", to allow
for faster operations on dense links during B-tree traversal & lookup.
Refactor the "read" routine to use the internal version of the "op" routine,
to keep the code duplication as low as possible.
Tested on:
Mac OS X.4/PPC (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
These errors should be investigated more thoroughly later. The underlying
problem in links.c seems to be that files opened multiple times don't share
the same H5F_shared_t struct. Perhaps identifying when this is the case
would be helpful?
Tested on mir.
Add support for checksumming fractal heaps. This is always enabled for
the heap header and indirect blocks (as they are "pure" metadata) and is
optional for direct blocks, since they may be used for "raw" data.
Also, rearrange direct block routines in H5HFcache.c to be in a more
sensible location in the file. (probably should have been a separate checkin,
since the diffs are mostly useless for this checkin... *sigh*)
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add a bunch more regression tests:
- Verify opening a fractal heap more than once works correctly
- Check that 0 & 1-sized object insertions are handled correctly
- Check that bad heap IDs are detected when attempting to read an
object
- Check that doubling tables with different parameter values are
working correctly
Add some more checks to the code to detect 0-sized object insertions and
bad heap IDs for reading.
Tested On:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
- Migrate "direct block location" routine from H5HFman.c to H5HFdblock.c,
which is a more appropriate location
- Optimize performance of heap code by taking advantage of pinned
indirect blocks and use them without putting a metadata cache
protect/unprotect pair around them.
- Other minor compiler warning cleanups and optimizations...
Tested On:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Several changes, all mooshed together:
- Add support for "tiny" objects - which can be stored in the heap
ID itself, instead of in the heap data blocks.
- Flesh out support for compressed direct blocks, but comment it
out until John's got some metadata cache changes in place to
support it.
- Add support for applying I/O pipeline filters to 'huge' objects
- Refactor 'huge' object code to store information for 'huge' objects
directly in the heap ID, when there are I/O pipeline filters
applied to the heap (and the heap ID is large enough to hold the
information)
- Update h5debug tool to correctly handle 'huge' & 'tiny' objects.
- Misc. other code cleanups, etc.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Allow the heap ID length to be chosen at heap creation time, to allow
for making heap IDs long enough to directly embed the file offset & length
of 'huge' objects in the heap ID (which allows them to be retrieved directly
from disk, instead of requiring them to be looked up in the B-tree that tracks
'huge' objects)
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 9 (shanti)
Clean up a couple of compiler warnings in the 'huge' object code in the
library.
Add a bunch of 'huge' object regression tests
Tested on:
FreeBSD 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Move metadata cache address rename for fractal heap indirect block
immediately after reallocing the storage on disk - to prevent another
routine in the library from confusing the cache by allocating more storage
and attempting to insert it in metadata cache at the old block's location.
Gotta like those monte carlo/random insert & delete tests, which exposed
this... :-)
Tested:
Solaris 2.9 (shanti)
Not failing on other machines
Initial revision to add support for "huge" objects in the heap (which are
actually stored directly in the file, but are tracked with v2 B-tree that is
accessed through heap header).
Testing:
FreeBSD 4.11 (sleipnir)
Linux/64 2.4 (mir)
Linux/32 2.4 (heping)
Mac OS X.4 (amazon)
More tweaks on fractal heap statistics, to better separate information
about managed objects from information about "huge" objects.
Also, clean up some compiler warnings, etc. on 64-bit platforms & Windows.
Platforms tested:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago)
Refactor fields in heap header to clean out information from dropped
features and also group related fields together more nicely.
Add fields to heap header for storing information about "huge" objects.
Platforms tested:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago)
Refactor fractal heap IDs to include "flag byte" as part of the ID. This
byte will be used for the heap ID format version as well as flags to indicate
whether the heap object is a "tiny"/normal/"huge" object (with storage
mechanisms optimized for each type of object).
Platforms tested:
Linux/32 2.4 (chicago)
Too minor to require h5committest
"make check-vfd" will now run all tests in the test directory with different
file drivers (at least, all of those tests that use the testing framework's
FAPL). Tests that fail will be skipped.
This is not a perfect fix, but is better than nothing.
Along with this change, check-vfd should be added to the Daily Tests.
Fix the last scattered bunch of problems with the object deletion code,
which appears to be completely working now (for objects that are stored
within heap blocks - standalone objects aren't implemented yet).
Also, re-work the regression test to speed up some of the existing tests
and add in 100-200 more combinations of tests - overall, its probably even
slower than it was... :-/
Tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Mac OS X (amazon)
Checkpoint further work on object deletion code for fractal heaps. I
believe everything is finally working for this sub-task. More regression tests
soon, but I don't foresee any problems.
Machines:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Refactored object deletion code for fractal heap to fix scaling problems.
Passing all previous tests again now.
Tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Mac OS X (amazon)
Code checkpoint
Description:
Revise fractal heap code to support deletions in up to 2nd level deep
indirect blocks. Needs some revisions to support deeper indirect blocks...
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
h5committest
Added/adjusted Test Express mode.
Description:
btree2:
Added TestExpress mode to skip some long tests (the 5th recursive and
the two random tests.)
fheap:
Changed the express mode messages to be more informative.
Un-excluded some tests from express mode.
Platforms tested:
Red Storm.
Code cleanup
Description:
Trim trailing whitespace in Makefile.am and C/C++ source files to make
diffing changes easier.
Platforms tested:
None necessary, whitespace only change
Bug fix
Description:
Since the new default Express level is 1, some fheap tests were being skipped.
Solution:
At least temporarily, fheap now skips these tests if it is run at express level 2
or greater.
Platforms tested:
mir
