Bug fix (related to 544)
Description:
h5_get_file_size() was coded to return 0 if failed but
file size can be 0. Changed the failure return value to -1
which is allowed by the returned type of off_t which is a signed
type.
Also changed the checking code of the stat call to just == 0
since that is how it is defined.
Test:
Could test it in heping only. Both Sol and Copper failed to
compiled due to error in the Direct IO VFD code.
So new flush2 tests generates false failures for DLL version on windows.
Two cases of flush2 tests were skipped temporarily on windows.
The inserted windows macros should be removed when a better solution is found.
Tested on vs6.0 and vs 7.1 and Linux.
Update datasets and the layout, attributes and fill-value
object header messages to use the latest version
of the file format flag.
Tested on:
FreeBSD 4.11 (sleipnir)
Linux/64 2.4 (mir)
Linux/32 2.4 (heping)
option --enable-direct-vfd/--disable-direct-vfd to enable/disable Direct I/O support. The default
is enabled. There's a small test in test/vfd.c. Another way to test it is to set environment
variable HDF5_DRIVER to "direct" and run "make check" in the test/ directory. There'll be some
further improvement in the following checkin including allowing user to provide memory boundary
value, file block size, and copying buffer size.
Add "use the latest format" support for dataspace object header encode/
decode routines and clean up format a bit for the latest format (new to 1.8.x
releases)
Remove storing 'perm' parameter for array datatypes in memory and the file,
and add test to make certain that if any user applications are attempting to
store them, we get some reports back. (Should be unlikely, since the RefMan
says that the parameter is not implemented and is unsupported).
Carry those changes into the tests, etc.
Clean up a bunch more compiler warnings.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/FORTRAN & C++
Linux/64 2.4 (mir) w/enable-1.6-compat
Description: Add a new part to the flush test that checks to see what happens in case a file is flushed, and
then a new dataset is created and the program exits without flushing this subsequent dataset.
The test verifies that, at the very least, the data written out before the H5Fflush call is correct.
and quote its arguments. Also checks for the 'socket' library on
Solaris.
If this patch passes the Daily Tests and makes the user happy, I'll
port it back to the 1.6 branch.
Tested on mir and sol.
Finish 'use the latest version of the format' changes to the datatype
object header message. Compound and enumerated types will now be encoded
more efficiently by packing the field names & member offsets better.
Tested on:
Linux 2.6/32 (chicago)
Linux 2.6/64 (chicago2)
Add the "use the latest version of the format" support to datatype
messages. And a regression test to check that it's working.
Also, found that we have been over-allocating space for compound datatype
messages (when they have array datatypes or can use the latest version of the
format) and trimmed the size back.
Clean up datatype & dataspace encode/decode routines by having them
allocate & release "fake" file structures, which gets them out of needing to
"know" about the internals of file structures.
Other minor whitespace/formatting cleanups, etc.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Further minor modifications to the file format for tracking links in groups.
This is tentatively the "final" file format for groups.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Fix file handle destroy routine to not attempt to flush out partially
initialized file handles (when opening a file fails).
Tested on: (until they finished testing, this time... :-)
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/64 2.4 (mir) w/1.6 compat
Linux/32 2.4 (heping) w/FORTRAN & C++
Mac OSX/32 10.4.8 (amazon)
Add test to fractal heaps to exercise issues with opening the same heap
through two different file handles.
Fix issues with file handle contexts in metadata cache callbacks for heap
components.
Fix bug in file close handling where cached information was being
invalidated even when another file handle was open to the file.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/64 2.4 (mir) w/1.6 compat
Linux/32 2.4 (heping) w/FORTRAN & C++
Mac OSX/32 10.4.8 (amazon)
File format is not stable, don't keep files produced!
Description:
First stage of checkins modifying the format of groups to support creation
order. Implement "dense" storage for links in groups.
Try to clarify some of the symbols for the H5L API.
Add the H5Pset_latest_format() flag for FAPLs, to choose to use the newest
file format options (including "dense" link storage in groups)
Add the H5Pset_track_creation_order() flag for GCPLs, to enable creation
order tracking in groups (although no index on creation order yet).
Remove --enable-group-revision configure flag, as file format issues are
now handled in a backwardly/forwardly compatible way.
Clean up lots of compiler warnings and other minor formatting issues.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/FORTRAN & C++
Linux/64 2.4 (mir) w/enable-v1.6 compa
Mac OSX/32 10.4.8 (amazon)
AIX 5.3 (copper) w/parallel & FORTRAN
Description:
Add a second test case to the serial flush tests. This tests the case when the file is not flushed out to disk and verifies that it fails as expected.
Platforms:
Linux (heping)
Review, revise & checkin in Peter's latest round of object copy changes,
which add basic support for datasets & attributes with reference datatypes.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add 'loookup3' checksum routine and switch to using it for metadata
checksums - it's just as "strong" as the CRC32 and about 40% faster in general
(with some compiler optimizations, it's nearly as fast as the fletcher-32
algorithm).
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Some of the tests cannot be run on VMS since they try to open
the same file twice.
Solution:
Bypass the tests according to the H5_CANNOT_OPEN_TWICE variable setting.
Platforms tested:
VMS server and heping.
Add "op" routine to perform operation on heap object "in situ", to allow
for faster operations on dense links during B-tree traversal & lookup.
Refactor the "read" routine to use the internal version of the "op" routine,
to keep the code duplication as low as possible.
Tested on:
Mac OS X.4/PPC (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add a CRC algorithm to the library, initially for "small" (<256 byte)
metadata blocks.
Update checksum tests to verify it's working correctly.
Tested:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
(Will be testing on more platforms after checkin)
Improve density of the B-tree further. For greater depths of B-trees,
the gains are over 100%...
Also, don't split internal nodes with 3->4 splits, use a 1->2 split
instead, so that the density of the nodes around a split is maximized.
Tested:
Mac OS X/PPC 10.4 (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Split edge nodes in the tree with a 1->2 node split, instead of a 2->3 node
split, which creates a more dense tree when a pattern of record insertions
occurs (because it leaves behind full nodes instead of 2/3 full nodes).
Tested:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Linux/32 2.4 (heping)
Solaris/64 2.9 (shanti)
Thread safe error test fails due to the changes in the error stack.
Solution:
Updated the expected error stack.
Platforms tested:
heping (too minor, probably will fail on Tuesday anyway)
Refactor the file storage of "twig" nodes in the B-tree to allow them to
store more records, increasing the average density of the B-tree 30-40%.
Increase # of records in "insert lots" regression test to still create
B-tree of depth 4
Update h5debug to interpret difference of 'branch' and 'twig' internal
nodes in B-tree correctly.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Re-order the fheap & btree2 tests so that the btree2 test runs first,
because the fractal heaps use v2 B-trees for tracking huge objects.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
These errors should be investigated more thoroughly later. The underlying
problem in links.c seems to be that files opened multiple times don't share
the same H5F_shared_t struct. Perhaps identifying when this is the case
would be helpful?
Tested on mir.
Add support for checksumming fractal heaps. This is always enabled for
the heap header and indirect blocks (as they are "pure" metadata) and is
optional for direct blocks, since they may be used for "raw" data.
Also, rearrange direct block routines in H5HFcache.c to be in a more
sensible location in the file. (probably should have been a separate checkin,
since the diffs are mostly useless for this checkin... *sigh*)
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Since these examples need to follow filesystem paths, the Makefiles need
to create directories in the examples directory; added this to the
Makefile.am.
Tested on Windows, mir, juniper
Tweak the library's new faster fletcher32 algorithm to always produce the
same checksum as the previous fletcher32 code in the fletcher32 I/O pipeline
filter and switch the filter to use the library's version of the algorithm.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Too minor to require h5committest
Break out a bunch of the misc. routines that were in src/H5.c into more
specific modules.
Add optimized fletcher32 checksum routine, for checksumming metadata as
well as raw data.
Tested On:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Will test further after checkin...
Add a bunch more regression tests:
- Verify opening a fractal heap more than once works correctly
- Check that 0 & 1-sized object insertions are handled correctly
- Check that bad heap IDs are detected when attempting to read an
object
- Check that doubling tables with different parameter values are
working correctly
Add some more checks to the code to detect 0-sized object insertions and
bad heap IDs for reading.
Tested On:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
- Migrate "direct block location" routine from H5HFman.c to H5HFdblock.c,
which is a more appropriate location
- Optimize performance of heap code by taking advantage of pinned
indirect blocks and use them without putting a metadata cache
protect/unprotect pair around them.
- Other minor compiler warning cleanups and optimizations...
Tested On:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Object header locations (H5O_loc_t's) can now "hold open" a file and
decrement its open object count when they close. This means that
locations (H5O_loc_t's and H5G_loc_t's) should always be freed.
Added more thorough tests to ensure that external files are closed.
flag, and pin an entry as it is inserted.
The objective is to avoid some function call overhead in fheap.
Also added matching test code in test/cache.c & test/cache_common.c
(also testpar/t_cache.c checked in separately by accident)
h5commit tested
Several changes, all mooshed together:
- Add support for "tiny" objects - which can be stored in the heap
ID itself, instead of in the heap data blocks.
- Flesh out support for compressed direct blocks, but comment it
out until John's got some metadata cache changes in place to
support it.
- Add support for applying I/O pipeline filters to 'huge' objects
- Refactor 'huge' object code to store information for 'huge' objects
directly in the heap ID, when there are I/O pipeline filters
applied to the heap (and the heap ID is large enough to hold the
information)
- Update h5debug tool to correctly handle 'huge' & 'tiny' objects.
- Misc. other code cleanups, etc.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
