Further minor modifications to the file format for tracking links in groups.
This is tentatively the "final" file format for groups.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
File format is not stable, don't keep files produced!
Description:
First stage of checkins modifying the format of groups to support creation
order. Implement "dense" storage for links in groups.
Try to clarify some of the symbols for the H5L API.
Add the H5Pset_latest_format() flag for FAPLs, to choose to use the newest
file format options (including "dense" link storage in groups)
Add the H5Pset_track_creation_order() flag for GCPLs, to enable creation
order tracking in groups (although no index on creation order yet).
Remove --enable-group-revision configure flag, as file format issues are
now handled in a backwardly/forwardly compatible way.
Clean up lots of compiler warnings and other minor formatting issues.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/FORTRAN & C++
Linux/64 2.4 (mir) w/enable-v1.6 compa
Mac OSX/32 10.4.8 (amazon)
AIX 5.3 (copper) w/parallel & FORTRAN
revised binary flags, added a new file to the test generator program to
be used in the binary tests
usage is now
-o F, --output=F Output raw data into file F
-b F, --binary=F Binary output, of form F (into file -o F).
Recommended usage is with --dataset=P
Form F of binary output is: MEMORY for memory type,
FILE for the disk file type, LE or BE for pre-existing
little or big endian types
example
./h5dump -d integer -b MEMORY -o out.bin tbinary.h5
Review, revise & checkin in Peter's latest round of object copy changes,
which add basic support for datasets & attributes with reference datatypes.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Improve density of the B-tree further. For greater depths of B-trees,
the gains are over 100%...
Also, don't split internal nodes with 3->4 splits, use a 1->2 split
instead, so that the density of the nodes around a split is maximized.
Tested:
Mac OS X/PPC 10.4 (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Code cleanup.
Description:
Removed argc and argv from the function arguments of h5tools_get_fapl() and
h5tools_fopen(). They were used to call MPI_Init() which was no longer
needed.
Tested:
heping (serial and parallel).
Refactor the file storage of "twig" nodes in the B-tree to allow them to
store more records, increasing the average density of the B-tree 30-40%.
Increase # of records in "insert lots" regression test to still create
B-tree of depth 4
Update h5debug to interpret difference of 'branch' and 'twig' internal
nodes in B-tree correctly.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Since these examples need to follow filesystem paths, the Makefiles need
to create directories in the examples directory; added this to the
Makefile.am.
Tested on Windows, mir, juniper
Several changes, all mooshed together:
- Add support for "tiny" objects - which can be stored in the heap
ID itself, instead of in the heap data blocks.
- Flesh out support for compressed direct blocks, but comment it
out until John's got some metadata cache changes in place to
support it.
- Add support for applying I/O pipeline filters to 'huge' objects
- Refactor 'huge' object code to store information for 'huge' objects
directly in the heap ID, when there are I/O pipeline filters
applied to the heap (and the heap ID is large enough to hold the
information)
- Update h5debug tool to correctly handle 'huge' & 'tiny' objects.
- Misc. other code cleanups, etc.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Clean up compiler warnings/failures in test/links.c, especially when
--disable-production flag used with --enable-group-revision
Modify binary dumping in h5dump to clean up files created [a band-aid
solution to not actually creating the files in the srcdir, but better than
just leaving the files around... :-/ ]
Tested:
FreeBSD 4.11 (sleipnir) (w/ configure flags above)
Too minor to require h5committest
Bug fixes.
Description:
There were MPI_init and MPI_finalize calls in the code of h5tools_fopen in
parallel mode. But if a non-MPI tool is invoke to open a non-existing
file, it tries to open the non-existing file with different VFD and
eventaully came to try with the MPIO or the MPIPOSIX vfd, then it would
try to do MPI_Init which would fail in the MPI environment if the a.out
was not launched by MPI properly.
Solution:
MPI_Init and MPI_Finalize in general should be called by the MPI application,
not called by a library subroutine as in the manner that was done here.
Removed the MPI_init and MPI_Finalize calls. Used MPI_Initialized to
verify if this has been launched as an MPI application in the proper
manner before attempting to use the MPIO or the MPIPOSIX VFD to open
the file.
Tested:
In tg-ncsa parallel, where it had failed explicitly and also in Heping,
using both serial and parallel mode.
Users can create external links using H5L_create_external(). These links
point to an object in another HDF5 file. Users can alter the behavior of
external links or create new kinds of links by registering callbacks
using the H5L interface.
Added tests, tools support, etc.
Also a number of other, minor changes have been made (some restructuring of
the H5L interface, for instance).
Additional documentation and examples are forthcoming.
Refactored free space manager to use metadata cache for serialized free
space sections. This speeds up the fractal heap test considerably...
Tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4/32 (chicago)
Linux 2.4/64 (mir)
Mac OS X (amazon)
the $srcdir properly. It is not right to chdir into testfiles and write
files there because in real srcdir mode, one should not changes things
in the srcdir area which could be shared by multiple builds simultanteously.
Solution: added the proper $srcdir components to the source file name.
Also clean up the indentation by cb.
Tested: only by hand in heping.
"make check-vfd" will now run all tests in the test directory with different
file drivers (at least, all of those tests that use the testing framework's
FAPL). Tests that fail will be skipped.
This is not a perfect fix, but is better than nothing.
Along with this change, check-vfd should be added to the Daily Tests.
1. changed the -F flag option names to "BE and "LE" for big and little endian
2. added a more verbose usage message for these options
3. add a new test
4. add a make clean instruction to *.bin
bug fix
calling h5tools_get_fapl running the mpio driver caused H5FD_pl_copy to
fail silently in some conditions. a MPI call was made before MPI_Init.
solution: corrected the MPI call to be made after MPI_Init and added error return
conditions to H5FD_pl_copy and h5tools_get_fapl
tested on copper parallel, mir, shanti
it uses the API function H5Gcopy to copy groups and datasets from file
to file, read from a command line
note: the test script is not finished yet
tested on heping, copper, shanti
Feature
Description:
Revised Link APIs.
Solution:
New link APIs use H5L*
H5*create_expand do not create links to the objects created; this must
be done manually with H5Llink.
Added APIs to link an object given its ID (H5Llink), to copy links (H5Lcopy),
and changed creation APIs (H5Lcreate_hard and H5Lcreate_soft) and query
API (H5Lget_linkinfo instead of H5Gget_objinfo).
All old APIs are still supported in H5Gdeprec.c .
Platforms tested:
sol, mir, copper
Misc. update:
Forgot to update MANIFEST and release docs. Will do after checkin.
