This feature is still in progress; Shared Object Header Messages are not
complete as a feature and are not thoroughly tested. There are still
"TODO" comments in the code (comments with the word "JAMES" in them,
so as not to be confused with other TODO comments).
Hopefully this checkin will reduce the liklihood of conflicts as I finish
implementing this feature.
All current tests pass on juniper, copper (parallel), heping, kagiso, and mir.
Add new H5Lget_val_by_idx() routine & tests.
Also includes most of changes for H5Ldelete_by_idx() routine.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
AIX/32 5.? (copper)
Finished implementation of H5Lget_info_by_idx for all cases: old vs. new
group formats, compact vs. dense new link storage, increasing vs. decreasing
vs. native iteration order.
Also, refactor symbol table "foo by index" routines to be more generic
and share more code by using a single B-tree iteration callback which makes
callbacks to a specific "get <foo>" callback.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
AIX/32 5.? (copper)
Should disable linking against shared libraries in Fortran for compilers that
don't support shared libraries.
Should also fix problem when the wrong Fortran file extension was specified.
If these changes don't solve the Daily Test issues, I'll look at backing out
the autotool version change until I have time to fix them.
Tested on heping, kagiso, juniper.
Fix retrieving name for references to root group.
Also, move 'ref' test earlier in testing, right after 'getname' test
Tested on:
Linux/64 2.6 (chicago2)
Implement basic framework for H5Lget_info_by_idx and get it working for
creation order indices on compact groups.
Clean up code a bit.
Close resource link in user-defined link traversal.
Tested on:
Linux/32 2.6 (chicago)
Update H5Iget_name() tests to reflect that the name of the object for an
ID can be found in many more situations now.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Refactor generic property list initialization code to put property list
specific routines in property list modules, instead of scattered to the four
winds. Also, introduce property list class initialization objects, to make
adding new property list classes in the library easier.
Fix daily test failure by using H5Pget_elink_prefix() API routine instead
of looking at the "raw" generic property list information.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/C++ & FORTRAN
Linux/64 2.4 (mir) w/build-all & 1.6 compat
the linkval buffer, per Elena and Frank's suggestions while revising
the documentation. Added error checking using this size, as well as a
couple of tests.
Tested on juniper, kagiso, and sol.
Finish internal work necessary to track creation order in v2 B-tree when
group is in "dense" storage form.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add support for compress heap blocks in "managed" heaps.
Also, fix bug when the first direct block moves between having a parent
and not having one (and vice versa), which occurs when the heap moves between
having a direct root block and having an indirect root block.
Tested on:
FreeBSD 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/C++ & FORTRAN
Linux/64 2.4 (mir) w/build-all & 1.6 compat
Reduce memory usage for common cases of I/O pipeline filter memory usage.
Also, clean up some more code...
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Finish new version of the I/O pipeline message, which is much smaller than
the previous version. This version is used with the "use the latest version
of the format" flag.
Closed several memory leaks/overruns (found with valgrind).
Also, lots of compiler & formatting cleanups.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Fix several errors in the "latest version" of the object header format
changes and enable the new version when requested now.
Clean up several confusing or duplicated sections of code.
Tested on:
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
FreeBSD/32 4.11 (sleipnir)
through driver info functions in H5FD_direct_open, H5FD_direct_read, and H5FD_direct_write. The
external change is to add a new API function H5Pget_fapl_direct to query 3 control values, the
memory boundary, file system block size, and the maximal buffer size for copying data.
handle flush callbacks which can dirty other entries, and
resize and/or rename the target entry.
This feature is needed by the fractal heap code.
Also added associated test code.
H5Commit tested. Test failed on heping, but the
error appears to be a syntax error in an un-related
file.
Tests on copper & sol passed, along with tests on
phoenix.
Add 'initval' parameter to "lookup3" checksum routine (and implicitly to
the metadata checksum routine), to allow chaining several checksums together
easily (which isn't used by these modules, but will be used in my next checkin)
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
memory boundary, file block size, and maximal copy buffer size; second in H5FD_direct_write and
H5FD_direct_read, the library checks whether data buffer is aligned. If it is, then write and
read the data directly instead of making a copy buffer.
Bug fix (related to 544)
Description:
h5_get_file_size() was coded to return 0 if failed but
file size can be 0. Changed the failure return value to -1
which is allowed by the returned type of off_t which is a signed
type.
Also changed the checking code of the stat call to just == 0
since that is how it is defined.
Test:
Could test it in heping only. Both Sol and Copper failed to
compiled due to error in the Direct IO VFD code.
So new flush2 tests generates false failures for DLL version on windows.
Two cases of flush2 tests were skipped temporarily on windows.
The inserted windows macros should be removed when a better solution is found.
Tested on vs6.0 and vs 7.1 and Linux.
Update datasets and the layout, attributes and fill-value
object header messages to use the latest version
of the file format flag.
Tested on:
FreeBSD 4.11 (sleipnir)
Linux/64 2.4 (mir)
Linux/32 2.4 (heping)
option --enable-direct-vfd/--disable-direct-vfd to enable/disable Direct I/O support. The default
is enabled. There's a small test in test/vfd.c. Another way to test it is to set environment
variable HDF5_DRIVER to "direct" and run "make check" in the test/ directory. There'll be some
further improvement in the following checkin including allowing user to provide memory boundary
value, file block size, and copying buffer size.
Add "use the latest format" support for dataspace object header encode/
decode routines and clean up format a bit for the latest format (new to 1.8.x
releases)
Remove storing 'perm' parameter for array datatypes in memory and the file,
and add test to make certain that if any user applications are attempting to
store them, we get some reports back. (Should be unlikely, since the RefMan
says that the parameter is not implemented and is unsupported).
Carry those changes into the tests, etc.
Clean up a bunch more compiler warnings.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/FORTRAN & C++
Linux/64 2.4 (mir) w/enable-1.6-compat
Description: Add a new part to the flush test that checks to see what happens in case a file is flushed, and
then a new dataset is created and the program exits without flushing this subsequent dataset.
The test verifies that, at the very least, the data written out before the H5Fflush call is correct.
and quote its arguments. Also checks for the 'socket' library on
Solaris.
If this patch passes the Daily Tests and makes the user happy, I'll
port it back to the 1.6 branch.
Tested on mir and sol.
Finish 'use the latest version of the format' changes to the datatype
object header message. Compound and enumerated types will now be encoded
more efficiently by packing the field names & member offsets better.
Tested on:
Linux 2.6/32 (chicago)
Linux 2.6/64 (chicago2)
Add the "use the latest version of the format" support to datatype
messages. And a regression test to check that it's working.
Also, found that we have been over-allocating space for compound datatype
messages (when they have array datatypes or can use the latest version of the
format) and trimmed the size back.
Clean up datatype & dataspace encode/decode routines by having them
allocate & release "fake" file structures, which gets them out of needing to
"know" about the internals of file structures.
Other minor whitespace/formatting cleanups, etc.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Further minor modifications to the file format for tracking links in groups.
This is tentatively the "final" file format for groups.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Fix file handle destroy routine to not attempt to flush out partially
initialized file handles (when opening a file fails).
Tested on: (until they finished testing, this time... :-)
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/64 2.4 (mir) w/1.6 compat
Linux/32 2.4 (heping) w/FORTRAN & C++
Mac OSX/32 10.4.8 (amazon)
Add test to fractal heaps to exercise issues with opening the same heap
through two different file handles.
Fix issues with file handle contexts in metadata cache callbacks for heap
components.
Fix bug in file close handling where cached information was being
invalidated even when another file handle was open to the file.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/64 2.4 (mir) w/1.6 compat
Linux/32 2.4 (heping) w/FORTRAN & C++
Mac OSX/32 10.4.8 (amazon)
File format is not stable, don't keep files produced!
Description:
First stage of checkins modifying the format of groups to support creation
order. Implement "dense" storage for links in groups.
Try to clarify some of the symbols for the H5L API.
Add the H5Pset_latest_format() flag for FAPLs, to choose to use the newest
file format options (including "dense" link storage in groups)
Add the H5Pset_track_creation_order() flag for GCPLs, to enable creation
order tracking in groups (although no index on creation order yet).
Remove --enable-group-revision configure flag, as file format issues are
now handled in a backwardly/forwardly compatible way.
Clean up lots of compiler warnings and other minor formatting issues.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/FORTRAN & C++
Linux/64 2.4 (mir) w/enable-v1.6 compa
Mac OSX/32 10.4.8 (amazon)
AIX 5.3 (copper) w/parallel & FORTRAN
Description:
Add a second test case to the serial flush tests. This tests the case when the file is not flushed out to disk and verifies that it fails as expected.
Platforms:
Linux (heping)
Review, revise & checkin in Peter's latest round of object copy changes,
which add basic support for datasets & attributes with reference datatypes.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add 'loookup3' checksum routine and switch to using it for metadata
checksums - it's just as "strong" as the CRC32 and about 40% faster in general
(with some compiler optimizations, it's nearly as fast as the fletcher-32
algorithm).
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Some of the tests cannot be run on VMS since they try to open
the same file twice.
Solution:
Bypass the tests according to the H5_CANNOT_OPEN_TWICE variable setting.
Platforms tested:
VMS server and heping.
Add "op" routine to perform operation on heap object "in situ", to allow
for faster operations on dense links during B-tree traversal & lookup.
Refactor the "read" routine to use the internal version of the "op" routine,
to keep the code duplication as low as possible.
Tested on:
Mac OS X.4/PPC (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add a CRC algorithm to the library, initially for "small" (<256 byte)
metadata blocks.
Update checksum tests to verify it's working correctly.
Tested:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
(Will be testing on more platforms after checkin)
Improve density of the B-tree further. For greater depths of B-trees,
the gains are over 100%...
Also, don't split internal nodes with 3->4 splits, use a 1->2 split
instead, so that the density of the nodes around a split is maximized.
Tested:
Mac OS X/PPC 10.4 (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Split edge nodes in the tree with a 1->2 node split, instead of a 2->3 node
split, which creates a more dense tree when a pattern of record insertions
occurs (because it leaves behind full nodes instead of 2/3 full nodes).
Tested:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Linux/32 2.4 (heping)
Solaris/64 2.9 (shanti)
Thread safe error test fails due to the changes in the error stack.
Solution:
Updated the expected error stack.
Platforms tested:
heping (too minor, probably will fail on Tuesday anyway)
Refactor the file storage of "twig" nodes in the B-tree to allow them to
store more records, increasing the average density of the B-tree 30-40%.
Increase # of records in "insert lots" regression test to still create
B-tree of depth 4
Update h5debug to interpret difference of 'branch' and 'twig' internal
nodes in B-tree correctly.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Re-order the fheap & btree2 tests so that the btree2 test runs first,
because the fractal heaps use v2 B-trees for tracking huge objects.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
These errors should be investigated more thoroughly later. The underlying
problem in links.c seems to be that files opened multiple times don't share
the same H5F_shared_t struct. Perhaps identifying when this is the case
would be helpful?
Tested on mir.
Add support for checksumming fractal heaps. This is always enabled for
the heap header and indirect blocks (as they are "pure" metadata) and is
optional for direct blocks, since they may be used for "raw" data.
Also, rearrange direct block routines in H5HFcache.c to be in a more
sensible location in the file. (probably should have been a separate checkin,
since the diffs are mostly useless for this checkin... *sigh*)
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Since these examples need to follow filesystem paths, the Makefiles need
to create directories in the examples directory; added this to the
Makefile.am.
Tested on Windows, mir, juniper
Tweak the library's new faster fletcher32 algorithm to always produce the
same checksum as the previous fletcher32 code in the fletcher32 I/O pipeline
filter and switch the filter to use the library's version of the algorithm.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Too minor to require h5committest
Break out a bunch of the misc. routines that were in src/H5.c into more
specific modules.
Add optimized fletcher32 checksum routine, for checksumming metadata as
well as raw data.
Tested On:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Will test further after checkin...
Add a bunch more regression tests:
- Verify opening a fractal heap more than once works correctly
- Check that 0 & 1-sized object insertions are handled correctly
- Check that bad heap IDs are detected when attempting to read an
object
- Check that doubling tables with different parameter values are
working correctly
Add some more checks to the code to detect 0-sized object insertions and
bad heap IDs for reading.
Tested On:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
- Migrate "direct block location" routine from H5HFman.c to H5HFdblock.c,
which is a more appropriate location
- Optimize performance of heap code by taking advantage of pinned
indirect blocks and use them without putting a metadata cache
protect/unprotect pair around them.
- Other minor compiler warning cleanups and optimizations...
Tested On:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Object header locations (H5O_loc_t's) can now "hold open" a file and
decrement its open object count when they close. This means that
locations (H5O_loc_t's and H5G_loc_t's) should always be freed.
Added more thorough tests to ensure that external files are closed.
flag, and pin an entry as it is inserted.
The objective is to avoid some function call overhead in fheap.
Also added matching test code in test/cache.c & test/cache_common.c
(also testpar/t_cache.c checked in separately by accident)
h5commit tested
Several changes, all mooshed together:
- Add support for "tiny" objects - which can be stored in the heap
ID itself, instead of in the heap data blocks.
- Flesh out support for compressed direct blocks, but comment it
out until John's got some metadata cache changes in place to
support it.
- Add support for applying I/O pipeline filters to 'huge' objects
- Refactor 'huge' object code to store information for 'huge' objects
directly in the heap ID, when there are I/O pipeline filters
applied to the heap (and the heap ID is large enough to hold the
information)
- Update h5debug tool to correctly handle 'huge' & 'tiny' objects.
- Misc. other code cleanups, etc.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Allow the heap ID length to be chosen at heap creation time, to allow
for making heap IDs long enough to directly embed the file offset & length
of 'huge' objects in the heap ID (which allows them to be retrieved directly
from disk, instead of requiring them to be looked up in the B-tree that tracks
'huge' objects)
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 9 (shanti)
Clean up a couple of compiler warnings in the 'huge' object code in the
library.
Add a bunch of 'huge' object regression tests
Tested on:
FreeBSD 4.11 (sleipnir)
Linux/64 2.4 (mir)
Solaris/64 2.9 (shanti)
Move metadata cache address rename for fractal heap indirect block
immediately after reallocing the storage on disk - to prevent another
routine in the library from confusing the cache by allocating more storage
and attempting to insert it in metadata cache at the old block's location.
Gotta like those monte carlo/random insert & delete tests, which exposed
this... :-)
Tested:
Solaris 2.9 (shanti)
Not failing on other machines
Initial revision to add support for "huge" objects in the heap (which are
actually stored directly in the file, but are tracked with v2 B-tree that is
accessed through heap header).
Testing:
FreeBSD 4.11 (sleipnir)
Linux/64 2.4 (mir)
Linux/32 2.4 (heping)
Mac OS X.4 (amazon)
Update code formatting a little and refactor to add a better mechanism
for performing callbacks when removing records from the B-tree or deleting
entire B-tree.
Testing:
FreeBSD 4.11 (sleipnir)
Linux/64 2.4 (mir)
Linux/32 2.4 (heping)
Mac OS 10.4 (amazon)
Daily test for ttsafe failed on several machines due to the changed error stack.
Solution:
Updated the source code to use new error stack.
Platforms tested:
shanti
More tweaks on fractal heap statistics, to better separate information
about managed objects from information about "huge" objects.
Also, clean up some compiler warnings, etc. on 64-bit platforms & Windows.
Platforms tested:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago)
Refactor fields in heap header to clean out information from dropped
features and also group related fields together more nicely.
Add fields to heap header for storing information about "huge" objects.
Platforms tested:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago)
Refactor fractal heap IDs to include "flag byte" as part of the ID. This
byte will be used for the heap ID format version as well as flags to indicate
whether the heap object is a "tiny"/normal/"huge" object (with storage
mechanisms optimized for each type of object).
Platforms tested:
Linux/32 2.4 (chicago)
Too minor to require h5committest
Clean up compiler warnings/failures in test/links.c, especially when
--disable-production flag used with --enable-group-revision
Modify binary dumping in h5dump to clean up files created [a band-aid
solution to not actually creating the files in the srcdir, but better than
just leaving the files around... :-/ ]
Tested:
FreeBSD 4.11 (sleipnir) (w/ configure flags above)
Too minor to require h5committest
Users can create external links using H5L_create_external(). These links
point to an object in another HDF5 file. Users can alter the behavior of
external links or create new kinds of links by registering callbacks
using the H5L interface.
Added tests, tools support, etc.
Also a number of other, minor changes have been made (some restructuring of
the H5L interface, for instance).
Additional documentation and examples are forthcoming.
ttsafe test failed on the tts_error call. The error was due to the outdated
expected error stack; fixed.
Test was also moved to use new error APIs.
Platforms tested: Linux 2.4 (heping) and Solaris 2.9 (shanti)
"make check-vfd" will now run all tests in the test directory with different
file drivers (at least, all of those tests that use the testing framework's
FAPL). Tests that fail will be skipped.
This is not a perfect fix, but is better than nothing.
Along with this change, check-vfd should be added to the Daily Tests.
Fix the last scattered bunch of problems with the object deletion code,
which appears to be completely working now (for objects that are stored
within heap blocks - standalone objects aren't implemented yet).
Also, re-work the regression test to speed up some of the existing tests
and add in 100-200 more combinations of tests - overall, its probably even
slower than it was... :-/
Tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Mac OS X (amazon)
Checkpoint further work on object deletion code for fractal heaps. I
believe everything is finally working for this sub-task. More regression tests
soon, but I don't foresee any problems.
Machines:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Refactored object deletion code for fractal heap to fix scaling problems.
Passing all previous tests again now.
Tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Mac OS X (amazon)
1) Added trace file support to the metadata cache. This allows capture
of all metadata cache calls in trace files for purposes of optimization
and debuging.
2) Added an expunge entry function. This allows an entry to be deleteded
from the cache without writing it to disk even if it is dirty.
3) Added a function call to resize pinned entries.
4) Added code to deal with entries that are dirty on load. This is
needed in support of a bug fix which can alter object headers on
load to repair files.
5) Added progress reporting code to the "MDC API smoke check" test in
cache_api.c. To enable the progress reporting, set report_progress
to TRUE in mdc_api_call_smoke_check().
Tested with h5committest, and a parallel test on phoenix (dual athelon
linux box).
adjustment
Description:
These tests (fheap, btree2) are under development and are not used by
the library yet. Move them to the end so that their failure do not block
other current library code tests.
Platforms tested:
heping.
Feature
Description:
Revised Link APIs.
Solution:
New link APIs use H5L*
H5*create_expand do not create links to the objects created; this must
be done manually with H5Llink.
Added APIs to link an object given its ID (H5Llink), to copy links (H5Lcopy),
and changed creation APIs (H5Lcreate_hard and H5Lcreate_soft) and query
API (H5Lget_linkinfo instead of H5Gget_objinfo).
All old APIs are still supported in H5Gdeprec.c .
Platforms tested:
sol, mir, copper
Misc. update:
Forgot to update MANIFEST and release docs. Will do after checkin.
Code checkpoint
Description:
Revise fractal heap code to support deletions in up to 2nd level deep
indirect blocks. Needs some revisions to support deeper indirect blocks...
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
h5committest
Added/adjusted Test Express mode.
Description:
btree2:
Added TestExpress mode to skip some long tests (the 5th recursive and
the two random tests.)
fheap:
Changed the express mode messages to be more informative.
Un-excluded some tests from express mode.
Platforms tested:
Red Storm.
Code cleanup
Description:
Trim trailing whitespace in Makefile.am and C/C++ source files to make
diffing changes easier.
Platforms tested:
None necessary, whitespace only change
New feature
Description:
Check in Peter's code to add support for "shallow copy", "create
intermediate groups", "no attributes" and "expand soft links" support.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago) w/ & w/o group-revision enabled
h5committest
Description: Mac OS 10.4 on PowerPC chip has some errors to convert (unsigned)
long long to long double. When the bit sequences are 0x003ff..., 0x007fff...,
0x00ffff..., 0x01ffff..., 0x7fffff..., the converted values are twice as big
as they should be.
Solution: Detect the error in configure and disable the compiler conversion
and test case.
Platforms tested: h5committest, Mac OS 10.4, and fuss.
Bug fix
Description:
Since the new default Express level is 1, some fheap tests were being skipped.
Solution:
At least temporarily, fheap now skips these tests if it is run at express level 2
or greater.
Platforms tested:
mir
Configuration feature
Description:
'make install' now tests both static and shared libraries if both are installed.
Solution:
Previously, shared libraries were only tested when static libraries were not installed.
Also cleaned up line in commence.am that was including HL library in all Makefiles.
Platforms tested:
mir (Makefile change only)
testing feature
Description:
HDF5 Test Express change; now accepts values 0 to 3.
Solution:
As explained in the RFC and documented in the code, there are now four
levels of "express-ness", 0 (no limit) to 3 (1 min limit).
It is now up to test writers to take advantage of the various levels
to tune their tests' run times.
Platforms tested:
mir (minor change)
Description: Data conversion from long double to (unsigned) long long
returns some incorrect values on Mac OS 10.4 and SGI IRIX64 6.5. The
conversions start to go wrong when the long double is
20041683600089727.779961 (0x4351ccf385ebc8a0bfcc2a...). If adjusting
the values higher by assigning 0x...c8a0cf... or 0x...c8a0df..., the
converted values go wild.
Solution: Detect this error in configure.in and disable compiler
conversions from long double to (unsigned) long long for Mac OS 10.4
and IRIX64 6.5.
Platforms tested: h5committest, Mac OS 10.4, and IRIX64 6.5.
Code checkpoint
Description:
Add in more new features for the fractal heap code, mostly bringing in
more ability for deleting objects (which isn't completely working yet).
Also, checkpoint free space manager code, which is essentially complete
(although it needs some more work after the metadata cache has some additional
features)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
h5committest
Bug fix
Description:
Fix bug which could lead to files with incorrect count of messages in the
header for an object. The exact sequence of operations is complicated and
is described in the release notes.
Solution:
Mark merged null header messages as dirty.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Mac OS/X (amazon)
Bug fix - bugzilla #552
Description:
On Cray X1, trying to use : as an argument confused the system.
Solution:
Added a test in configure to see if : as an argument is bad.
If so, skipped the test.
Platforms tested:
mir, Cray X1 (change to configure only)
Feature
Description:
Sometimes the parallel prefix is given in the form of nfs:/mnt/pfs which
if given to a non-MPIO VFD like the default H5Fcreate, it would fail.
Added a new routine h5_rmprefix which would return the non-prefix component
part of the file name which would be okay for the default H5Fcreate and
such.
Platforms tested:
Tested in heping parallel.
Added Exress Test mode.
Description:
The test takes a long time to run. It tempted developers to skip the
h5committest.
Solution:
Added Express Test mode to allow tester an option to run it in express
mode (shorter time.) There are already encoded QAK macro which skips
many tests. Follow that pattern to skip tests when HDF5TestExpress is
set.
Platforms tested:
Tested in heping, both serial and parallel.
1) Check in potential fix to unreproduceable bug in t_cache observed on
Cobalt
2) Check in code supporting added pinned entry features in the metadata
cache.
Description:
1) Elena encountered a bug in t_cache when running on Cobalt. On
investigation I found a potential race condition in t_cache that
could explain her observation.
2) Quincey requested additions to the metadata cache allowing renaming
of pinned entryies, and marking of pinned or protected entries as
dirty.
Solution:
1) Modified t_cache.c to prevent the potential race condition. Elena
was unable to reproduce the bug on Cobalt, so we don't know if my
fix actually addressed the problem.
2) Added code supporting the requested functions. The changes were
relatively trivial, but required substantial new test code.
Platforms tested:
h5committest, serial test on heping, parallel test on phoenix.
Misc. update:
Code checkpoint
Description:
Update fractal heap code to allow objects to be deleted
Fix bugs in fractal heap object insertion
Improve free space manager code to allow sections to be merged and
shrink the container
Another try at making the Windows compilers happy...
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OS X.4/PPC (amazon)
Linux 2.4 (chicago & heping)
Solaris 2.9 (shanti)
AIX 5.? (copper) w/parallel
Bug fix.
Description:
The ${TR}, though avoid the error, is inconvenient. It is needed
because configure.in use the char range in the style of 'a-z'.
The other style of '[a-z]' is more commonly accepted by all tr
but autoconf tends to strip away [], making the syntax rather
clumsy.
Solution:
Learned from autoconf that it avoids the use of character range
by just spell all the letters out. Changed our tr commands to
use those variables defined by autoconf. Also removed the
definition of ${TR} since it will cause inconsistancy when
autoconf also use plain 'tr' in its generated code.
The Makefile.in are changed because the elimination of ${TR} from
configure triggered its removal from all Makefil.in. That is okay
because ${TR} is not used at all in Makefile.
Platforms tested:
h5committested. (sol failed to connected). also tested in shanti
using both /usr/ucb/tr and /bin/tr (the bad one before.)
Misc. update:
Updated both INSTALL and RELEASE files.
Code checkpoint
Description:
Checkpoint fractal heap improvements, as well as move the free space
manager code that it's using into a separate package.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4/64 (mir) w/C++ & FORTRAN
Linux 2.4/32 (heping)
Solaris 2.9 (shanti)
AIX 5.? (copper) w/FORTRAN & parallel
Description: Function prototype H5E_walk_t and structure H5E_error_t wasn't backward
compatible.
Solution: Make them compatible with v1.6 and provide new definitions of H5E_walk_stack_t
and H5E_error_stack_t.
Platforms tested: fuss and h5committest.
Purpose: Maintenance
Description: dtypes.c compilation failed with gcc 4.1 due to the
missing prototype of the opaque_funcs function.
Failure was reported by a user who tried to build
hdf5-1.8.0-alpha2.
I reproduced the failure on sleipnir with gcc41
compiler.
Solution: Added prototype definition.
Platforms tested: sleipnir with gcc41, heping and copper
Misc. update:
Bug fix
Description:
A datatype wasn't being closed in th5s.c
Solution:
This minor oversight caused an error with my Object Header code.
The datatype is now closed in the test.
Platforms tested:
windows, mir, copper
Bug fix
Description:
Fix two test bugs:
1) Unitialized variable bug in test of H5C_get_entry_status()
(in test/cache.c)
2) Incorrect initialization of constant in t_cache when running
with MPE. (in testpar/t_cache.c)
Solution:
1) Don't test uninitialize variable.
2) Correct initialzation of constant in MPE case.
Platforms tested:
Tested on Heping and Copper.
Misc. update:
Code checkpoint
Description:
More progress on fractal heap, can now re-open an existing heap, although
the free space algorithm still needs work.
Also, use the new "pinned entry" metadata cache code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
Solaris 9 (shanti)
Linux 2.4/64 (mir)
Code cleanup/feature twist
Description:
Adjust recent H5AC routines to take H5F_t pointers instead of H5C_t
pointers, to match the rest of the H5AC routines.
This change propagated into a few of the tests, which also had some
compiler warnings cleaned up...
Platforms tested:
FreeBSD 4.11 (sleipnir) w/parallel
Linux 2.4/64 (mir) w/C++ & FORTRAN
Add pinned entry capability to cache.
Description:
For frequently accessed cache entries, the protect/unprotect overhead
is sometimes a bottleneck.
Solution:
Allow entries to be pinned in the cache. Pinned entries can't be
evicted, but can be flushed or modified.
Platforms tested:
h5committested -- minus one small typo in test/cache.c whose fix was
tested on copper and heping only.
Misc. update:
Several bug fixes
Description:
Added config file for Cray X1 (previous file was misnamed)
Simplified some code in hyperslab.c test that seemed to have been
confusing compiler on Cray X1.
Fixed typo in config/commence.am
Cleaned up hl/src/Makefile.am
Solution:
All four fixes should be straightforward. The failure on Cray was
very difficult to debug, but involved arithmetic errors.
This change seems to fix it.
Platforms tested:
heping, copper, sol, some Cray X1 (more testing when system comes back up)
Description:
The file size test in C++ library failed on Copper because the
value returned by h5_get_file_size was intepreted incorrectly
due to different interger sizes.
Solution:
H5private.h: Added check to use stat64 and off64_t where appropriate.
h5test.c and h5test.h: used h5_stat_size_t in place of off_t.
tattr.cpp: used h5_stat_size_t in place of off_t.
Platforms tested:
Linux 2.4 (heping)
AIX 5.1 (copper)
SunOS 5.8 64-bit (sol) - still on going
Bug fix.
Description:
It did not handle the case when version sub-release string is not
empty.
Solution:
Fixed the sed substitution to remove the sub-release string too.
Platforms tested:
Heping.
Code checkpoint
Description:
Add more regression tests for odd insertion patterns
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
Linux 2.4/64 (mir)
Solaris 2.9 (shanti)
Code checkpoint
Description:
Add in more fractal heap support for odd allocation patterns (entries that
are too large for various block sizes, etc.)
Broke out fractal heap code into more source code modules.
Refactored fractal heap test to make it more understandable and
maintainable for the long term.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
Linux 2.4/64 (mir)
Solaris 2.9 (shanti)
Anti-feature
Description:
Revert changes to H5G_stat_t struct, to make it compatible with the 1.6.x
branch again. The information that was added to the H5G_stat_t struct will
be reported through other API routines.
Platforms tested:
FreeBSD 4.11 (sleipnir) w/C++
Linux 2.4/64 (mir) w/C++ & Fortran
Solaris 2.9 (shanti)
new feature
Description:
1) separated the HL library into "public" and "private" header files, with the same caracteristics as the basic library
2) added the public headers to hdf5.h (with a conditional include macro, defined in configure.in)
3) added the path to HL in all Makefile.am 's , because of the inclusion in hdf5.h
Solution:
Platforms tested:
linux 32, 64
AIX
solaris
with fortran and c++
(one packet table example fails)
Misc. update:
"Hide" file format changes (for now)
Description:
Add ifdef's (controlled by the --enable-group-revision configure flag)
to disable group revision changes to the file format, in order to allow alpha
release to go ahead without releasing an unsupported version into the wild.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 32-bit (heping)
Linux 2.4 64-bit (mir)
Solaris 2.9 (shanti)
feature
Description:
Added the GetTestExpress and SetTestExpress to support the TestExpress mode
according to $HDF5TextExpress. For now, it is just defined or not, the
actual value does not matter.
Platforms tested:
heping(serial), mir (PP).
Bug fix.
Description:
getenv_all would fail if one calls it without calling MPI_initialized first.
This may happen if one uses PHDF5 library to build a serial nature code
and does not make any MPI calls.
Solution:
Added code to detect if MPI has been initialized. If not, use the
original getenv.
Platforms tested:
heping (serial) and mir (parallel).
Bug fix
Description:
Correct the handling of the group creation property list when copying
objects with the "create intermediate groups" flag.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest (once I pay attention to the compiler
warnings... :-)
Code checkpoint
Description:
Update fractal heap code to allow skipping direct blocks (that are too
small to hold object) when the root indirect block is created.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
New/expanded features
Description:
Check in Peter's changed for the object copy code:
- Allow/fix copying datasets using named variable-length datatypes
- Start adding framework for property list to control how object
copying occurs.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code checkpoint
Description:
Checkpoint recent additions to fractal heap code, which allow for better
support of objects that don't fit into current direct block, requiring
skipped direct blocks to be tracked in free section list.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago & heping)
Linux 2.4 (mir) w/64-bit & FORTRAN & C++
Mac OSX (amazon)
Code checkpoint
Description:
Check in fractal heap code to add basic support for skipping direct blocks
when an object is too large to fit in one.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Code checkpoint
Description:
Checkpoint fractal heap code at a stable point in development. It appears
that adding objects to heaps and reading them back out is now working
properly (as long as there are no gaps in the heap and no standalone objects
are created).
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Linux 2.4 (mir) w/64-bit & FORTRAN & C++
Solaris 9 (shanti) w/64-bit
Description: cross_read.c is a new program to test reading data created
on an OpenVMS, a little-endian, and a big-endian machine. The three
data files are vms_data.h5, be_data.h5, and le_data.h5. To generate
these data files, run gen_cross.c on these machines and change the file
names.
Platforms tested: fuss and h5committest
Misc. update: MANIFEST
Bug fix
Description:
Previous checkin did a bad thing; 'make clean' failed in example directories.
Solution:
Fixed commence.am so that examples no longer break, and fixed a mistake
in conclude.am.
Platforms tested:
heping (minor makefile change)
Misc. update:
Bug fix
Description:
make check-clean didn't clean results of example tests
Solution:
Fixed Makefiles so that check-clean recurses into example directories.
Also a little Makefile cleanup.
Platforms tested:
mir, modi4, heping, copper
Description: OpenVMS doesn't support denormalized floating-point values.
Disable the denormalized and special values for floating to floating-point
tests.
Platforms tested: OpenVMS - the only concerned machine.
Misc. update:
Code cleanup
Description:
Clean up some compiler warnings (esp. those flagged on Windows builds)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code checkpoint
Description:
Update fractal heap code to insert & read heaps up to 64MB in size
(with my current configuration paramaters) and add initial support for
iteratively walking down nested indirect blocks.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Solaris 9 (shanti)
Linux 2.4 (mir) w/64-bit
Code update
Description:
Add basics of routine for reading information back out of a fractal heap.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Checkpoint code
Description:
Rework code to support increasing size of root indirect block, in order
to add [at least] a second row of direct blocks.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Solaris 9 (shanti) w/64-bit
Linux 2.4 (mir)
Code checkpoint
Description:
Checkpoint fractal heap changes - mostly internal changes to better
handle more direct blocks from root indirect block.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago)
Mac OS X (amazon)
Solution: Support VAX float and double types with new byte order. There're some
details not yet included,
1. the alignment detection for VAX order in H5detect.c.
2. support for special values in library conversion functions.
3. the infinity for VAX in H5T_init_inf.
4. support for VAX types in printing hexadecimal format.
Platforms tested: h5committest and fuss.
Code checkpoint
Description:
The fractal heap code is reasonably stable and is passing all its
current tests, so checkpoint everything with CVS.
Also, add "statistics" module for v2 B-trees code, which is only a
stub right now.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
Linux 2.4 (chicago)
New features
Description:
Add "find node greater than or equal to key" and "remove first" operations
to skip lists.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Description: By default on VMS system file cannot be opened twice; also
VMS has file versioning. This brakes some of our tests.
Solution:
Tests are commented out with H5_CANNOT_OPEN_TWICE and H5_HAVE_FILE_VERSIONS
that are defined in the vms/h5pubconf.h file only. This change cannot affect
any other platform.
Platforms tested: heping, VMS server
Misc. update:
Bug fix/Feature
Description:
Modify the library to use rand_r, when available, instead of srand. If
rand_r is not available, it will try to use srandom/random, and then finally
fall back to srand/rand.
Solution:
A couple places in the library use the srand() function to seed the random
number generator. This can cause problems on certain platforms and could theoretically
cause problems for users who expect a certain sequence of random numbers following their
own call to srand().
Most platforms have an implementation of rand_r, which is identical to rand, except that
it allows for explicit storage of the seed value. The configure script will now check
for the existence of rand_r to facilitate its use in the library.
Two new functions are added, to replace the macros HDsrand/HDrand. These functions are HDrand and
HDsrand. HDrand will call rand_r, if available, or random(), if available, and fall back to rand() if neither
of those is available. HDsrand will store the seed value locally, if rand_r is available. Otherwise, it will call
srandom if available, and fall back to srand otherwise.
Platforms tested:
heping, mir. Really need to test on Red Storm, since that platform motivated this fix, but the machine
is currently not available.
Misc. update:
Bug fix (sorta)
Description:
Disable the fractal heap tests until I can look into them more thoroughly.
(they are failing on 64-bit machines currently)
Platforms tested:
None - trivial change.
New feature & code cleanup
Description:
Update fractal heap to be able to insert objects into a direct block
hanging off the header.
Extract "octal dump" code into separate routine so that both the local
heaps & fractal heaps can use it.
Other code cleanups & support to get this far.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Incrementtal checkin
Description:
Revise & update v2 B-tree code to separate the internal package-specific
routines from the library-callable "private" routines.
Similar updates for the fractal heap code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
New feature
Description:
Check in initial "fractal heap" code, for supporting the group redesign.
Also, remove some remnants of the segmented heap/block tracker/B+tree code
which I didn't find during the last pass.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX (amazon)
Linux 2.4
Maintenance
Description:
Remove half-built (half-baked? :-) implementation of B+tree, block tracker
and segmented heaps. (In preparation for starting the new heap coding)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
Add files that I forgot in my last checkin.
Description:
cache_api.c cache_common.c cache_common.h are new files that result
from breaking the old cache test into two tests.
Solution:
See above.
Platforms tested:
h5committested before the last checkin. Will run another
h5committest shortly, but it shouldn't be necessary as there
are no new changes.
Misc. update:
Bug fix/additional tests
Description:
Add additional tests for holding open an object of a given type (dataset,
group, etc) while incorrectly trying to open that object with a different
type of API routine (i.e. using the group open call on a dataset, etc.)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix & new feature
Description:
Support variable-length datatypes in compact data storage and chunked
data storage, along with attributes.
Bug fix on the H5T_vlen_set_loc to allow for changing the file on a
variable-length datatype on disk.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Can't h5committest right now, due to missing cache files.
1) Add parallel test for metadata cache
2) Split serial test for metadata cache into two parts
3) Fix bug in which cache was flushed needlessly when the
cache wasn't full.
4) Performance improvements
5) Update API for parallel cache coherency bug fix.
Description:
See above.
Solution:
See above.
Platforms tested:
h5committest
Misc. update:
Code cleanup
Description:
Check in some of the code cleanups from working on the external link
support. (This doesn't include any of the external link features)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OSX.4 (amazon)
Linux 2.4
Bug fix.
Description:
getenv_all, like some implementations of getenv, keeps an internal buffer
for the getenv values. Application must make a copy of the content of the
returned pointer before calling getenv_all again IF it wants to use the
first content later. The previous code violated this rule and caused
error.
Solution:
Did a small revamp of the h5_fixname to fix the above problem by moving
the getenv_all for HDF5_NOCLEANUP eariler to avoid it clashing the
call of getenv_all for HDF5_PARAPREFIX. No a good permenant fix. Need
to really recode this routine.
Platforms tested:
h5committested.
Bug fix.
Description:
getenv_all will be called even when it is NOT an MPI application.
Solution:
Moved the use of getenv_all("HDF5_NOCLEANUP") to fix_name so that it will
be invoked only if a parallel driver is used.
Platforms tested:
hand tested in copper.
bug fix 504
Description:
testpar/t_mpi would hang if $HDF5_NOCLEANUP is set. E.g.,
% env HDF5_NOCLEANUP=yes mpirun -np 3 ./t_mpi
This happened because the environment variables are not exported to all
mpi processes by the mpirun command. So, some attempted to do cleanup
while others don't and some hang waiting for others to act.
Solution:
Instead individual program checking getenv, they all just called h5_cleanup
no matter. h5_cleanup now uses getenv_all to check the $HDF5_NOCLEANUP if
it is in parallel mode.
Platforms tested:
h5committested, tested pp in heping too.
Feature
Description:
Added character encoding and attribute creation property lists.
Solution:
Attributes' character encoding is set via the ACPL.
The default is ASCII, with UTF-8 being the other option currently.
Platforms tested:
heping, mir, sleipnir, copper
Bug fix
Description:
Retrieving an object's name could fail (in various ways) under certain
circumstances (mostly having to do with mounted files).
Solution:
Re-write & simplify "get object name" code to fix error in a better way
than adding yet another hack to the previous pile of hacks... :-)
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Extra test
Description:
Add test to make certain that trying to call H5Fopen() on a non-existant
file fails.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code cleanup
Description:
Fix a bunch of warnings flagged by Windows compilers.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
New feature
Description:
Add in a combination of Peter's & my code to support copying
variable-length data from one file to another, although currently only
supported with contiguous data storage.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Bug fix
Description:
Fortran type generation was broken in two ways. Fixed both.
Solution:
Firstly, there were a couple of path problems. Fixed a typo and
specified the full path of a file.
Secondly, the dependencies weren't right when building with HDF5-specific
commands (make lib, make check-s, etc.). Tweaked dependencies
to fix the problem.
Platforms tested:
mir, modi4, sleipnir
Configure feature
Description:
Added 'make trace' target.
Solution:
Added tracing to 1.7. This was done automatically in 1.6, but left out
of 1.7 until now (oops!).
Tracing in 1.7 only happens manually, when the user types 'make trace.'
Tracing automatically requires more framework than it's worth.
I also fixed a couple of tracing bugs and ran trace.
Platforms tested:
mir, sleipnir, modi4
Misc. update:
Bug fix
Description:
Before this checkin, 'gmake check-s' would fail if there was a file in
the current directory named 'check-s'.
This is fixed under gmake (not sure how to fix for other makes).
Solution:
check, progs, install, etc. are what gmake calls "phony" targets,
which means that no file should be created. These targets can be
specified by a line of the form
.PHONY: check progs install ...
Automake adds this line for targets it knows about, but HDF5 has a
lot of custom rules. This checkin adds a .PHONY line for those rules.
I believe that only gmake recognizes the .PHONY line (at least, pmake
doesn't seem to), but a partial solution is better than none.
This error should occur very rarely anyway (the user has to manually
create files with names like 'build-check-s' or '_test').
Platforms tested:
mir, sleipnir, modi4
More tests
Description:
Add more tests for proper behavior of groups with different group
creation property settings.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Solaris 2.8
Bug fix
Description:
Fixed bug in hl/src Makefile that was deleting y.tab.c.
Running bin/reconfigure changed a few other files as well (somebody
probably updated Makefiles.am but forgot to bin/reconfigure).
Solution:
Automake in its wisdom adds the line "rm *.tab.c" to every distclean
in every directory. I assume that it assumes that such files are
generated during the build. Supplied an empty distclean-compile
target to remove this line from the Makefiles.in.
Platforms tested:
mir, sleipnir, shanti
New feature
Description:
Check in baseline for compact group revisions, which radically revises the
source code for managing groups and object headers.
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
This initiates the "unstable" phase of the 1.7.x branch, leading up
to the 1.8.0 release. Please test this code, but do _NOT_ keep files created
with it - the format will change again before the release and you will not
be able to read your old files!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
Solution:
There's too many changes to really describe them all, but some of them
include:
- Stop abusing the H5G_entry_t structure and split it into two separate
structures for non-symbol table node use within the library: H5O_loc_t
for object locations in a file and H5G_name_t to store the path to
an opened object. H5G_entry_t is now only used for storing symbol
table entries on disk.
- Retire H5G_namei() in favor of a more general mechanism for traversing
group paths and issuing callbacks on objects located. This gets us out
of the business of hacking H5G_namei() for new features, generally.
- Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t
- Lots more...
Platforms tested:
h5committested and maybe another dozen configurations.... :-)
New feature
Description:
Add in baseline "object copy" code from Peter [in the form of a new API
routine: H5Gcopy()]. There's still some work to do (like handling variable-
length datatypes and possibly support for references) and it hasn't been tested
on mounted files yet, but the core functionality is there and working
correctly.
I've also got a set of patches to update the 1.6 branch with tweaks to
keep the branches mostly in sync, but Elena will kill me if I import them
before the 1.6.5 release is out... :-)
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committested
Bug fix/feature
Description:
Added support for -shlib in h5fc and h5c++.
Made check-install use -shlib when only shared libraries have been installed.
Solution:
h5fc and h5c++ didn't recognize -shlib. Stole code from h5cc to link against
shared libraries.
When static libraries are disabled, the examples Makefiles will automatically
use the -shlib option to link against shared libraries. Thus,
--disable-static and make check-install should work together.
Platforms tested:
heping(disable-static, enable-static, fortran, c++), modi4 (disable-static, fortran, c++, parallel, enable-static)
Bug fix
Description:
Failed parallel tests now cause make to exit with an error.
Solution:
Edited config/conclude.am to throw an error if parallel test programs fail.
Platforms tested:
heping, modi4
Added additional targets to build-all
Description:
Surveyed files to find out which ones are not built.
Added test/space_overflow and perform/benchpar to programs built with
--enable-build-all.
Discovered that tools/misc/pdb2hdf5 and tools/misc/h5import were
obsolete and deleted them.
Platforms tested:
mir, modi4, sol
Misc. update:
Update of release file and removal of --enable-pdb2hdf5 to follow.
Bug fix
Description:
test/gen_new_fill.c used some old APIs that didn't compile with the current
library. Brought it up-to-date.
Some other files (test/gen_old_*) are designed to be compiled only
with older versions of the library. Removed rules to try to build
them in the current Makefiles.
Platforms tested:
mir
Bug fix
Description:
Changed configure.in to use an environment variable TR to set the path
to the tr utility.
Solution:
There are two kind of tr on Solaris with slightly different syntax.
HDF5's configure relies on the "standard" tr. Traditionally, HDF5ers
have needed to make sure that the "right" tr was found before the
wrong one in their path; now they can use an environment variable.
Platforms tested:
mir, shanti, sol
Misc. update:
Forgot to update release notes. Off to do that now.
Bug fix
Description:
Changed the name of a file from the last fix.
createnoenc.c is not gen_noencoder.c
Solution:
This is just a more standard name for the file.
Platforms tested:
tested in previous check-in; forgot to commit to cvs.
Makefile bug fix
Description:
Previously, automake didn't output rules to build perform/mpi-perf or
the test/gen_* programs.
Now these can be built by typing 'make mpi-perf' (or 'make foo') or by
configuring with --enable-build-all.
Solution:
Automake doesn't like having rules for programs it doesn't build. Tricked
it by having these programs built "sometimes"--whenever the user enables
--build-all. This should be used mostly for testing and to ensure that
these helper programs compile.
***IMPORTANT***
These programs do *not* currently compile. When --enable-build-all is used
(not the default), gen_new_fill fails because it uses an old API. This is
an existing "bug" that has simply been exposed by this checkin.
Platforms tested:
sleipnir, modi4, sol
Misc. update:
Avoid bug for family file driver backward compatibility with 1.6.
Description:
Because there is no srcdir on windows, string pathname should be
initialized as NULL before using strcat.
Solution:
Changed the code as above.
Platforms tested:
too minor to test.
Misc. update:
Bug fix
Description:
Check in Kent's fixes to hyperslab rebuilding code that should correctly
detect situation when different sub-hyperslabs exist in a higher dimension.
Solution:
Remember previous sub-hyperslab information and compare that with newly
generated sub-hyperslab to make certain they are the same.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Fix "double root open" bug and apply same fix to datasets & named datatypes.
Also fix bug in symbol table entry lookup code that would "transfer" an
object from one file handle to another file handle when the two file handles
were opened on the same actual file.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
bug fix.
Description:
When a parallel test script test fails, make would continue because the
way it was setup inside a for loop. Fixed it by issuing an exit 1 inside
the loop.
There was also a typo error in the newer command comparision that it
must be $${chkname} in order to be valid. Also, the test script itself
was not checked in the newer lists. All fixed.
Platforms tested:
h5committested and also hand tested in heping pp mode.
Repair synchronization bug in the metadata cache in PHDF5
Also repair numerous other bugs that surfaced in testing the
bug fix.
Description:
While operations modifying metadata must be collective, we allow
independant reads. This allows metadata caches on different processes
to adjust to different sizes, and to place the entries on their dirty
lists in different orders. Since only process 0 actually writes
metadata to disk (all other processes thought they did, but the writes
were discarded on the theory that they had to be collective), this made
it possible for another process to modify metadata, flush it, and then
read it back in in its original form (pre-modification) form. The
possibilities for file corruption should be obvious.
Solution:
Make the policy that only process 0 can write to file explicit, and
visible to the metadata caches. Thus only process 0 may flush dirty
entries -- all other caches must retain dirty entries until they are
informed by process 0 that the entries are clean.
Synchronization is handled by counting the bytes of dirty cache entries
created, and then synching up between the caches whenever the sum
exceeds an (eventually user specified) limit. Dirty metadata creation
is consistent across all processes because all operations modifying
metadata must be collective.
This change uncovered may bugs which are repaired in this checkin.
It also required modification of H5HL and H5O to allocate file space
on insertion rather than on flush from cache.
Platforms tested:
H5committest, heping(parallel & serial)
Misc. update:
new features
Description:
add support for compiling the library and testphdf5 in Windows
Solution:
Platforms tested:
Linux
AIX
Solaris
Windows VC6
Misc. update:
Feature.
Description:
Added ALARM_ON, ALARM_OFF feature to terminate tests that run more than
the default time limit. So far, the feature is applied in the standard test
frame.
Platforms tested:
heping PP.
New feature.
Description:
Added the time command to the make check target to report time usage
of the execute of each test and test scripts. This gives us some idea
how long each test takes and some vague idea it is compute bound or
not.
powerpc-ibm-aix5.x:
Change $RUNPARALLEL default setting to allow it being invoked by the
time command.
Platforms tested:
h5committested.
Description:
On VMS dsets test failed screaming about corrupted stack.
Debugging didn't show any obvious problems. Careful examination
of compilation warning messages (something like "function is used as
data in dsets") made me to search for duplicated names.
It turned out that two variables H5Z_SET_LOCAL and H5Z_CAN_APPLY
were confused with H5Z_set_local and H5Z_can_apply functions from
H5Z.c file.
Solution:
Changed variable names to H5Z_SET_LOCAL_TEST and H5Z_CAN_APPLY_TEST.
Platforms tested: Alpha VMS, heping; minor change
Misc. update:
Feature
Description:
Added H5_CFLAGS, etc. to 1.7 branch.
Now compilation flags can be put in H5_*FLAGS and they'll be used when
building hdf5 but not in h5cc.
Platforms tested:
mir, sleipnir, modi4
Misc. update:
Bug fix
Description:
Disabled C++ shared libraries for Sun Workshop compiler.
Solution:
This bug only seems to happen when using the -xarch=v9 flag to compile in
64-bit mode, but disabling shared libraries entirely for this compiler is
an easier fix (I don't know how to detect 64 bit mode from the command line).
The framework for disabling shared libraries for other C++ compilers is
in place.
Platforms tested:
sol, mir, sleipnir, modi4
Bug fix/feature
Description:
Disabled shared libraries for a number of Fortran compilers that don't
support them.
This allows other compilers to support shared Fortran libraries.
Solution:
Added a conditional, SHARED_FORTRAN_CONDITIONAL, which is true if
Fortran supports shared libraries. It is set in configure.in.
Platforms tested:
mir, sleipnir, colonelk, heping
Description: H5T_register() replaces any existing data conversion path
with a new path.
Solution: Added a parameter to H5T_register() and H5T_path_find() to
signal the library whether the new conversion path is from API function
H5Tregister() or from private function like H5T_init_interface(). If it
is from H5Tregister(), replace existing path. If it's from H5T_init_interface()
and the library is trying to register default hard conversion functions,
don't replace existing path because the path is registered by the library.
For example, the library registered H5T_conv_int_float() and is trying to
register H5T_conv_long_float(). The size of long is equal to int. There's
no need to replace the path H5T_conv_int_float().
Platforms tested: h5committest and fuss.
Description: The API function H5Tis_hard() which was recently checked
in has been renamed to H5Tcompiler_conv() to be more descriptive. No
other changes.
Platforms tested: fuss - simple change.
Misc. update: config/lt_vers.am to update libtool version.
Bug fix
Description:
Building with --disable-static seems to have been broken on several platforms.
Fixed this.
Configure will now disable shared libraries automatically when using pgf90
or when building on Cygwin.
Solution:
To avoid errors when using shared libraries with pgf90 (which I had not
realized were compiler-specific), I had manually set convenience libraries
to use the -static linking flag. Apparently, this is not necessary, and
caused these libraries to be created as empty archives when --disable-static
was used.
Also, some libraries were including other libraries, which was not
necessary.
I also fixed code in configure.in to correctly detect whether shared
or static libraries are enabled, and moved code that disables shared libraries
to before libtool is created (rather than editing libtool after the fact).
Despite repeated warnings that only one shared library can be linked at a
time, I have yet to have shared libraries break the linking of tests on
any system. We'll see if the Daily Tests turn up anything.
Platforms tested:
mir (fortran, C++), sleipnir (C++), modi4 (fortran, C++, parallel),
sol (fortran, C++), cygwin
Description: Brought back changes from SX-6.
Solution: Tests will not compile. Fixed.
Platforms tested: SX-6, heping and mir with new g95
Misc. update:
Description: Quincey added the condition branch (else if (*((ST*)S) != (ST)((DT)(*((ST*)S)))))
in the definition of the macro H5T_CONV_Fx_CORE. It handles a special situation
when the source is "float" and assigned the value of "INT_MAX". Compilers do roundup
making this value "INT_MAX+1". This branch is to check that situation and return
exception for some compilers, mainly GCC.
The branch if (*((ST*)S) > (DT)(D_MAX) || (S_PREC < D_PREC && *((ST*)S) ==
(DT)(D_MAX))) is for some compilers like Sun, HP, IBM, and SGI where under
the same situation the "int" doesn't overflow.
A test was added to dt_arith.c to check 2 conversions, from float to int where the
float is INT_MAX, and from double to signed char where the double is SCHAR_MAX.
Platforms tested: h5committest and fuss.
Code cleanup/reorganization
Description:
Merge back some more changes extracted from the "compact group" set.
This bunch cleans up and prepares the H5G_* routines for eventual import of
new features.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Mac OS X.4
Code cleanup
Description:
Merge back changes from "compact group" work that improve the
infrastructure of the library and may impact others. In this round of
merging, that includes:
- Move datatype allocation into single internal routine, instead of
duplicated code that was spread out in a dozen or so places.
- Clean up guts of object header routines (H5O_*) to allow for some of
the fancieroperations that need to be performed on groups, along with
some general improvements.
- Added a new error code
- Some minor cleanups in other code....
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Mac OS X
Bug fix
Description:
Incorrectly terminated the received string in getenv_all
Solution:
Tasks would incorrectly terminat the environment string they received from the
root task in getenv_all.
They did the equivalent of:
env[strlen(str) + 1] = '\0'.
This resulted in a single "garbage" character inserted
at the end of the string. This wasn't noticed until now because it seems that on most
platforms this garbage character was a 0 anyway.
This has been corrected to;
env[strlen(str)] = '\0'.
Platforms tested:
Red Storm (where the bug was discovered)
heping (pp)
Misc. update:
Description: Put in a new API funciton, H5Tis_hard. It checks whether
the conversion function from a native type to another native type is a
compiler (hard) conversion. Also checked a test in test/dt_arith.c.
Platforms tested: h5committest and fuss.
Bug fix & code cleanup
Description:
Address most of datatype conversion exception handling bug that Ed
Hartnett reported. (He's reported a different problem now, but we're closer
at least).
Also, condense exception handling #ifdef's into one location instead of
spread out in so many places.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code cleanup
Description:
Changed some scale+offset filter symbols from H5_SO_... -> H5Z_SO_...
to indicate that they are in the H5Z package.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
update dsets.c for scale-offset filter.
Description:
H5Psetscaleoffset has changed parameters.
Solution:
Platforms tested:
sol 2.7, Linux 2.4, AIX 6.4(32-bit and 64-bit)
Misc. update:
tidy up.
Description:
MPE option created *.clog file whenever it is executed in MPI.
The cleanup of *.clog files were done in individual Makefile.in.
Often, it is forgotten.
Solution:
Moved the cleaning of *.clog files to CHECK_CLEANFILES in commence.am
so that it is applied whenever check-clean is called.
Platforms tested:
heping pp using MPE.
Bug fixes
Description:
This checkin fixes an occasional error on kelgia on sol during distclean.
It also causes test scripts to depend properly on the programs they're
supposed to be testing.
Solution:
The kelgia bug was due to some files being cleaned by automake and manually.
Removed the manual cleaning in src/Makefile.am.
Test script dependencies now need to be specified manually, since the
makefile can't guess what they test from their name. Currently all test
scripts in a given directory have a single list of dependencies--this was
easy and seems to be sufficient.
These dependencies are listed in the SCRIPT_DEPEND variable in the Makefile.am.
Platforms tested:
heping, mir, modi4, sol
Misc. update:
