Add first pass of "dense" attribute storage to objects. Lots of parts of
this are stubbed out, but all the tests are passing and I'll work on the corner
cases soon.
Eliminated several unused parameters from object header message callback
routines.
Other, miscellaneous code cleanups, etc. (and probably some things I've
forgotten about... :-)
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
AIX/32 5.? (copper)
Finish moving object header message routines into their own source code
module, along with renaming them to have "H5O_msg_" prefix...
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
AIX/32 5.? (copper)
Migrate more object header routines to use the H5O_msg_ prefix and put
them into the src/H5Omessage.c code module.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
AIX/32 5.? (copper)
Fix problem with copying objects when the dataset to copy has an "older"
version of the layout message and needs to compute the size of the dataset's
storage after the dataspace information is available.
Minor random cleanups also...
Tested on:
FreeBSD/32 4.11 (sleipnir)
Too minor/weird to test w/h5committest...
Make the memory size of the max. compact & min. dense attribute storage
variables align with the property list size, instead of the size on disk, to
better handle big-endian machines.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
AIX/32 5.? (copper)
Add fields for tracking attributes on objects to object header prefix,
when using the latest version of the format.
Reduce size used to store type of header message from 2 bytes to 1. (If we
have >256 message types, we probably have a different problem... :-)
Also, add "birth" time for object (i.e. creation time) [named to align
with similar fields in file systems]
Correct bug in H5Gget_objinfo() - retrieve the ctime field from object info,
instead of the mtime field.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Reduce the size of some of the group information to more reasonable bounds.
(16-bit values instead of 32-bit values).
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
When using the latest version of the file format, move the "modification
time" information into the object header prefix, which is more efficient.
Also add "access time" and "change time" (for metadata) fields, all of which
take about the same space as the previous modification time header message.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
General cleanups, in preparation for adding some attribute-tracking fields
to the object header prefix.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Finish removing library's internal code that uses H5G_get_objinfo() and
retarget it at either getting the link information or the object information,
as appropriate. (Still need to add user-level tests for H5Oget_info(), but
since several internal components of the library depend on the internal version,
it appears to be working correctly).
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/322.4 (heping)
Linux/64 2.4 (mir)
AIX/32 5.? (copper)
Mac OS X/32 10.4.8 (amazon)
Refactor internal code that retrieves the object's type to use the new
H5O_type_t (which only definitions for object types, instead of links as well)
instead of the older H5G_obj_t (which included links, etc.)
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add src/H5Oalloc.c file forgotten in previous object header checkin
Break out more object header routines into separate file (debugging routines
this time).
Fix "//" style comment in recent ISOHM source changes.
Fix 'size_t' vs. 'unsigned' problem (visible in 64-bit testing) in recent
ISOHM test changes.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Finish migrating "object copy" supporting routines into src/H5Ocopy.
Clean up a few minor errors _en passant_
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
This feature is still in progress; Shared Object Header Messages are not
complete as a feature and are not thoroughly tested. There are still
"TODO" comments in the code (comments with the word "JAMES" in them,
so as not to be confused with other TODO comments).
Hopefully this checkin will reduce the liklihood of conflicts as I finish
implementing this feature.
All current tests pass on juniper, copper (parallel), heping, kagiso, and mir.
Fix several errors in the "latest version" of the object header format
changes and enable the new version when requested now.
Clean up several confusing or duplicated sections of code.
Tested on:
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
FreeBSD/32 4.11 (sleipnir)
More fixes to make the I/O & memory operations more efficient when loading
& flushing object header chunks.
Add in concept of a chunk having a "gap" at the end - unused space that is
not large enough to hold a full null message that describes it.
Clean up formatting a bit.
Latest version of object header format not enabled yet, still working on a
few bugs...
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Overhaul usage of object header chunks to reduce I/O operations and
memory allocations. The object header prefix is now stored in the first
object header chunk's "image" in memory.
Also, lots of formatting cleanups.
Taught h5debug tool about new object header format (which isn't enabled
just yet).
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Clean up some object header code in advance of more major changes in the
near future.
Fix small initialization error in attribute message name buffer copy.
Tweak down the default # of messages for new object headers.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/FORTRAN & C++
Mac OS/32 10.4.8 (amazon)
Tweak down some of the static array definitions for new & re-loaded object
header data structures, to reduce memory use in common cases.
Tested on:
Linux/32 2.6 (chicago)
File format is not stable, don't keep files produced!
Description:
First stage of checkins modifying the format of groups to support creation
order. Implement "dense" storage for links in groups.
Try to clarify some of the symbols for the H5L API.
Add the H5Pset_latest_format() flag for FAPLs, to choose to use the newest
file format options (including "dense" link storage in groups)
Add the H5Pset_track_creation_order() flag for GCPLs, to enable creation
order tracking in groups (although no index on creation order yet).
Remove --enable-group-revision configure flag, as file format issues are
now handled in a backwardly/forwardly compatible way.
Clean up lots of compiler warnings and other minor formatting issues.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/FORTRAN & C++
Linux/64 2.4 (mir) w/enable-v1.6 compa
Mac OSX/32 10.4.8 (amazon)
AIX 5.3 (copper) w/parallel & FORTRAN
Code cleanup
Description:
Trim trailing whitespace in Makefile.am and C/C++ source files to make
diffing changes easier.
Platforms tested:
None necessary, whitespace only change
New feature
Description:
Check in Peter's code to add support for "shallow copy", "create
intermediate groups", "no attributes" and "expand soft links" support.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (chicago) w/ & w/o group-revision enabled
h5committest
New/expanded features
Description:
Check in Peter's changed for the object copy code:
- Allow/fix copying datasets using named variable-length datatypes
- Start adding framework for property list to control how object
copying occurs.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix & new feature
Description:
Support variable-length datatypes in compact data storage and chunked
data storage, along with attributes.
Bug fix on the H5T_vlen_set_loc to allow for changing the file on a
variable-length datatype on disk.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Can't h5committest right now, due to missing cache files.
New feature
Description:
Add in a combination of Peter's & my code to support copying
variable-length data from one file to another, although currently only
supported with contiguous data storage.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
New feature
Description:
Check in baseline for compact group revisions, which radically revises the
source code for managing groups and object headers.
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
This initiates the "unstable" phase of the 1.7.x branch, leading up
to the 1.8.0 release. Please test this code, but do _NOT_ keep files created
with it - the format will change again before the release and you will not
be able to read your old files!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
Solution:
There's too many changes to really describe them all, but some of them
include:
- Stop abusing the H5G_entry_t structure and split it into two separate
structures for non-symbol table node use within the library: H5O_loc_t
for object locations in a file and H5G_name_t to store the path to
an opened object. H5G_entry_t is now only used for storing symbol
table entries on disk.
- Retire H5G_namei() in favor of a more general mechanism for traversing
group paths and issuing callbacks on objects located. This gets us out
of the business of hacking H5G_namei() for new features, generally.
- Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t
- Lots more...
Platforms tested:
h5committested and maybe another dozen configurations.... :-)
New feature
Description:
Add in baseline "object copy" code from Peter [in the form of a new API
routine: H5Gcopy()]. There's still some work to do (like handling variable-
length datatypes and possibly support for references) and it hasn't been tested
on mounted files yet, but the core functionality is there and working
correctly.
I've also got a set of patches to update the 1.6 branch with tweaks to
keep the branches mostly in sync, but Elena will kill me if I import them
before the 1.6.5 release is out... :-)
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committested
Code cleanup
Description:
Merge back changes from "compact group" work that improve the
infrastructure of the library and may impact others. In this round of
merging, that includes:
- Move datatype allocation into single internal routine, instead of
duplicated code that was spread out in a dozen or so places.
- Clean up guts of object header routines (H5O_*) to allow for some of
the fancieroperations that need to be performed on groups, along with
some general improvements.
- Added a new error code
- Some minor cleanups in other code....
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Mac OS X
Code cleanup
Description:
Clean up internals of group creation & iteration code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Mac OS X (nile)
Too minor to require h5committest
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix
Description:
Correct memory leak when a dataset is attempted to be opened, but turns
out to be a group or named datatype.
Also, clean up code that was leading to the leak and zero out empty
group entries to help prevent similar errors in the future.
Platforms tested:
FreebSD 4.11 (sleipnir)
Too minor to require h5committest
Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-)
Description:
Generally speaking, this is the "signed->unsigned" change to selections.
However, in the process of merging code back, things got stickier and stickier
until I ended up doing a big "sync the two branches up" operation. So... I
brought back all the "infrastructure" fixes from the development branch to the
release branch (which I think were actually making some improvement in
performance) as well as fixed several bugs which had been fixed in one branch,
but not the other.
I've also tagged the repository before making this checkin with the label
"before_signed_unsigned_changes".
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & fphdf5
FreeBSD 4.10 (sleipnir) w/threadsafe
FreeBSD 4.10 (sleipnir) w/backward compatibility
Solaris 2.7 (arabica) w/"purify options"
Solaris 2.8 (sol) w/FORTRAN & C++
AIX 5.x (copper) w/parallel & FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN
Linux 2.4 (heping) w/FORTRAN & C++
Misc. update:
Code cleanup & optimization
Description:
Improve ADF/CGNS benchmark by reducing the number of internal attribute
copies made during creations, opens and writes.
Added new H5O_iterate() routine for iterating through messages of a certain
type in the object header (attributes are the only message currently that can
have multiple instances in the object header).
Cross-pollinated various minor code cleanups to reduce diffs between
branches.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
Too minor to require h5committest
Code cleanup
Description:
Update null dataspace changes to try to write older version of dataspace
information whenever possible.
Refactor common code to only one location.
Allow I/O operations to succeed on null dataspaces.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Description: Mainly are header message changes for dataspace. In last round
of check-in, a new header message for dataspace to created, which is not a good
way. Now, there will be no new message for dataspace, but just add the type of
dataspace in the message while increment its version number. Backward compatibility
is addressed. The attribute design is modified accordingly.
Platforms tested: h5committest
Description: NULL dataspace. This step is mainly for dataspace header
message and a test.
Solution: The test mainly checks NULL dataspace features. Backward compatibility
is tested in the fill value test.
Platforms tested: h5committest
Bug fixes and code cleanup
Description:
Lots of changes here:
- Fixed bug #691 - when shared datatypes are used in attributes they
are incorrectly copied into the attribute instead of referring
the the named datatype in the file. This required bumping the
version of the attribute message. The new version of the attribute
message is only written out when a shared datatype is used in
the attribute. [Also, this format change made the size of the
attribute smaller.]
- Added information to attribute debugging routine so that shared
datatypes are displayed correctly with the h5debug tool.
- Refactored the H5O* routines to extract code that was common to
several routines into subroutines to call.
- Added 'link' method for H5O message sub-classes, which increments
the link count on shared objects when a message is created which
shares them.
- Corrected [unreported] bug where the link count was not being
decremented on the shared object when a object header message
with a reference to that object was deleted from the file.
- Reduced size of shared message from 49 bytes (which was incorrect
anyway and should have been 48 bytes) to 10 bytes, which required
bumping the version of "shared" messages.
- Refactored some of the shared datatype routines to allow for easier
queries of "committedness" internally to the library and also
added routine to easily increment/decrement the reference count of
a shared datatype.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code cleanup
Description:
Change some variables that are keywords in C++ to non-keywords.
Platforms tested:
FreeBSD 4.8 (sleipnir)
too minor to require full h5committest
New feature
Description:
Track changes in the internal metadata API with clearing the metadata
dirty flag without flushing object.
Added ability to delete an object header in the file and restore all the
space referenced by various header messages.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Cleanup
Description:
The H5O_FPHDF5 object was left in this header file by mistake when
the H5Ofphdf5 module was removed
Solution:
Removed it.
Platforms tested:
Linux
Code cleanup
Description:
Clean up miscellaneous warnings which have crept into the code.
Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD.
Adjust gcc compiler flags to be more concise for production mode.
Refactor the H5O code so that there is a stronger boundary between code
in the H5O package and code in the library which just calls H5O routines.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2
Misc. update:
Update MANIFEST if you add or remove any file.
