Add 'notify' callback to metadata cache clients, so that they can be
aware when the cache has taken/will take certain actions. Notifications are
urrently limited to when an entry has successfully entered the cache (via an
insertion or a load from the file) and when an entry is about to be evicted
from the cache.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
When an attribute was created with a datatype or dataspace that was shared in
the same object header that the attribute was in, the attribute could not be
deleted. Changes made to ensure that the attribute can be deleted both when the
attribute is in the object header and when it is shared in the heap. Object
header message decode routines now take an "open_oh" parameter to enable them to
avoid opening the same object header twice.
Tested: jam, smirom (h5committest)
Description: When attempting to copy an object with a message shared in its own
object header, the library attempts to protect the same object header twice.
Previously, it was possible for the object header to be protected with write
access in one or both of these protects, which would be illegal. The library
should now always protect with read only access in this case. The conditions
for fixing incorrect datatype versions have been made weaker to support this
change. The version will only be corrected if the object header the datatype
is in is modified.
Tested: jam, smirom (h5committest)
Bring r16305 back from revise_chunks branch:
Add detection of C99 "designated initializers" to configure script and
use new H5_HAVE_C99_DESIGNATED_INITIALIZER macro to conditionally compile
default layout variables in src/H5Pdcpl.c
Also, minor code cleanups, etc.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(Other platforms tested on branch)
Bring file free space branch changes through r15795 into trunk, which
includes a fair bit of code cleanup & rearrangement along with a couple of
bug fixes also.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Centralize all macros for declaring "magic numbers"/signatures for
objects in the file into src/H5Fprivate.h, so it's easier to know what values
have already been defined, etc.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
datatype versions are encountered.
Description: The library now recognizes some problems with datatype versions in
H5O_decode_helper(), and, if not performing strict format checks, automatically
corrects them. Framework added for other message decode routines to
automatically correct file errors. Datatype version information added to
h5debug.
Tested: kagiso, smirom, linew (h5committest)
Add work-around to allow reading files that were produced with a buggy
earlier version of the library, which could create objects with the wrong
object header message count. There is now a configure flag
"--enable-strict-format-checks" which triggers a failure on reading a file
with this sort of corruption (when enabled) and allows the object to be read
(when disabled). The default value for the "strict-format-checks" flag is
yes when the "debug" flag is enabled and no when the "debug" flag is disabled.
Note that if strict format checks are disabled (allowing objects with
this particular kind of corruption to be read) and the file is opened with
write access, the library will re-write the object header for the corrupt
object with the correct # of object header messages.
This closes bugzilla bug #1010.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Clean up ISOHM code further and get rid of several non-optimal ways
of working with object headers.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.9 (amazon)
Check in "unique, but sharable" optimization to ISOHM code, which
allows object header messages that are only used in one object to remain in
the sole user's header, but migrates messages that are used in more than one
header into the ISOHM heap.
Tested on:
Mac OS X/32 10.4.9 (amazon)
FreeBSD/32 6.2 (duty)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Add "fail if unknown" and "mark if unknown" flags to object header messages.
This gives the library a clean way to tell older libraries that a message should
not be just ignored if it's unknown, but that other actions should occur.
Tested on:
Mac OS X/32 10.4.9 (amazon)
FreeBSD/32 6.2 (duty)
FreeBSD/64 6.2 (liberty)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
4294967296LL is not supported by older C like Windows VC6.
Solution:
/* use <= 2**32 -1 to stay within 4 bytes integer range */
HDassert(chunk0_size <= 4294967295UL);
Platforms tested:
kelgia, copper and osage.
dsets failed on kelgia with assertion failure in H5Ocache.c. Kelgia
could not deal with the comparison of uint64_t variable with 4294967296 constant.
Solution:
By John's suggestions used suffix LL for the constant.
Platforms tested:
kelgia, kagiso, and smirom with icc compiler
Add some sanity checking to improve the resilience of the object header
decoding in the face of a bad reference, or a corrupted file.
Tested on:
Mac OS X/32 10.4.9 (amazon)
Eliminate storing # of links in "link info" message, regenerate it
when the object is opened instead.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.8 (amazon)
Move ref. count of # of links to an object out of the object header's
prefix and make it a header message instead (since it's a "rare" occurence),
eliminating some more space for each object in the file.
Inserting this "ref. count" message exposed a flaw in the library's
mechanism for locating a message to promote to another chunk and replace
with a continuation message, which required some additional work to fix.
It's still not completely robust, but it's working for more cases now and
detects failures robustly.
Reduced the minimum size of an object header chunk to just enough to
contain a header message prefix and continuation message.
Tested on:
FreeBSD/32 6.2 (duty)
Fix mis-calculation of chunk #0 size encoding and also catch a few more
places which adjust the size of chunk and have them adjust the encode flags
also. Add some assertions to object header sanity checking routine.
Tested on:
FreeBSD/32 6.2 (duty)
Eliminate message count from new version of object header prefix -
it can be computed when the header is loaded and the table of messages is
built.
Tested on:
FreeBSD/32 6.2 (duty)
Move attribute tracking information out of object header prefix and
make it into a message that is inserted only when attributes are present on
the object.
Tested on:
FreeBSD/32 6.2 (duty)
The main purpose of this checkin was to eliminate the
space used for tracking creation time indices when there is no way they
can be used (i.e. attributes can't be shared in the file and the user hasn't
turned on attribute creation tracking), however there were some other minor
changes which crept in:
- Fix a cache locking deadlock when a shared attribute and one of its
components end up in the same fractal heap direct block.
(This is fixed the "slow" way for right now, until John has time
to add support for readers/writer locking to the cache.
- Optimize attribute copying when a copy will be kept during a v2 B-tree
search.
- When freeing a block on disk, attempt to merge it with the metadata
and "small data" aggregators.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 6.2 (duty)
Add object creation property (H5P[s|g]et_obj_track_times) to disable
storing timestamps on objects, which makes the object's header size smaller.
Also, added object header status flags to H5O_info_t struct (for
H5Oget_info/H5Oget_info_by_idx) and cleaned up other field names in the
struct as well.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.8 (amazon)
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
Add support for inserting attributes into creation order index.
Also, update support for dense link & attribute storage in h5debug.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.8 (amazon)
More progress on creation order for attribbutes - they are now basically
working for "compact" attribute storage.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.8 (amazon)
Remove the "scaffolding" for shared message method invocation and
simplify the way shared messages are dealt with in general.
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.8 (amazon)
Move filter pipeline message class to new "shared message interface"
regime, along with minor fixups to implementation of calling that interface.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Figured out a way to phase in individual message class support for
the new shared message "interface", so check that in before working on the
message classes themselves.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 6.2 (duty)
Add "set_crt_index" and "get_crt_index" methods for the object header
message class.
Unify fractal heap definitions for shared messages and attributes, under
"object header" fractal heap definitions.
Initial code for adding creation order index to object header messages.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Migrate more object header routines to use the H5O_msg_ prefix and put
them into the src/H5Omessage.c code module.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
AIX/32 5.? (copper)
Rename too generic "H5O_FLAG_<foo>" flags that are used as flags for
header messages to "H5O_MSG_FLAG_<foo>", in order to make their use clearer.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
Propagate object creation properties up into group, dataset and named
datatype property lists, when those property lists are retrieved for
existing objects in a file.
Also, add H5Tget_create_plist() API routine, to allow named datatype
property lists to be retrieved for named datatypes.
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
AIX/32 5.? (copper)
Add fields for tracking attributes on objects to object header prefix,
when using the latest version of the format.
Reduce size used to store type of header message from 2 bytes to 1. (If we
have >256 message types, we probably have a different problem... :-)
Also, add "birth" time for object (i.e. creation time) [named to align
with similar fields in file systems]
Correct bug in H5Gget_objinfo() - retrieve the ctime field from object info,
instead of the mtime field.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
When using the latest version of the file format, move the "modification
time" information into the object header prefix, which is more efficient.
Also add "access time" and "change time" (for metadata) fields, all of which
take about the same space as the previous modification time header message.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Finish removing library's internal code that uses H5G_get_objinfo() and
retarget it at either getting the link information or the object information,
as appropriate. (Still need to add user-level tests for H5Oget_info(), but
since several internal components of the library depend on the internal version,
it appears to be working correctly).
Tested on:
FreeBSD/32 4.11 (sleipnir)
Linux/322.4 (heping)
Linux/64 2.4 (mir)
AIX/32 5.? (copper)
Mac OS X/32 10.4.8 (amazon)
Fix several errors in the "latest version" of the object header format
changes and enable the new version when requested now.
Clean up several confusing or duplicated sections of code.
Tested on:
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
FreeBSD/32 4.11 (sleipnir)
handle flush callbacks which can dirty other entries, and
resize and/or rename the target entry.
This feature is needed by the fractal heap code.
Also added associated test code.
H5Commit tested. Test failed on heping, but the
error appears to be a syntax error in an un-related
file.
Tests on copper & sol passed, along with tests on
phoenix.
More fixes to make the I/O & memory operations more efficient when loading
& flushing object header chunks.
Add in concept of a chunk having a "gap" at the end - unused space that is
not large enough to hold a full null message that describes it.
Clean up formatting a bit.
Latest version of object header format not enabled yet, still working on a
few bugs...
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Overhaul usage of object header chunks to reduce I/O operations and
memory allocations. The object header prefix is now stored in the first
object header chunk's "image" in memory.
Also, lots of formatting cleanups.
Taught h5debug tool about new object header format (which isn't enabled
just yet).
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Speculatively attempt to read in the entire first chunk in object header
with the object header prefix. If successful (which it is most of the time),
this reduces by 1 the # of I/Os required to read in the object header. Since
the object header typically was taking 2 I/Os, this should cut the number of
I/O operations when opening an object's header in half.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/C++ & FORTRAN
Clean up some object header code in advance of more major changes in the
near future.
Fix small initialization error in attribute message name buffer copy.
Tweak down the default # of messages for new object headers.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/FORTRAN & C++
Mac OS/32 10.4.8 (amazon)
