Adjust 'temporary' file space code to always be within the file's
address space, even when the file uses 32-bit addresses. (It's not really
necessary to be within the file's address space, since objects with temporary
addresses can't be written to the file, but it can't really hurt and it makes
the testing easier)
Tested on:
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode and with 'make check-vfd'
Description:
When using H5T_copy on committed datatypes that are already open, H5T_copy would
properly use the already existing shared struct, but would still deep copy all
of the fields in that struct. This would cause memory leaks, and in the case of
a compound containing a vlen (or reference), the change in size would cause the
size of the resulting type to be set to an incorrect value. Changed H5T_copy to
properly avoid deep copies when using a reopened shared struct.
Tested: jam, linew, smirom (h5committest), purify on jam
Disable testing the 'temporary' space allocation code when using the
family VFD, since the test causes too many files to be opened.
Tested on:
Linux/64 2.6 (smirom) w/check-vfd
Rework the "proxy address" code to be more general and implement a better
"temporary address" feature, that will allow for metadata allocations to be
deferred to when the metadata is actually flushed to the file.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.7 (amazon) in debug mode
Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Change previous "depend/undepend" routine names to be "support/unsupport"
and add new "depend/undepend" routines, which make the extensible array a child
flush dependency of another piece of metadata in the file.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.7 (amazon) in debug mode
Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Add library private routines to allow metadata cache entries to be set as
flush dependency children on extensible array entries. This will allow chunk
proxies to be set as child flush dependencies for the extensible array, adding
another necessary layer to the flush dependency chain for SWMR access.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Add test for cache client 'notify' callback.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bring r16848 from revise_chunks branch to trunk:
Add test (and bugfixes) for detecting if a filter makes a chunk size
larger than can be encoded in a 32-bit variable (in the file).
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committest not needed, multi-platform testing done on branch)
Bring r16846 from revise_chunks branch back to trunk:
Fix broken (for how long?) H5_ASSIGN_OVERFLOW macro to actually detect
overflows during assignments, along with several errors it [now] detected.
Cleaned up a fix minor warnings and/or pieces of code also.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committest not needed - multi-platform test performed on branch)
Description:
Previously, there was no versioning for H5Z_class_t. This prevented applications
written for 1.6 using custom filters from being able to use the 1.8 library.
There is now an H5Z_class1_t and H5Z_class2_t to enable compatibility. H5Zregister is
*not* versioned, it determines which version of the struct has been passed in by the
value of the first field (id or version, both are ints).
Tested: jam, linew, smirom (h5committest), jam (--with-default-api-version=v16)
Description:
When writing data to a dataset, the data transform was performed after type conversion.
This caused an error if the file type was non-native. This has been changed so data
transforms are always performed on the memory type.
Tested: jam, linew. smirom (h5committest)
- Updated bin/reconfigure to use latest version of automake (1.10.2).
Re-generated Makefile.in's by running bin/reconfigure.
- Added libtool version numbers to c++, fortran, hl, hl c++, and hl fortran
libraries.
Tested:
jam, liberty, smirom
library source directory. Since the test opens the files with write mode, it's safer to
make copies of the original files. I added the copying step in test_family_compat in vfd.c.
I changed the Makefile.am to clean up the copied files. I created a new family_v16_00000.h5
with v1.6 library.
Tested on jam - supposed to be simple change.
Description:
The H5L interface was not marked as uninitialized when H5L_term_interface was
called. This caused the interface to not initialize itself later on. This
prevented external links from working after calling H5close, because they were
never re-registered.
Tested: jam, smirom, linew (h5committest)
Description:
Versions of the library between 1.3.0 and 1.6.3 have a bug which prevents them
from opening any file that does not have the root group's symbol table
information cached in the root group's entry in the superblock. Prior to 1.8
this was not an issue as this information was always cached. However, 1.8.0
stopped writing this information (which is not required by the file format
specification), and these older versions can therefore not read files created or
last written by versions 1.8.0 to 1.8.2. This fix modifies the library to once
again add this information to the superblock (when using the old file format).
Tested: jam, linew, smirom (h5committest)
Description:
Changed Skip list package to use a deterministic skip list. This allows the
skip list package to avoid calling rand() and srand(), even on machines without
rand_r(). There is no longer a p-value or maximum level for skip lists.
Tested: jam, smirom, linew (h5committest)
Description:
Previously, H5Lcopy and H5Lmove would (through H5L_move) improperly apply the
"create intermediate groups" property to the source path traversal, and not the
destination. Fixed it to apply the property to the destination and not the
source. Also fixed H5Lcreate_ud to reject internal link classes without
throwing an assertion.
Tested: linew, jam, smirom (h5committtest)
some argument check in the internal function H5I_find_id and took out the assertion check.
I also removed the argument check in H5Iis_valid because it's in H5I_find_id now.
Tested on jam - simple change. Tested v1.8 already.
Pass some user data down into the extensible array client context creation
callback.
Tested on:
FreeBSD/32 6.3 (duty)
Too minor to require h5committest
Remove unnecessary 'udata' parameter from H5AC_set() and 'notify' callback
(introduced with checkin that added the 'notify' callback)
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Add 'notify' callback to metadata cache clients, so that they can be
aware when the cache has taken/will take certain actions. Notifications are
urrently limited to when an entry has successfully entered the cache (via an
insertion or a load from the file) and when an entry is about to be evicted
from the cache.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Add mechanism for querying if an entry in the metadata cache is the
parent or child in a flush dependency relationship.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Revise how code for array statistics works.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Modify metadata cache flush dependency feature to allow it to work
with entries that are pinned through the cache API calls.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
Due to a bug in H5F_super_read, every time a file with a user block was opened
it would grow by the size of the user block. The bug has been fixed, and
comments have been added to clarify when an eoa address should be relative and
when it should be absolute.
Tested:
jam, linew (h5committest)
v1.8 library, v1.8 library writes the driver info block in the superblock. But v1.6
doesn't write it. This caused the data after the superblock to be overwritten. The
solution is to use a flag to indicate when the original file doesn't have the driver
info, v1.8 library doesn't write it either.
Tested on jam. I have some trouble to access smirom and linew using h5committest.
Ensure that metadata cache entries with flush dependencies are written
to the file in the correct order (for serial I/O, at least).
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Description:
Some files apparently exist in the wild which have corrupt symbol table messages
on the root group. These files can be opened by 1.6 (which uses the cached
information in the superblock) but not by 1.8. This patch fixes 1.8 and 1.9 so
they can now open these files, and will correct them if necessary. Also fix
some potential (rare) problems with array datatype versions.
Tested: jam (parallel; h5committest not working on linew or smirom)
Add infrastructure & tests for "flush dependencies" in metadata cache,
which allow relationships that specify which order to flush metadata entries in.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Clean up many compiler warnings and make test return non-zero exit code
when a failure is detected.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Bug Fix
Description:
Fixing BZ #1381. The --includedir=DIR configure option, which is used
to specify the installation location of C header files, did not work
correctly as the path was hard-coded in config/commence.am. I'm presuming
this is because an older version of automake didn't know where to put
c header files. In any case, removing this line now defaults the includedir
to the same directory that it is currently hard-coded to, and also fixes
the configure flag to allow customization of this value.
Tested:
jam, liberty
Remove another call to H5E_clear_stack() from within the library.
Clean up lots of compiler warnings.
Tested on:
Mac OS X/32 10.5.6 (amazon)
(followup on other platforms forthcoming)
test the correctness of the data when the fill value is defined or not. The
library should let the chunks bypass the cache depending on the size of the
chunks and whether to write fill value to the chunks.
Tested on jam - simple change.
Description:
In some situations it was possible for the fill value to not be written to parts
of a chunked dataset, particularly when extending and/or shrinking. Prior to
the fix for the chunk cache (1015) these bugs would have been exceedingly rare.
Tested: jam, smirom, linew (h5committest)
the way h5ls prints types, it starts searching for NATIVE types first. One solution would be h5ls not to detect these native types, using for example the same print datatype function that h5dump does, that would make the output look the same on all platforms ("32-bit little-endian integer" would be printed instead). Drawback, this "native" information would not be available. Other solution is to have not one but 2 expected outputs and make the shell script detect the endianess and compare with one output or other
tested: h5committest
Clean up code and eliminate resource leaks. Also avoid "null" I/O when
chunk doesn't exist and we can skip it.
Tested on:
Mac OS X/32 10.5.6 (amazon)
(too minor to require h5committest)
Clean up (i.e. remove) more internal calls to H5E_clear_stack(), along with
some other minor code cleanups.
Tested on:
Mac OS X/32 10.5.6 (amazon)
(too minor to require h5committest)
Description:
The meaning of the "nbytes" field in H5D_rdcc_t was not clear, and some places
assumed it was the maximum size of the chunk cache, while some assumed it was
the current size of the chunk cache. The end result was that only 1 chunk could
be held in cache at a time. This field has been replaced by "nbytes_max" and
"nbytes_used". Performance of cached I/O should improve greatly.
Tested: jam, smirom (h5committest)
file handles.
Description:
An attribute's "oloc" field which specifies the file it resides in was located
in the attribute's "shared" structure. So when an attribute was opened multiple
times all of the handles for that attribute pointed to the same file id, even if
different file id's were used to open the different handles for the attribute.
The "oloc" has been moved to the top level H5A_t struct.
Tested: jam, smirom (h5committest)
Description:
Since the new object header format, it has been possible for a situation to be
created where none of the messages are large enough to hold a continuation
message and there are no null messages to merge with. This makes it impossible
to add a new object header chunk. This case will now be handled by moving every
message in the last chunk to the newly allocated one, except for null messages
which are deleted.
Tested: jam, smirom (h5committest)
Description:
When an attribute was created with a datatype or dataspace that was shared in
the same object header that the attribute was in, the attribute could not be
deleted. Changes made to ensure that the attribute can be deleted both when the
attribute is in the object header and when it is shared in the heap. Object
header message decode routines now take an "open_oh" parameter to enable them to
avoid opening the same object header twice.
Tested: jam, smirom (h5committest)
on disk, the library still loaded it in the cache, which is redundant. I changed it to bypass the
cache and added a test in dsets.c.
Tested on jam and smirom.
Bring r16435 from revise_chunks branch back to trunk:
Expand object copy tests for chunked datasets to include 1-D datasets
with an unlimited dimension. (Fix typo in comment for test/links.c)
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(more thoroughly tested already on revise_chunks branch)
Call h5_fixname (with an array of test filenames) for generating the
filename to create and then call h5_cleanup() when the tests pass, to delete the files
created and close the FAPL from h5_fileaccess().
Defined a macro
#define TESTING2(WHAT) {printf(" Testing %-62s",WHAT); fflush(stdout);}
Similar to TESTING, except that it has a initial indentation space.
The effect is for nested loop tests
Testing with old file format:
Testing with fill value, no compression PASSED
tested: windows, linux
Bring r16416 from revise_chunks branch to trunk:
Bring closer to standard standalone test format, add checks for using
the latest file format and close dataset ID leaked.
Tested on:
FreeBSD/32 6.3 (duty)
(too minor to require h5committest)
Description:
Attribute object copy routines have been moved from H5Oattr.c to H5Aint.c.
These routines are now shared between compact and densely stored attributes.
New routines written to support the copying of dense attributes. This patch wasmostly written by Peter Cao.
Tested: jam, smirom (h5committest)
Changed max macro in earray.c to avoid Windows conflict.
Changed if(test) to assert(test) in H5FDcore.c - H5FD_core_open()
Tested on:
Windows: XP32, XP64, Vista32, Vista64
H5committest.
Description: H5Tset_order will now properly reject H5T_ORDER_NONE for most
datatypes. Previously this could cause major problems as the file could not be
flushed.
Tested: jam, smirom (h5committest)
Description: H5Tinsert will now detect when a compound type that was previously
not packed becomes packed due to out of offset order insertion of a member.
H5Tinsert will now attempt to keep members sorted by offset order. This should
improve performance of H5Tinsert in all cases due to the fact that it no longer
needs to check every other member for overlapping, and should improve
performance of H5Tpack and possibly type conversion when compounds are packed
out of order.
Tested: jam, smirom (h5committest)
Bug Fix
Description:
Removing some unnecessary system headers from test/cache.c which snuck
their way into one of my previous check-ins ... thanks to Allen for
catching this while testing on Windows.
Tested:
Windows, Linux
Description:
If a compound type was packed except for some extra space at the end, H5Tpack
would not modify the type and the extra space would remain. Changed
H5T_is_packed to fix this behaviour.
Tested: jam, smirom (h5committest - linew down)
Bug Fix
Descriotion:
Removing problematic debugging code, and switching a leftover TRUE verbose
statement to FALSE in cache_common.c
Tested:
jam, liberty
Adding code to maintain a min_clean_fraction in the cache in serial mode.
Description:
The metadata cache now has the ability to maintain a min_clean_fraction
when in serial mode. The default initial cache size has been changed
from 1MB to 2MB, and the default min_clean_fraction has been set at 30%.
This check-in includes modifications to H5C.c to support maintaining a
min_clean_size, including the addition of clean_index_size and
dirty_index_size trackers, modifications to the H5C_make_space_in_cache
algorithm, as well as associated test code and additional statistics
tracking variables.
Maintaining the min_clean_fraction addresses the possibility of
experiencing a "metadata blizzard" when the cache gets completely
full with dirty entries. Upon having to make space, the cache would
previously need to flush every single entry in the cache before coming
across a clean entry which could be evicted. This resulted in unnecessary
flushing of oftentimes hot entries in the cache. Maintaining the
min_clean_fraction ensures that, when space is needed, clean entries
are more readily available to evict.
Tested:
jam, smirom, linew (h5committest)
Description: Added H5Pset/get_elink_cb to allow the user to specify a callback
function to be called whenever an external link is traversed. Added
H5Pset/get_elink_acc_flags to allow the user to specify the file access flags
to use to open the target file of an external link. All these properties are set on a LAPL.
Tested: jam, linew, smirom (h5committest)
new test program: It adds tests for several ranks, use of fill value or not, compression, different fill value allocation times, use of different storage layouts, and external files
tested: windows, linux
Add support for paging large data blocks to extensible arrays
Clean up allocation/destroy code for extensible array index, super and
data blocks
Add a couple of routines to set/get bit values in memory buffers
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Add initial support for data block 'page init' bitmasks to super
blocks.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Initial changes to allow data blocks to be paged in, instead of loaded
completely.
Also, refactored test code to be smaller and more maintainable.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Switch H5Gcreate -> H5Gcreate2 and H5Gopen -> H5Gopen2, so that they aren't
switched with the API compability flags are changed
Tested on:
Mac OS X/32 10.5.5 (amazon)
Linux/64 2.4 (smirom) w/API compability
Description: Fixes a problem in H5F_close_mounts where it wouldn't correctly
reshape the "child" array when unmounting files. Test added for this case.
Also fixed a potential bug in H5F_unmount where that routine reshapes the child
array.
Tested: kagiso linew smirom (h5committest)
Description: Adds 3 new public functions: H5Pset_chunk_cache,
H5Pget_chunk_cache, and H5Dget_access_list. First implementation of dataset
access properties. See RFC_chunk_cache_functions for details.
Tested: kagiso, linew, smirom (h5committest)
Add more regression tests for extensible array data structure
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Add random iteration to extensible array tests
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Minor cleanups of new testing code.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
More tests for extensible arrays.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Description: H5Iis_valid function added. This function checks if the id is
valid (i.e. it is currently registered and visible to the application), and
returns TRUE if it is valid, or FALSE otherwise. It should never return an
error.
Tested: kagiso, linew, smirom (h5committest)
Description: H5Ovisit and H5Ovisit by name will now check for a positive return
value from the first callback. Test added for this case.
Tested: kagiso
th user's definition of H5_USE_16_API. The H5_USE_16_API_DEFAULT is defined through
configure with the flag --with-default-api-version=v16.
Tested with h5committest.
Description:
- Remove need to set LD_LIBRARY_PATH when using shared szip library.
- Libtool 2.2.6a is now used to generate libraries.
- 'make check install' dependency bug is fixed, and should no longer
break the build.
- removed hard coding of shell in config/commence.am, as this causes
problems on Solaris with the new version of libtool.
- RELEASE.txt with appropriate changes.
Tested:
- kagiso, smirom, linew (merged from 1.8, pretty quick tests)
Description: Added a new function, H5O_alloc_shrink_chunk, which removes all
null messages from a chunk and shrinks the chunk appropriately. Modified
H5O_merge_null to call this function when a null message is created with a size
>= 64k. Tests added for this functionality.
Tested: kagiso (h5committest on 1.8 version)
Correct a minor error in file free space allocation which was affecting
the 'multi' VFD and preventing some tests from fully working with it.
Wholesale revisitation of all the places where tests were disabled
with various VFDs and remove or correct all these so that _only_ the tests
which _really_ can't work with particular VFDs are skipped during a
'make check-vfd' test.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Various bug fixes & cleanups with VFDs and file free space changes,
along with cleaning up compiler warnings.
Enable more VFDs (for 'make check-vfd' testing) for many tests.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Description:
This checkin includes revisions 15845, 15846, and 15853 from the 1.8 branch. From the svn log:
r15845:
Purpose: Add Windows-specific version of HDftruncate
Description:
Windows doesn't include a version of the system call ftruncate. There is a similar call, _chsize_s, which performs very similarly. Thus, we map HDftruncate to _chsize_s in our Windows header file, H5win32defs.h.
r15846:
Purpose: Declare loop variable at beginning of function
Description:
On Windows, we were getting compile errors because h5test.c includes a function with variable declarations in the middle of the call. The Microsoft C compiler demands that all variables be declared at the top of the function. This checkin simply moves the declaration to the top to satisfy Visual Studio.
r15853:
Purpose: Decrease size of points array in links test
Description:
In the external_set_elink_fapl2 links test, there is a large array of points which declared on the stack for testing. Previously, the array was 1000 x 1000, which was too large for Visual Studio to handle. As a result, we were getting stack overflows during the test. We've reduced the number to 400, as this seems to be below the limit. The exact number of points in the array isn't important to the test.
Tested:
VS2005 on WinXP
Skip some free space manager tests when using a VFD that doesn't
support the feature being tested.
Use a particular memory type with the file free space sections.
Uncomment the metadata aggregator feature for the fmily VFD, as it
appears to be working after the file free space changes.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Fix various problems with a the core & sec2 VFDs.
Improve the h5_get_file_size() routine to handle files created with
VFDs that use multiple files.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Skip some file free space tests that deal with aligned allocations when
using the stdio VFD, which has its own allocation process that drops the
unaligned fragments instead of tracking them with the free space manager.
Tested on:
Mac OS X/32 10.5.5 (amazon) w/stdio VFD
Bring file free space branch changes through r15795 into trunk, which
includes a fair bit of code cleanup & rearrangement along with a couple of
bug fixes also.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Change H5Tcommit() call to H5Tcommit2() call, so that library builds with
the default API set to v16 work.
Tested on:
Linux/64 2.4 (smirom) w/default API set to v16
Description: Fixes a possible datatype id leak that could occur during compound
datatype conversion, or more precisely, when unregistering those conversions.
Datatype ids normally registered by the library are no longer visible to the
application via H5Fget_obj_ids and H5Fget_obj_count.
Tested: kagiso, linew, smirom (h5committest)
to use when opening the external linked target file.
1. Two new public routines are added to H5Plapl.c as well as
"del/copy/close" callbacks for the property itself.
2. Modify H5L_extern_traverse() to use the fapl set via
H5Pset_elink_fapl() and retrieve via H5Pget_elink_fapl().
3. Add 3 tests to links.c to verify H5Pset/get_elink_fapl().
Also fix the compiler warning for the "if condition" in
H5_build_extpath() of H5system.c.
Fix some minor problems w/extensible array super blocks and extend
tests.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Add support for extensible array "super block".
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Extend testing for data block reachable from the index block.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Correct a couple of minor bugs for handling data blocks from index
block and extend tests further.
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Further progress on extensible array implementation - add support for
more data blocks from index block.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Description: The optimized "subset" compound conversion function would
improperly copy the "blank" space at the end of compound types. Modified
H5T_conv_struct_init to detect when the subset type has extra space at the end,
and calculate the size of the data that should be copied into the conversion
buffer for each element. Changes to the functions that implement these
conversions.
Tested: kagiso, linew, smirom (h5committest)
Correct small error in allocating data block elements and add some more
tests.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Add base support for extensible array "data blocks" to code, tests and
h5debug.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Description:
We've run into a few Windows-specific problems with the new H5EA interface. In particular, this checkin makes the following changes:
-- Change shutdown defined in earray.c to shutdown_ea, to avoid conflicting with another Windows-specific system call.
-- Prefix H5EA_get in H5EAprivate.h with H5_DLL, so the function is properly exported in Windows DLL version.
These changes shouldn't break any other platforms.
Tested:
VS2005 on WinXP
h5committest (linew kagiso smirom)
Description: Added configure test to see if pointer alignment restrictions are enforced (as in dereferencing an unaligned pointer causes an error). Added code in H5Tvlen.c to avoid dereferencing unaligned pointers, conditionally compiled based on the configure test. Added test case in dtypes.c which would previously cause such machines to fail.
Tested: kagiso, smirom, linew (h5committest); linew64
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
Add additional test and make a few cleanups in extensible array unit
test.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
More extensible array features: add support for setting/getting
elements in the array's index block.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Update extensible array code with function to open an existing earray,
add more tests and avoid running the test when core/split/family/multi VFDs
are used.
Clean up fractal heap test code a bit and expand some of the tests a
little bit also.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Initial checkin of extensible array data structure prototype code and
regression tests.
Initial definitions for revised FUNC_ENTER/LEAVE and error reporting
macros, which are being vetted in the extensible array code.
Minor warning and formatting cleanups in other sections of code.
Tested on:
Mac OS X/32 10.5.4 (amazon) in debug mode
Mac OS X/32 10.5.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Fix bug where library could core dump when an invalid location ID was
passed to H5Giterate() (and add test for this case).
Tested on:
Mac OS X/32 10.5.4 (amazon)
Too minor to require h5committest
> 1.
Description: Added a new field 'app_count' to H5I_id_info_t struct, to track
the reference count on an id due to the application. the old 'count' field
tracks the total. Generally any id visible to the application gets placed
in app_count. Added app_ref boolean parameter to H5I_inc_ref, H5I_dec_ref,
H5I_register, H5I_clear_type, and a few other functions, to specify whether
the operation(s) being performed on the id(s) are due to the application
(TRUE) or not (FALSE). Test added for this case.
Tested: kagiso, smirom, linew (h5committest)
Description:
Moved mount table from top file structure to shared file structure. Moved
parent out of mount table and back into top file structure. Mounted files can
now be accessed from any handle of the parent file. Changes to how files are
closed. Stricter cycle checking on mounted files. Removed unused function
H5F_has_mount().
Tested:
committest in 1.8 branch. Committing now due to the urgency of the fix. No
changes here are specific to the trunk, but I will keep an eye on the daily
tests.
the file didn't have the data. It happened because each handle had its own
object structure, and the empty one overwrote the data with fill value. This is
fixed by making some attribute information like the data be shared in the
attribute structure.
Tested on smirom, kagiso, and linew.
Add check to avoid mounting the a file on a group twice, then the mounts
are done on the same HDF5 file, but opened with separate H5Fopen calls.
Also add new 'mounted' flag to the H5G_info_t struct, queried with the
H5Gget_info() API call, to allow applications to detect and avoid this
situation.
This probably fixes Bz#1070 also, I'll check with Dan Anov (who reported
a different sort of behavior, but seems to have the same underlying problem).
Tested on:
Mac OS X/32 10.5.4 (amazon)
Linux/64 2.6 (chicago)
Description:
On Windows, certain users were having trouble with the "ohdr" test, which does some processing on object header messages. The errors were hard to reproduce on our machines, and we eventually determined that the errors were timezone-specific.
The bug is triggered on Windows when processing timestamps very near the "Epoch" (midnight on 1/1/1970)-- the mktime() function does some automatic adjustment on the time to correct for timezones. In the USA, the correction adds a few hours; in Europe, it subtracts, thus giving us times pre-Epoch.
This only affects Windows because the Windows mktime() function cannot handle times before 1970-- other systems seemingly can.
The fix is to simply create timestamps only as early as 01/02/1970. This way, any timezone adjustment will still be post-Epoch.
This bug only affects the ohdr test, and shouldn't be a problem in the library. The earliest timestamps that will actually be read will be around the time HDF5 was created (~1996-7, per Quincey).
Tested:
VS2005 on WinXP
h5committest (kagiso, linew, smirom)
Description:
As part of our Windows cleanup, we try to remove windows-specific tweaks in the source code. There are many instances where Windows code is introduces via ifdef's. We re-evaluate whether they are still required, and found that many of them are not. Others we change to "feature"-specific code, rather than Windows-specific.
Tested:
VS2005 on WinXP
VS.NET on WinXP
h5committest (kagisopp, smirom, linew)
Description:
On Windows, the pthread_self function cannot be used to print the returned thread ID for debugging. Instead, we need a separate function, GetCurrentThreadId. To eliminate some Windows ifdef's in the code, we create two new function macros which can be used by all platforms. It is conditionally defined in H5win32defs.h, and globally in H5private.h.
Tested:
VS2005 w/ pthreads on WinXP
kagiso w/ pthreads
Description:
In library code, we try not to use system calls directly, but instead use the HD{function} macro instead. This way, we can map special versions of the call on particular systems. Previously, it was all done in H5private.h. However, in an effort to clean up platform-specific definitions, we move all of the Windows macros into a separate file, win32defs.h. This way, we can use the non-Posix versions that Visual Studio sends warnings about.
Some macros are set specifically in the platform-specific header files. Then, any macros left unset will be set by the "default" implementation in H5private.h.
This checkin also cleans up various source files to use the HD* macros when possible.
Tested:
VS2005 on WinXP
VS.NET on WinXP
h5committest (kagiso, linew, smirom)
Finish omnibus chunked dataset I/O refactoring, to separate general
actions on chunked datasets from actions that are specific to using the v1
B-tree index.
Cleaned up a few bugs and added some additional tests also.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.2 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Description:
On Linux-like systems, we can get the ID of the current thread through a pthread_self. However on Windows, the return cannot be cast as a threadID, so we simply couldn't get the ID. Previously we simply gave up and printed a message that we couldn't get an ID. Instead, though, we can use the Windows-specific call to GetCurrentThreadId(), which achieves the same goal. This way we can provide better debug output with threadsafe features.
Tested:
VS2005 on WinXP
VS.NET on WinXP
(other platforms not tested because change is within _WIN32 ifdef)
Description:
The fortran Makefile.am used HDF_FORTRAN to indicate it is part of the
Fortran API source so that conclude.am will give fortran api prefix in the
test output. The symbox HDF_FORTRAN is also used in configure for a different
purpose (indicated --enable-fortran). They conflicted.
Similar problem for the symbol HDF_CXX.
Solution:
Changed all the involved Makefile.am to use "FORTRAN_API" instead. It is
a more appropriate name. Same for CXX_API.
Along the way, discovered that the Makefile.am of hl/fortran/test and
hl/cxx/test did not have those symbols at all. Added them in.
Platform tested:
Kagiso only. It is a trivia change.
Description: The test program h52gifgentst was getting installed in the bin
directory during 'make install', and it shouldn't. Make now
builds the program for use in testing but doesn't install it
during 'make install'.
Tested: kagiso
Detect chunks that are >4GB before dataset gets created and return error
to application.
Tweak lots of internal variables that hold the chunk size/dimensions to
use an 'uint32_t', instead of a 'size_t', so that the integer size is constant.
Correct a number of our tests which were creating datasets with chunks
that were >4GB and add some specific tests for >4GB chunk size detection.
Minor whitespace & other code cleanups.
Tested on:
Mac OS X/32 10.5.2 (amazon)
Forthcoming testing on other platforms...
Improvement.
Description:
src/libhdf5.settings was the initial configure summary and is installed.
Then configure is changed to dump a summary of the configure settings to
the output and also append it to src/libhdf5.settings. That created
two different output formats and duplicated information. This is the
initial attempt to clean up this confusion and unify the output format.
It is decided to use the src/libhdf5.settings template as the unified means.
This requires more macros symbols be defined. The following symbols are
all related to generating the src/libhdf5.settings file.
AC_SUBST(EXTERNAL_FILTERS)
AC_SUBST(MPE) MPE=no
AC_SUBST(STATIC_EXEC) STATIC_EXEC=no
AC_SUBST(HDF_FORTRAN) HDF_FORTRAN=no
AC_SUBST(FC) HDF_FORTRAN=no
AC_SUBST(HDF_CXX) HDF_CXX=no
AC_SUBST(CXX) HDF_CXX=no
AC_SUBST(HDF5_HL) HDF5_HL=yes
AC_SUBST(GPFS) GPFS=no
AC_SUBST(LINUX_LFS) LINUX_LFS=no
AC_SUBST(INSTRUMENT) INSTRUMENT=no
AC_SUBST(CODESTACK) CODESTACK=no
AC_SUBST(HAVE_DMALLOC) HAVE_DMALLOC=no
AC_SUBST(DIRECT_VFD) DIRECT_VFD=no
AC_SUBST(THREADSAFE) THREADSAFE=no
AC_SUBST(STATIC_SHARED)
AC_SUBST(enable_shared)
AC_SUBST(enable_static)
AC_SUBST(UNAME_INFO) UNAME_INFO=`uname -a`
The src/libhdf5.settings.in has CONDITIONAL's added to it too. The
untrue conditions turned into a "#" and these lines are cleaned by the
post processing script.
Platform tested:
h5committest on kagiso, smirom and linew.
Description: Applying update to autotools that was applied to 1.8 a couple
of weeks ago to the trunk.
Updated bin/reconfigure script to reflect the new versions of
libtool and automake in the /home1/packages/ directory.
Rearranged configure.in script. When using libtool 2.2.2, the
libtool script doesn't generate until later in the configuration
process, so I had to move a test that parsed through the libtool
script to a point after where it was actually being generated.
Ran libtoolize on the project, and ran bin/reconfigure to
regenerate configure and Makefile.in's throughout.
Tested: kagiso, smirom, linew (h5committest)
Omnibus raw data I/O revisions, with wide-ranging changes and
refactoring, in order to prepare for implementing "fast append" feature.
These changes remove the majority of the code duplication for raw data
I/O which has crept in over the last ten years and introduces a more object-
oriented design for operating on different types of dataset storage.
Chunked storage no longer has it's own I/O routines, it is now handled
as either contiguous (if chunk is not pulled into the cache) or compact (if the
chunk is cached in memory).
No bug or feature changes, at least intentionally... :-)
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.2 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Description:
Vailin has been working on some new tests for converting Windows paths. She found a bug that is making these two tests fail, but didn't have time to fix it. We've commented out the two tests until she has time to fix the bug. This won't affect other platforms because it's Windows-specific code.
Tested:
VS2005 on WinXP
remove HDputenv() from external_link_env() test
(will add script later to set HDF5_EXT_PREFIX for running the test)
modify and add more comments
2. src/H5private.h: remove #define for HDputenv()
Tested on kagiso, linew and smirom.
Bring r14737 back from the 1.8 branch: Fix bug which would
incorrectly encode the member offsets for compound datatypes whose size was
between 256 & 511 bytes, when the "use the latest format" feature was enabled.
Tested on:
Mac OS X/32 10.5.2 (amazon) w/debug
FreeBSD/32 6.2 (duty) w/production
Minor bug fix to H5Aget_num_attrs() to return error when an invalid
location ID is passed in.
Tested on:
Mac OS X/32 (amazon)
Too minor to require h5committest
callback functions modifies the skip list or LRU out from under a partial
or complete flush of the cache. This is a test for a situation that
should not occur, but we test anyway.
Also added code to skip longer tests in cache_api in express tests.
Tested serial and parallel on phoenix (debian --x86-32), and commit test.
Elena commit tested as well, and ran a manual test under MacOS X.
large entry, or to a large increases in the size of an existing entry.
This required some additions to the cache configuration structure, and
thus will require changes in the metadata cache documentation.
The basic idea is to monitor the size of entries as they are loaded,
inserted, or increased in size. If the size of the entry (or increase)
exceeds some user selected fraction of the size of the cache, increase
the size of the cache.
Note that this fix was designed quickly -- while it deals with the
use case that exposed the problem, we may have to revisit the issue
later.
Tested serial and parallel on Phoenix, and h5committest.
Correct the prototype for H5Sselect_elements() to take an 'hsize_t *' for
the coordinates, instead of 'hsize_t **'.
Tested on:
Mac OS X/32 10.5.1 (amazon)
Add work-around to allow reading files that were produced with a buggy
earlier version of the library, which could create objects with the wrong
object header message count. There is now a configure flag
"--enable-strict-format-checks" which triggers a failure on reading a file
with this sort of corruption (when enabled) and allows the object to be read
(when disabled). The default value for the "strict-format-checks" flag is
yes when the "debug" flag is enabled and no when the "debug" flag is disabled.
Note that if strict format checks are disabled (allowing objects with
this particular kind of corruption to be read) and the file is opened with
write access, the library will re-write the object header for the corrupt
object with the correct # of object header messages.
This closes bugzilla bug #1010.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
