Commit Graph

103 Commits

Author SHA1 Message Date
Quincey Koziol
85504f5376 [svn-r26195] Description:
Switch from enum H5D_CHUNK_BTREE to H5D_CHUNK_IDX_BTREE, along with a couple
of minor whitespace touchups.

Tested on:
    Mac OSX/64 10.10.2 (amazon) w/serial & parallel
    Linux/32 2.6.x (jam) w/serial
2015-02-17 15:11:39 -05:00
Quincey Koziol
3de25ee226 [svn-r24998] Description:
Check in Neil's changes for correctly handling case where library is
reinitialized through a deprecated routine.

Tested on:
    Linux 2.4.x/32 (jam)
2014-04-09 00:08:21 -05:00
Quincey Koziol
ac0ef34a41 [svn-r24769] Description:
Check in Mohamad's changes to support collective I/O on point selections,
along with some other minor cleanups.

Tested on:
    Mac OSX/64 10.9.2 (amazon) w/parallel & serial
    (h5committest forthcoming)
2014-03-07 14:12:20 -05:00
Quincey Koziol
424a41f878 [svn-r23713] Description:
Clean up warnings, switch library code to use Standard C/POSIX wrapper
macros, remove internal calls to API routines, update checkapi and checkposix
scripts.

Tested on:
    Mac OSX/64 10.8.3 (amazon) w/C++ & FORTRAN
    Big-Endian Linux/64 (ostrich)
2013-05-21 12:30:54 -05:00
Jonathan Kim
0aa8d4d88a [svn-r22735] Purpose:
HDFFV-8143 Provide a routine(s) for telling the user why the library broke collective data access

Description:
    Added H5Pget_mpio_no_collective_cause() function that retrive reasons why the collective I/O was broken during Read/Write IO access.
    Reasons to break collective I/O:
     - SET_INDEPENDENT
     - DATATYPE_CONVERSION
     - DATA_TRANSFORMS
     - MPIPOSIX
     - NOT_SIMPLE_OR_SCALAR_DATASPACES  (NULL Space)
     - POINT_SELECTIONS
     - NOT_CONTIGUOUS_OR_CHUNKED_DATASET  (Compact or External-Storage)
     - FILTERS  

Tested: 
    jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
2012-09-04 12:00:38 -05:00
Quincey Koziol
75f4e51a1a [svn-r22582] Description:
Refactor ID class registration to be more like how other "class"s in the
library are registered, and clean up compiler warnings.

Tested on:
    Mac OSX/64 10.7.4 (amazon) w/debug
    (Too minor to require h5comittest)
2012-07-16 14:54:26 -05:00
Quincey Koziol
a07c8f9247 [svn-r22287] Description:
Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward
the final design (as exemplified by the H5EA & H5FA code).

Tested on:
    Mac OSX/64 10.7.3 (amazon) w/debug & parallel
2012-04-16 16:20:26 -05:00
Vailin Choi
b643da672a [svn-r22164] Fixed a bug in H5Ocopy(): When copying an opened object, call the object's
flush class action to ensure that cached data is flushed so that H5Ocopy will get
the correct data.  (HDFFV-7853)
2012-03-28 01:21:14 -05:00
Quincey Koziol
3d9e0b86ac [svn-r21923] Description:
Add FUNC_ENTER macros for package-private routines and begin process of
switching package routines to use them.  All H5G routines are currently
finished.

Tested on:
    Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
2012-02-09 21:16:52 -05:00
Quincey Koziol
cab5c372bf [svn-r20135] Description:
Bring r20134 from metadata journaling merging branch to trunk:

	Bring changes from metadata journaling branch to "merging" branch:

Unify routine to mark a dataset's metadata as changed.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
                w/C++ & FORTRAN, w/threadsafe, in debug mode
        Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                w/szip filter, w/threadsafe, in production mode
        Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
2011-02-21 19:03:37 -05:00
Quincey Koziol
fbdae40a22 [svn-r19638] Description:
Bring some of Neil's changes to help support SWMR access in the v1
B-tree code from the revise_chunks branch back to the trunk.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
                w/C++ & FORTRAN, w/threadsafe, in debug mode
        Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                w/szip filter, w/threadsafe, in production mode
        Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
        Mac OS X/32 10.6.4 (amazon) in debug mode
        Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
                in production mode
        Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
2010-10-19 11:54:27 -05:00
Quincey Koziol
dc4756fbf4 [svn-r19498] Description:
Optimize the vector-vector memcpy() routine even further, for a total
of ~2x speedup.  :-)

	Make a generic vector-vector operation routine and convert other
vector-vector read & write routines to use generic routine instead of multiple
copies of the basic algorithm.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
                w/C++ & FORTRAN, w/threadsafe, in debug mode
        Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                w/szip filter, w/threadsafe, in production mode
        Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode
        Mac OS X/32 10.6.4 (amazon) in debug mode
        Mac OS X/32 10.6.4 (amazon) w/C++ & FORTRAN, w/threadsafe,
                in production mode
        Mac OS X/32 10.6.4 (amazon) w/parallel, in debug mode
2010-10-01 00:39:05 -05:00
Neil Fortner
686d856802 [svn-r19006] Purpose: Improve performance of the the chunk cache
Description:
Previously, the chunk cache would, when looking for a chunk, alwaylook for the
chunk in the b-tree before checking the cache.  Reworked the functions that
look up chunks to always check the cache first and avoid the b-tree lookup if
the chunk is in cache.

Tested: jam, linew, amani (h5committest)
2010-06-15 16:21:03 -05:00
Neil Fortner
3f837fc323 [svn-r18894] Purpose: Improve allocation performance of filtered datasets with non-default VL
fill values.

Description:
Previously, H5D_chunk_allocate would, if a dataset were filtered and had a non-default VL fill value, allocate a new buffer for every chunk.  Changed
H5D_chunk_allocate to reuse the existing buffer if possible.  Also other
miscellaneous cleanup.

Tested: jam, linew, amani (h5committest)
2010-05-25 15:39:52 -05:00
Neil Fortner
2db0e8ecf3 [svn-r18868] Purpose: Fix bug in dataset shrinking algorithm
Description:
Previously, it was possible for a chunk to be flushed due to chunk operations in
the callback from H5B_iterate in H5D_chunk_prune_by_extent.  Because flushing
the chunk can force it to be reallocated if it is filtered, this can change the
contents of the chunk b-tree in the middle of H5B_iterate.  Because H5B_iterate
uses a locally cached copy of the b-tree, this causes subsequent operations
to be passed incorrect data.  Rewrote H5D_chunk_prune_by_extent to avoid
H5B_iterate entirely.

Also fixed a bug in the dataset expand algorithm that could cause extra chunks
to be created.

Tested: jam, linew, smirom (h5committest)
2010-05-20 16:55:55 -05:00
Quincey Koziol
b9ca2f8f6f [svn-r18650] Description:
Bring back various minor tweaks & cleanups from the revise_chunks
branch.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
                w/C++ & FORTRAN, w/threadsafe, in debug mode
        Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x,
                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
2010-04-27 17:15:14 -05:00
Neil Fortner
d3d4d8782b [svn-r18445] Purpose: Fix bug 1637
Description:
Modified algorithm for extending a dataset with early allocation so it only
deals with the new chunks.  Formerly, it would loop over all chunks, checking
to see if each existed in cache and on disk, causing major performance issues
with large numbers of chunks.

Tested: jam, linew, amani (h5committest)
2010-03-24 11:41:14 -05:00
Quincey Koziol
ee383d182f [svn-r17553] Description:
Bring general fixes/improvements from file_free_space branch back to
trunk.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.8 (amazon) in debug mode
        Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-09-29 15:08:01 -05:00
Quincey Koziol
d591ac4bf7 [svn-r17403] Description:
Eliminate 'closing' flag to dataset flush routine, since all cached
dataset information has already been flushed when a file is closed and the
datasets themselves will be closed (and the dataset caches destroyed).

    Skip calling more flush routines when the file is closing, in preparation
for eventual separation of the 'flush' concept from the 'shutdown the cache'
concept.

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.8 (amazon) in debug mode
    Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-08-20 23:54:31 -05:00
Quincey Koziol
055d727f3f [svn-r17402] Description:
Switch from using a 'flags' parameter to the flush routine(s), to just
using a single 'closing' boolean parameter, since that's all we're doing with
the flags anyway and this makes things more obvious.

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.8 (amazon) in debug mode
    Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-08-20 23:26:15 -05:00
Quincey Koziol
5ddff83f67 [svn-r17284] Description:
Remove unneccessary dest. layout information from chunked dataset copying
routine.

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.7 (amazon) in debug mode
    Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-07-30 23:31:47 -05:00
Quincey Koziol
0eb775f358 [svn-r17281] Description:
Further refactoring of the dataset layout information, to separate the
storage information from the layout info.

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.7 (amazon) in debug mode
    Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-07-30 22:08:12 -05:00
Quincey Koziol
e1ceea54ff [svn-r17275] Description:
Refactor dataet storage information further, moving the chunk operations
into the storage struct and also fine-tune the chunk index structure for index
callbacks.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.7 (amazon) in debug mode
        Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-07-30 12:51:49 -05:00
Quincey Koziol
ccc0d85878 [svn-r17269] Description:
Refactor how serialized size of layout message is computed, slightly.

Tested on:
    Mac OS X/32 10.5.7 (amazon) debug & production
    FreeBSD/32 6.3 (duty) debug
    Too minor to require h5committest
2009-07-29 21:34:54 -05:00
Quincey Koziol
6909bc4319 [svn-r17243] Description:
Refactor how the layout message is created/written/read in the
dataset's object header.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.7 (amazon) in debug mode
        Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-07-27 17:11:29 -05:00
Quincey Koziol
405ea51bb7 [svn-r17230] Description:
Various minor tuneups noticed in working on the revise_chunks branch.

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.7 (amazon) in debug mode
    Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-07-23 16:46:20 -05:00
Quincey Koziol
c243e1e134 [svn-r17148] Description:
Refactor how chunked dataset information is computed, moving it earlier and
avoiding more recomputation.

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.7 (amazon) in debug mode
    Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-07-02 17:34:20 -05:00
Quincey Koziol
47e4e67e3c [svn-r17146] Description:
Bring a bunch of misc. improvements & tweaks from the revise_chunks
branch back to the trunk, so that future merges won't be so painful.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.7 (amazon) in debug mode
        Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-07-02 16:10:35 -05:00
Quincey Koziol
100ff923d9 [svn-r16973] Description:
Refactor chunk cache entry information to remove some [actually] unused
fields.

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.6 (amazon) in debug mode
    Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-05-21 13:46:33 -05:00
Quincey Koziol
d7d8cf1159 [svn-r16851] Description:
Clean up compiler warnings

Tested on:
    Mac OS X/32 10.5.6 (amazon)
    (too minor to require h5committest)
2009-04-23 17:55:18 -05:00
Quincey Koziol
92b11988c8 [svn-r16532] Description:
Pass the chunk "user data" to H5D_chunk_unlock(), so that chunks with
an address already aren't reallocated.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2009-03-02 15:57:18 -05:00
Neil Fortner
ef425724f7 [svn-r16523] Purpose: Fix bugs related to H5Dset_extent and fill values
Description:
In some situations it was possible for the fill value to not be written to parts
of a chunked dataset, particularly when extending and/or shrinking.  Prior to
the fix for the chunk cache (1015) these bugs would have been exceedingly rare.

Tested: jam, smirom, linew (h5committest)
2009-02-27 12:01:00 -05:00
Neil Fortner
0bdedf0a39 [svn-r16480] Purpose: Improve chunk cache
Description:
The meaning of the "nbytes" field in H5D_rdcc_t was not clear, and some places
assumed it was the maximum size of the chunk cache, while some assumed it was
the current size of the chunk cache.  The end result was that only 1 chunk could
be held in cache at a time.  This field has been replaced by "nbytes_max" and
"nbytes_used".  Performance of cached I/O should improve greatly.

Tested: jam, smirom (h5committest)
2009-02-12 15:46:32 -05:00
Raymond Lu
725b4d7ddb [svn-r16465] Performance Improvement(bug #1450). When a chunk is bigger than the cache size and isn't allocated
on disk, the library still loaded it in the cache, which is redundant.  I changed it to bypass the
cache and added a test in dsets.c.

Tested on jam and smirom.
2009-02-11 13:49:17 -05:00
Quincey Koziol
5cd054dd5c [svn-r16354] Description:
Bring r16353 back from revise_chunks branch:

        Refactor internal layout information, making it easier to add another
type of chunk index.

Tested on:
	FreeBSD/32 6.3 (duty)
	(other platforms tested with original patch)
2009-01-27 09:00:07 -05:00
Quincey Koziol
40dac44efe [svn-r16182] Description:
Rename internal routines, variables, macros, typedefs, etc. for chunked
dataset storage from "istore" to some variant of "chunk" or "btree".

Tested on:
        Mac OS X/32 10.5.5 (amazon) in debug mode
        Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
2008-12-11 14:29:44 -05:00
Neil Fortner
fd2285f7bc [svn-r16062] Purpose: Allow per-dataset rdcc configuration
Description: Adds 3 new public functions: H5Pset_chunk_cache,
H5Pget_chunk_cache, and H5Dget_access_list.  First implementation of dataset
access properties.  See RFC_chunk_cache_functions for details.

Tested: kagiso, linew, smirom (h5committest)
2008-11-12 13:07:51 -05:00
Quincey Koziol
9f60f016ab [svn-r15800] Description:
Bring file free space branch changes through r15795 into trunk, which
includes a fair bit of code cleanup & rearrangement along with a couple of
bug fixes also.

Tested on:
        Mac OS X/32 10.5.5 (amazon) in debug mode
        Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
2008-10-06 23:17:35 -05:00
Neil Fortner
5bbc0eedff [svn-r15682] Purpose: fix bug 1298
Description: The optimized "subset" compound conversion function would
improperly copy the "blank" space at the end of compound types.  Modified
H5T_conv_struct_init to detect when the subset type has extra space at the end,
and calculate the size of the data that should be copied into the conversion
buffer for each element.  Changes to the functions that implement these
conversions.

Tested: kagiso, linew, smirom (h5committest)
2008-09-22 18:41:31 -05:00
Quincey Koziol
bdd7d59902 [svn-r15628] Description:
Remove trailing whitespace from C/C++ source files, with the following
script:

    foreach f (*.[ch] *.cpp)
        sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
    end

Tested on:
    Mac OS X/32 10.5.5 (amazon)
    No need for h5committest, just whitespace changes...
2008-09-16 10:52:51 -05:00
Quincey Koziol
c465e18a68 [svn-r15609] Description:
Omnibus compiler warning cleanup & some reformatting also.

Tested on:
    Mac OS X/32 10.5.4 (amazon)
    Too minor to require h5committest
2008-09-10 14:10:41 -05:00
Quincey Koziol
771bae8888 [svn-r15131] Description:
Finish omnibus chunked dataset I/O refactoring, to separate general
actions on chunked datasets from actions that are specific to using the v1
B-tree index.

	Cleaned up a few bugs and added some additional tests also.

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.5.2 (amazon) in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2008-06-03 14:44:12 -05:00
Quincey Koziol
22f48585bd [svn-r15015] Description:
Detect chunks that are >4GB before dataset gets created and return error
to application.

    Tweak lots of internal variables that hold the chunk size/dimensions to
use an 'uint32_t', instead of a 'size_t', so that the integer size is constant.

    Correct a number of our tests which were creating datasets with chunks
that were >4GB and add some specific tests for >4GB chunk size detection.

    Minor whitespace & other code cleanups.

Tested on:
    Mac OS X/32 10.5.2 (amazon)
    Forthcoming testing on other platforms...
2008-05-15 22:04:56 -05:00
Scot Breitenfeld
aec106e324 [svn-r14903] Undoing change committed in r14902. 2008-04-30 14:51:13 -05:00
Scot Breitenfeld
5773fd34bc [svn-r14902] Merged fortran_1_8 branch changes r14505:14901 into the trunk. New fortran wrappers added. 2008-04-30 14:23:26 -05:00
Quincey Koziol
495ca9c7bb [svn-r14860] Description:
Omnibus raw data I/O revisions, with wide-ranging changes and
refactoring, in order to prepare for implementing "fast append" feature.

	These changes remove the majority of the code duplication for raw data
I/O which has crept in over the last ten years and introduces a more object-
oriented design for operating on different types of dataset storage.

	Chunked storage no longer has it's own I/O routines, it is now handled
as either contiguous (if chunk is not pulled into the cache) or compact (if the
chunk is cached in memory).

	No bug or feature changes, at least intentionally... :-)

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.5.2 (amazon) in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2008-04-24 10:03:41 -05:00
Quincey Koziol
66addd2d2c [svn-r14334] Description:
- Avoid trying to update 1-D dataset's chunk indices (they can't change)
    - Cache a copy of a dataspace describing a chunk, when doing single
        element I/O
    - Keep a 'chunk info' struct around, for single element I/O
    - Avoid creating a skip list for chunk infos when performing I/O on single
        element

	Also, minor formatting cleanups to testing code

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.4.10 (amazon) in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2007-12-10 16:38:03 -05:00
Quincey Koziol
74c005fdb2 [svn-r14326] Description:
- Keep skip list for tracking chunks with dataset (instead of creating/
        destroying it for each I/O operation) and just delete the skip list
        nodes.
    - Avoid computations for normalizing selection offset when offset not set.
    - Avoid updating object modification time twice during dataset creation.
    - Avoid updating dataset layout message (and object modification time)
        until dataset is closed.

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.4.10 (amazon) in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2007-12-06 14:24:30 -05:00
Quincey Koziol
6028cd41b8 [svn-r14295] Description:
- Eliminate some redundant calls to retrieve datatype sizes in chunk read/
        write routines.
    - Change indexed storage "common" B-tree callback user data to avoid
        copying chunk offset and separate "downward" info from "upward" info.
    - Cache chunk info (nbytes/filter_mask/address) for last chunk accessed

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.4.10 (amazon) in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2007-11-27 16:19:42 -05:00
Quincey Koziol
3aa3af4fb6 [svn-r14278] Description:
- Remember # of elements in file selection, instead of querying more than
        once
    - Handle building chunk map for single element as special case

Tested on:
        FreeBSD/32 6.2 (duty) in debug mode
        FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Mac OS X/32 10.4.10 (amazon) in debug mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
2007-11-20 17:12:52 -05:00