Commit Graph

53 Commits

Author SHA1 Message Date
Quincey Koziol
e99906f517 [svn-r17268] Description:
Bring r17266 from 1.8 branch to trunk:

    Clean up various compiler warnings from IRIX64 build

Tested on:
    Mac OS X/32 (amazon) debug & production
    (yes, I know it's not an IRIX64 system :-)
    Too minor to require h5committest
2009-07-29 20:59:24 -05:00
Quincey Koziol
f824891a84 [svn-r16982] Description:
Add library private routines to allow metadata cache entries to be set as
flush dependency children on extensible array entries.  This will allow chunk
proxies to be set as child flush dependencies for the extensible array, adding
another necessary layer to the flush dependency chain for SWMR access.

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.6 (amazon) in debug mode
    Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-05-26 17:42:12 -05:00
Quincey Koziol
4d6fcb288a [svn-r16874] Description:
Add test for cache client 'notify' callback.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.6 (amazon) in debug mode
        Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-04-28 14:30:05 -05:00
Quincey Koziol
6283a0c823 [svn-r16632] Description:
Remove unnecessary 'udata' parameter from H5AC_set() and 'notify' callback
(introduced with checkin that added the 'notify' callback)

Tested on:
    FreeBSD/32 6.3 (duty) in debug mode
    FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
    Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
        in debug mode
    Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
        w/C++ & FORTRAN, in production mode
    Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
        w/szip filter, in production mode
    Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
        in production mode
    Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
    Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
    Mac OS X/32 10.5.6 (amazon) in debug mode
    Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
        in production mode
2009-03-31 09:58:02 -05:00
Quincey Koziol
0ac0358a80 [svn-r16630] Description:
Add 'notify' callback to metadata cache clients, so that they can be
aware when the cache has taken/will take certain actions.  Notifications are
urrently limited to when an entry has successfully entered the cache (via an
insertion or a load from the file) and when an entry is about to be evicted
from the cache.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.6 (amazon) in debug mode
        Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-03-31 08:20:42 -05:00
Quincey Koziol
f8fa1029b7 [svn-r16625] Description:
Add mechanism for querying if an entry in the metadata cache is the
parent or child in a flush dependency relationship.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.6 (amazon) in debug mode
        Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-03-30 12:45:30 -05:00
Quincey Koziol
bbe430429d [svn-r16618] Description:
Modify metadata cache flush dependency feature to allow it to work
with entries that are pinned through the cache API calls.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.6 (amazon) in debug mode
        Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-03-26 13:16:54 -05:00
Quincey Koziol
667b34c713 [svn-r16600] Description:
Ensure that metadata cache entries with flush dependencies are written
to the file in the correct order (for serial I/O, at least).

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.6 (amazon) in debug mode
        Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-03-24 12:52:17 -05:00
Quincey Koziol
cbd0928d89 [svn-r16587] Description:
Add infrastructure & tests for "flush dependencies" in metadata cache,
which allow relationships that specify which order to flush metadata entries in.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.6 (amazon) in debug mode
        Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-03-17 12:08:12 -05:00
Quincey Koziol
c6b03d804e [svn-r16576] Description:
Actually set the exit value for the test (*sigh*)
2009-03-12 15:14:15 -05:00
Quincey Koziol
56a02987bc [svn-r16574] Description:
Clean up many compiler warnings and make test return non-zero exit code
when a failure is detected.

Tested on:
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
        Mac OS X/32 10.5.6 (amazon) in debug mode
        Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
2009-03-12 13:08:20 -05:00
Quincey Koziol
b4ff3e6e79 [svn-r16560] Description:
Remove another call to H5E_clear_stack() from within the library.

    Clean up lots of compiler warnings.

Tested on:
    Mac OS X/32 10.5.6 (amazon)
    (followup on other platforms forthcoming)
2009-03-10 14:00:39 -05:00
Mike McGreevy
a665dd22af [svn-r16338] Purpose:
Bug Fix

Description:

    Removing some unnecessary system headers from test/cache.c which snuck
    their way into one of my previous check-ins ... thanks to Allen for 
    catching this while testing on Windows.

Tested:

    Windows, Linux
2009-01-22 14:56:31 -05:00
Mike McGreevy
041e7dbfed [svn-r16290] Purpose:
Adding code to maintain a min_clean_fraction in the cache in serial mode.

Description:

    The metadata cache now has the ability to maintain a min_clean_fraction
    when in serial mode. The default initial cache size has been changed
    from 1MB to 2MB, and the default min_clean_fraction has been set at 30%.

    This check-in includes modifications to H5C.c to support maintaining a
    min_clean_size, including the addition of clean_index_size and 
    dirty_index_size trackers, modifications to the H5C_make_space_in_cache 
    algorithm, as well as associated test code and additional statistics 
    tracking variables.

    Maintaining the min_clean_fraction addresses the possibility of
    experiencing a "metadata blizzard" when the cache gets completely
    full with dirty entries. Upon having to make space, the cache would
    previously need to flush every single entry in the cache before coming
    across a clean entry which could be evicted. This resulted in unnecessary
    flushing of oftentimes hot entries in the cache. Maintaining the
    min_clean_fraction ensures that, when space is needed, clean entries
    are more readily available to evict.

Tested:

    jam, smirom, linew (h5committest)
2009-01-09 14:53:30 -05:00
Quincey Koziol
9f60f016ab [svn-r15800] Description:
Bring file free space branch changes through r15795 into trunk, which
includes a fair bit of code cleanup & rearrangement along with a couple of
bug fixes also.

Tested on:
        Mac OS X/32 10.5.5 (amazon) in debug mode
        Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
                                in production mode
        FreeBSD/32 6.3 (duty) in debug mode
        FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
        Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
                                in debug mode
        Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
                                w/C++ & FORTRAN, in production mode
        Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
                                w/szip filter, in production mode
        Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
                                in production mode
        Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
        Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
2008-10-06 23:17:35 -05:00
Quincey Koziol
bdd7d59902 [svn-r15628] Description:
Remove trailing whitespace from C/C++ source files, with the following
script:

    foreach f (*.[ch] *.cpp)
        sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
    end

Tested on:
    Mac OS X/32 10.5.5 (amazon)
    No need for h5committest, just whitespace changes...
2008-09-16 10:52:51 -05:00
John Mainzer
d236e2a992 [svn-r14442] Modified metadata cache to adapt more quickly to the insertion of a
large entry, or to a large increases in the size of an existing entry.
This required some additions to the cache configuration structure, and
thus will require changes in the metadata cache documentation.

The basic idea is to monitor the size of entries as they are loaded,
inserted, or increased in size.  If the size of the entry (or increase)
exceeds some user selected fraction of the size of the cache, increase
the size of the cache.

Note that this fix was designed quickly -- while it deals with the 
use case that exposed the problem, we may have to revisit the issue
later.

Tested serial and parallel on Phoenix, and h5committest.
2008-01-18 12:52:12 -05:00
John Mainzer
b742150e4c [svn-r14064] Implemented function allowing the user to cork the metadata cache.
Also implemented associated test code.
2007-08-09 23:19:43 -05:00
John Mainzer
30b6c4bcc1 [svn-r13697] Fixed error in check_unprotect_ro_dirty_err() in cache.c. Basic problem
was a failure to clean up properly after a test.

I reproduced the failure on Phoenix before fixing it.  As the error
was relatively straight forward, I tested fix on Phoenix only.
2007-04-24 05:51:12 -05:00
Quincey Koziol
9ef709fc11 [svn-r13691] Description:
Comment out cache test until John can take a look at it.

Tested on:
	FreeBSD/32 6.2 (duty)
2007-04-20 22:55:12 -05:00
Quincey Koziol
aa96180fa9 [svn-r13690] Description:
Fully enable readers/writer locking in metadata cache and correct errors
in cache clients which were modifying a cache entry while only holding a
read lock on it.

Tested on:
    Mac OS X/32 10.4.9 (amazon)
    Linux/32 2.6 (chicago)
    Linux/64 2.6 (chicago2)
    FreeBSD/32 6.2 (duty)
    FreeBSD/64 6.2 (liberty)
2007-04-19 13:43:10 -05:00
Quincey Koziol
4e3029bf82 [svn-r13634] Description:
Remove a "//" comment from John's recent changes and clean up a few
warnings.

Tested on:
    Mac OS X/32 10.4.9 (amazon)
2007-04-11 10:26:43 -05:00
John Mainzer
030543bf0d [svn-r13618] Modified metadata cache to support multiple read only protects of
cache entries.  Added test code to test the new feature.  Also some 
minor cleanum

h5committested -- passed on copper and sol.  Failed on osage with 
a configuration error that appears unrelated to my changes.  Serial
test (debug mode) passes on Phoenix (Linux x86 2.6 kernel), so I 
went ahead with the checkin.
2007-04-09 13:58:42 -05:00
Albert Cheng
ed7d456e51 [svn-r13253] Updated all C and C++ style source code files with the THG copyright notice.
Tested platform:
Kagiso only since it is only a comment block change.  If it works in one
machine, it should work in all, I hope.  Still need to check the parallel
build on copper.
2007-02-07 09:56:24 -05:00
Quincey Koziol
bcda3c2dbf [svn-r12833] Description:
Clean up more warnings...

Tested on:
    Linux/32 2.6 (chicago)
    Linux/64 2.6 (chicago2)
2006-10-30 19:13:27 -05:00
John Mainzer
2d6bb59329 [svn-r12774] Modified flush code in the metadata cache to allow it to
handle flush callbacks which can dirty other entries, and
resize and/or rename the target entry.

This feature is needed by the fractal heap code.

Also added associated test code.


H5Commit tested.  Test failed on heping, but the 
error appears to be a syntax error in an un-related
file.

Tests on copper & sol passed, along with tests on 
phoenix.
2006-10-18 05:51:26 -05:00
John Mainzer
e8c1fdd554 [svn-r12595] Modified H5C_insert_entry() to accept the H5C__PIN_ENTRY_FLAG entry
flag, and pin an entry as it is inserted.

The objective is to avoid some function call overhead in fheap.

Also added matching test code in test/cache.c & test/cache_common.c
(also testpar/t_cache.c checked in separately by accident)

h5commit tested
2006-08-17 17:04:47 -05:00
John Mainzer
801b5b09f2 [svn-r12462] Committed a variety of metadata cache related changes:
1) Added trace file support to the metadata cache.  This allows capture
   of all metadata cache calls in trace files for purposes of optimization
   and debuging.

2) Added an expunge entry function.  This allows an entry to be deleteded
   from the cache without writing it to disk even if it is dirty.

3) Added a function call to resize pinned entries.

4) Added code to deal with entries that are dirty on load.  This is
   needed in support of a bug fix which can alter object headers on
   load to repair files.

5) Added progress reporting code to the "MDC API smoke check" test in
   cache_api.c.  To enable the progress reporting, set report_progress
   to TRUE in mdc_api_call_smoke_check().

Tested with h5committest, and a parallel test on phoenix (dual athelon
linux box).
2006-07-13 12:59:39 -05:00
Quincey Koziol
7be3afb278 [svn-r12440] Purpose:
Code cleanup

Description:
    Trim trailing whitespace in Makefile.am and C/C++ source files to make
diffing changes easier.

Platforms tested:
    None necessary, whitespace only change
2006-06-27 09:45:06 -05:00
John Mainzer
a415cc6b17 [svn-r12374] Purpose:
1) Check in potential fix to unreproduceable bug in t_cache observed on
   Cobalt

2) Check in code supporting added pinned entry features in the metadata
   cache.


Description:

1) Elena encountered a bug in t_cache when running on Cobalt.  On
   investigation I found a potential race condition in t_cache that
   could explain her observation.

2) Quincey requested additions to the metadata cache allowing renaming
   of pinned entryies, and marking of pinned or protected entries as
   dirty.


Solution:

1) Modified t_cache.c to prevent the potential race condition.  Elena
   was unable to reproduce the bug on Cobalt, so we don't know if my
   fix actually addressed the problem.

2) Added code supporting the requested functions.  The changes were
   relatively trivial, but required substantial new test code.


Platforms tested:

h5committest, serial test on heping, parallel test on phoenix.


Misc. update:
2006-05-24 02:36:28 -05:00
John Mainzer
720f6cb065 [svn-r12321] Purpose:
Bug fix


Description:

Fix two test bugs:

1) Unitialized variable bug in test of H5C_get_entry_status()
   (in test/cache.c)

2) Incorrect initialization of constant in t_cache when running
   with MPE. (in testpar/t_cache.c)


Solution:

1) Don't test uninitialize variable.

2) Correct initialzation of constant in MPE case.


Platforms tested:

Tested on Heping and Copper.


Misc. update:
2006-05-02 01:06:38 -05:00
Quincey Koziol
eb96132022 [svn-r12316] Purpose:
Code cleanup/feature twist

Description:
    Adjust recent H5AC routines to take H5F_t pointers instead of H5C_t
pointers, to match the rest of the H5AC routines.

    This change propagated into a few of the tests, which also had some
compiler warnings cleaned up...

Platforms tested:
    FreeBSD 4.11 (sleipnir) w/parallel
    Linux 2.4/64 (mir) w/C++ & FORTRAN
2006-04-29 14:28:23 -05:00
John Mainzer
8a7b9b3221 [svn-r12311] Purpose:
Add pinned entry capability to cache.


Description:

For frequently accessed cache entries, the protect/unprotect overhead
is sometimes a bottleneck.


Solution:

Allow entries to be pinned in the cache.  Pinned entries can't be
evicted, but can be flushed or modified.


Platforms tested:

h5committested -- minus one small typo in test/cache.c whose fix was
                  tested on copper and heping only.


Misc. update:
2006-04-28 08:27:54 -05:00
John Mainzer
dfb1f40cbb [svn-r11897] Purpose:
1) Add parallel test for metadata cache

2) Split serial test for metadata cache into two parts

3) Fix bug in which cache was flushed needlessly when the
   cache wasn't full.

4) Performance improvements

5) Update API for parallel cache coherency bug fix.

Description:

See above.

Solution:

See above.

Platforms tested:

h5committest


Misc. update:
2006-01-27 17:58:48 -05:00
Quincey Koziol
56e3f667d6 [svn-r11886] Purpose:
Code cleanup

Description:
    Check in some of the code cleanups from working on the external link
support.  (This doesn't include any of the external link features)

Platforms tested:
    FreeBSD 4.11 (sleipnir)
    Mac OSX.4 (amazon)
    Linux 2.4
2006-01-23 15:46:34 -05:00
John Mainzer
c100b0bf26 [svn-r11470] Purpose:
Repair synchronization bug in the metadata cache in PHDF5

Also repair numerous other bugs that surfaced in testing the
bug fix.


Description:

While operations modifying metadata must be collective, we allow
independant reads.  This allows metadata caches on different processes
to adjust to different sizes, and to place the entries on their dirty
lists in different orders.  Since only process 0 actually writes
metadata to disk (all other processes thought they did, but the writes
were discarded on the theory that they had to be collective), this made
it possible for another process to modify metadata, flush it, and then
read it back in in its original form (pre-modification) form.  The
possibilities for file corruption should be obvious.


Solution:

Make the policy that only process 0 can write to file explicit, and
visible to the metadata caches.  Thus only process 0 may flush dirty
entries -- all other caches must retain dirty entries until they are
informed by process 0 that the entries are clean.

Synchronization is handled by counting the bytes of dirty cache entries
created, and then synching up between the caches whenever the sum
exceeds an (eventually user specified) limit.  Dirty metadata creation
is consistent across all processes because all operations modifying
metadata must be collective.

This change uncovered may bugs which are repaired in this checkin.
It also required modification of H5HL and H5O to allocate file space
on insertion rather than on flush from cache.


Platforms tested:

H5committest, heping(parallel & serial)

Misc. update:
2005-09-27 00:20:11 -05:00
Quincey Koziol
6b45f5172c [svn-r11245] Purpose:
Code cleanup

Description:
    Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.

Solution:
    Ran this script in each directory:

foreach f (*.[ch] *.cpp)
    sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end


Platforms tested:
    FreeBSD 4.11 (sleipnir)
    Too minor to require h5committest
2005-08-13 15:53:35 -05:00
Quincey Koziol
ddf9e4a4af [svn-r11014] Purpose:
Code cleanup

Description:
    Refactor metadata cache to merge "dirtied" flag in with other flags for
H5AC_unprotect and H5C_unprotect.

Platforms tested:
    FreeBSD 4.11 (sleipnir)
    h5committest
2005-07-02 11:36:15 -05:00
John Mainzer
7f8e346026 [svn-r10978] Purpose:
Interim checkin of code changes moving management of the is_dirty flag
into the cache code.


Description:

   Prior to this checkin, management of the is_dirty flag was handled
above the level of the metadata cache.  This can no longer be allowed,
as it introduces a race condition in the proposed fix for a cache
coherency bug in PHDF5.


Solution:

   Move management fo the is_dirty flag to the cache code proper.
Entries are now marked as dirty via a flag on the unprotect call.


Platforms tested:

h5committested


Misc. update:
2005-06-24 01:30:29 -05:00
John Mainzer
f4da01545f [svn-r10834] Purpose:
Fix a compile issue with the SX6 port.


Description:

In cache.c, invalid_configs array was declared as const, which caused
problems when elements of the array were passed to H5Pset_mdc_config()


Solution:

Remove the const modifier from the declaration of invalid_configs.


Platforms tested:

Heping (serial)


Misc. update:
2005-05-31 19:47:40 -05:00
John Mainzer
724cbc5615 [svn-r10717] Purpose:
Remove C99 types from new metadata cache related API calls
(take 2 -- missed one in the previous check in)


Description:

Windows (and perhaps others) don't like int32_t and int64_t.  While
we have dealt with the issue internally, it is more of a problem
in API calls.


Solution:

Convert int32_t to int and int64_t to long int in the new metadata
cache related API calls.


Platforms tested:

heping


Misc. update:
2005-05-02 16:36:43 -05:00
John Mainzer
cc44e6af63 [svn-r10715] Purpose:
Remove C99 types from new metadata cache related API calls


Description:

Windows (and perhaps others) don't like int32_t and int64_t.  While
we have dealt with the issue internally, it is more of a problem
in API calls.


Solution:

Convert int32_t to int and int64_t to long int in the
H5AC_cache_config_t structure used by the new metadata cache
related API calls.  Added explicit type casts to convert
between internal and external representations.


Platforms tested:

h5committested


Misc. update:
2005-05-02 13:58:28 -05:00
John Mainzer
a9ba92287d [svn-r10688] Purpose:
Add API calls allowing user control of the metadata cache.


Description:

Prior to this update, the metadata cache was not configurable
from outside the library.


Solution:

Add API calls allowing the user to configure the metadata cache
either at file open time, or for any open file.  Also added calls
permitting the user to monitor cache size and hit rate.  These
latter facilities are needed for "manual" cache size control


Platforms tested:

h5committested


Misc. update:
2005-04-28 11:40:07 -05:00
John Mainzer
866b37b337 [svn-r9868] Purpose:
Reduce the run time of the cache tests.


Description:

Some of the cache tests run for quite a while, which has been
causing problems with our daily build and test.  In my last
modification to cache.c, I added code to skip these stress
tests unless we were building in production mode.  Since
things go faster in production mode, the extra tests are not
a major problem here.

However, this means that the stress tests are only run once
or twice a week.


Solution:

To try to deal with this, I modified the four longest tests
to throttle depending on whether NDEBUG is defined.  When
NDEBUG is not defined, runtime for the cache tests should
be about 1/5th its regular run time.  We will see if this
is enough of a reduction to avoid problems.

There is some doubt in my mind as to just how much good the
throttled tests do, but they should be better than nothing.
As mentioned before, the correct solution is to build some
proper, random test code.


Platforms tested:

h5committested
production and debug builds on heping.


Misc. update:
2005-01-24 16:02:48 -05:00
John Mainzer
3b90c189ca [svn-r9850] Purpose:
1) Provide facilities in cache to allow us to avoid a potential cache
   consistency bug in the parallel case.

2) Clean up a off by one sanity checking bug.

3) Turn off execution of long running tests in debug mode.


Description:

1) In the parallel case, all writes to metadata must be collective,
   but reads may not be.  In pricipal, this allows us to different
   contents in different caches.  This isn't a problem as long as the
   correct data is always on disk, but unless we can force certain
   writes immediately, that need not be the case.

2) & 3) should need no further explanation.


Solution:

1) Add code allowing us to mark cache entries, and then force
   these entries to be flushed at a later time.

   Note that to actually avoid the bug, we will have to modify
   existing code to use these new features.

2) & 3) should need no further explanation.


Platforms tested:

heping (serial debug and production)

committest (copper, sol, and heping).  test failed on heping in the
c++ portion of the build, but at Quincey's siggestion, I am proceeding
with the checkin.


Misc. update:
2005-01-20 17:40:37 -05:00
John Mainzer
7092659dfa [svn-r9790] Purpose:
Reduce run time of daily tests.


Description:

cache, the test program for the metadata cache has been taking a while
to execute.


Solution:

As a short term "fix", I have commented out all but one of the long
running test functions.  Of course that means that we aren't running
these tests at present.  I'm not sure that this is a good idea.


Platforms tested:

Serial on Heping.


Misc. update:
2005-01-10 12:41:36 -05:00
Quincey Koziol
7fae6be03c [svn-r9734] Purpose:
Code cleanup

Description:
    Convert chunk iteration code to use skip lists instead of threaded, balanced
binary trees.

Platforms tested:
    FreeBSD 4.10 (sleipnir) w/parallel & szip
    Too minor to require h5committest
2004-12-30 11:26:45 -05:00
Quincey Koziol
427ff7da28 [svn-r9727] Purpose:
Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-)

Description:
    Generally speaking, this is the "signed->unsigned" change to selections.
However, in the process of merging code back, things got stickier and stickier
until I ended up doing a big "sync the two branches up" operation.  So... I
brought back all the "infrastructure" fixes from the development branch to the
release branch (which I think were actually making some improvement in
performance) as well as fixed several bugs which had been fixed in one branch,
but not the other.

    I've also tagged the repository before making this checkin with the label
"before_signed_unsigned_changes".

Platforms tested:
    FreeBSD 4.10 (sleipnir) w/parallel & fphdf5
    FreeBSD 4.10 (sleipnir) w/threadsafe
    FreeBSD 4.10 (sleipnir) w/backward compatibility
    Solaris 2.7 (arabica) w/"purify options"
    Solaris 2.8 (sol) w/FORTRAN & C++
    AIX 5.x (copper) w/parallel & FORTRAN
    IRIX64 6.5 (modi4) w/FORTRAN
    Linux 2.4 (heping) w/FORTRAN & C++


Misc. update:
2004-12-29 09:26:20 -05:00
John Mainzer
5c415042a3 [svn-r9687] Purpose:
Modify the cache code (H5C) to support automatic cache resizing to
adapt to the work load at run time.


Description:

   Different applications require different sized caches to maintain
an acceptable hit rate.  This set of changes attempts to provide the
ability to adjust to circumstances automatically.


Solution:

   Added highly configurable code to allow the user to either set a
fixed cache size, or allow the cache to grow and shrink according to
conditions.

   If enabled, cache size increases are triggered when the hit rate
drops below a user specified threshold in a user specified interval.

   Cache size reductions (if enabled) are triggered when either the
hit rate exceeds some user specified threshold over a user specified
interval, when the cache contains "enough" entries that haven't been
accessed for a user specified interval, or some mix of the above.

   See the header comments on the H5C_auto_size_ctl_t structure in
H5Cprivate.h for further details.

   At present, the cache resize configuration options are not
accessible via the user API.  Must add this.


Platforms tested:

   h5committested, heping (serial), and copper (parallel)


Misc. update:
2004-12-17 20:30:34 -05:00
John Mainzer
5d05a022f9 [svn-r9023] *** empty log message *** 2004-08-05 13:14:21 -05:00