Commit Graph

2947 Commits

Author SHA1 Message Date
Allen Byrne
1dee0d4627 HDFFV-10143 Fix initial issues from review 2017-03-30 12:15:25 -05:00
Allen Byrne
bb80ae9854 HDFFV-10143 add APIs to manipulate plugin path table 2017-03-28 13:35:49 -05:00
mainzer
afd8f64c29 Omnibus checkin for several relatively minor modifications:
1) Added code test/page_buffer.c to verify that page buffering is
   disabled in parallel builds.

2) Added code to test/cache_image.c to verify correct interaction
   between evict on close and cache image -- in particular management
   of a file containing a cache image containing dirty metadata that
   has been opened R/O.  Also fix for the bug exposed.

3) Added code to testpar/t_cache_image.c to verify expected procedure
   for reading cache images, and also supporting stats collection code
   needed for the test.

4) Repair of an overactive sanity check in H5C__reconstruct_cache_contents().

5) Other minor tidies in passing.

Tested serial and parallel, debug and production on Jelly.
2017-03-28 11:21:40 -05:00
lrknox
b461f1818b Run all scripts using signal files between processes in their own directories to avoid accidental deletion of the signal files. 2017-03-21 22:58:41 -05:00
lrknox
42141478f2 Run swmr tests in their own subdirectory of test to avoid signal
file deletion by other tests.
2017-03-17 17:24:21 -05:00
lrknox
98567db192 output_filter.sh:
Comment added to address HDFFV-8270.  The sample ontput in the
        file's comments are not up-to-date with the scripts in the file
        that remove output unique to certain systems when running test
        scripts.  This output doesn't match expected output files for
        the tests, causing them to fail.  Ther output_filter.sh file
        removes such output.  Currently we don't have access to these
        systems to update the comments.

testflushrefresh.sh.in:
        flushrefresh and other SWMR tests have been failing
        occasionally but regularly since bringing them to the develop
        branch.  The logs and debugging point to unexpected deletion of
        signal files as the point of failure, and John Mainzer suggests
        that two such tests running simultaneously in the same directory
        may lead to the operating system deleting the other test's
        signal file. Running the flushrefresh test in its own directory
        seems to solve the problem.
2017-03-15 12:49:19 -05:00
Quincey Koziol
ecfdadfb1e Correct double sourcedir name on copied datafiles. 2017-03-14 20:58:21 -07:00
Quincey Koziol
0313cbb91d Final merge of page buffering branch to develop 2017-03-13 21:30:37 -07:00
Quincey Koziol
4fad103187 Bring changes to I/O parameters from page_buffering branch. 2017-03-13 08:21:28 -07:00
Quincey Koziol
0b78740ff4 Merge in reentrency changes to "make space in cache" from page_buffering branch. 2017-03-13 07:45:42 -07:00
Quincey Koziol
c4a36e0bb3 Minor cleanups and bring over "prefetched dirty" fixes for entries loaded from
a cache image.
2017-03-12 01:57:19 -08:00
Allen Byrne
71fa3dc480 Fix alignment 2017-03-08 11:51:17 -06:00
Allen Byrne
12b87f0f84 Fix vds tests
java wrong call
vds test - missing checks and if content
2017-03-07 10:06:38 -06:00
Allen Byrne
0df90f9f79 Merge pull request #321 in HDFFV/hdf5 from ~BYRN/hdf5_adb:develop to develop
* commit 'f4dd30f41ccee7bcd97fba065573c5201cbcc743':
  Update CMake formatting
  Update CMake files and formatting
  HDFFV-10120 verify nozlib checking with tools tests
  cmake non-code typo
2017-03-02 12:11:19 -06:00
Quincey Koziol
a04bf9bf7f Merge pull request #319 in HDFFV/hdf5 from merge_page_buffering_07 to develop
* commit '5df52404bc6f6323ebf064b3d82b8e683de96eba':
  Misc. small cleanups to sync against incoming page buffering changes.
2017-03-02 11:25:40 -06:00
Dana Robinson
acdd8463bf Added "flush" and "refresh" to testflushrefresh.sh error output so
it's easier to see where errors in the script occur.
2017-03-02 10:45:41 -05:00
Quincey Koziol
5df52404bc Misc. small cleanups to sync against incoming page buffering changes. 2017-03-02 06:24:46 -08:00
Allen Byrne
2562ec3a4b Update CMake files and formatting 2017-03-01 14:47:53 -06:00
Allen Byrne
cae5e0ad07 cmake non-code typo 2017-03-01 09:58:59 -06:00
Quincey Koziol
cf93261099 Correct issues from pull request reviews. 2017-02-28 08:41:57 -08:00
Quincey Koziol
5db402afbb Corrected version of 4b5e05c084:
Fix some more signed -> unsigned value issues with cache data structures,
    also misc. style cleanups.  All to align w/incoming page_buffering changes.
2017-02-27 21:45:55 -08:00
Quincey Koziol
e5fb4191de Revert "Fix some more signed -> unsigned value issues with cache data structures, also"
This reverts commit 4b5e05c084.
2017-02-27 21:23:44 -08:00
Quincey Koziol
4b5e05c084 Fix some more signed -> unsigned value issues with cache data structures, also
misc. style cleanups.  All to align w/incoming page_buffering changes.
2017-02-22 23:21:41 -08:00
Quincey Koziol
101fa7af8e Cleanups and normalizations against incoming page_buffering changes. 2017-02-19 18:28:52 -08:00
Allen Byrne
220cf7d326 Revert whitespace 2017-02-13 14:16:24 -06:00
Allen Byrne
960f694197 Close file before trying to remove it 2017-02-13 14:11:13 -06:00
Dana Robinson
d875f7f3e6 Minor warning fixes. 2017-02-09 11:44:38 -05:00
Dana Robinson
46ede7f039 Deleted an extraneous remove() call from a test function
that might be causing the flushrefresh test to fail.
2017-02-08 14:49:25 -05:00
Dana Robinson
bb354058c0 Added #ifdef QAK macros around log VFD calls in SWMR acceptance
tests.
2017-02-08 11:29:04 -05:00
Allen Byrne
c699bbc8bf HDFFV-10125 cache_common file already in test lib 2017-02-02 16:30:23 -06:00
Allen Byrne
2b383bb8ff HDFFV-10124 Using cache_common.h contains other include files 2017-02-02 11:55:01 -06:00
Allen Byrne
21d48c1d1f Fix Windows static only cache test checking 2017-01-31 09:26:44 -06:00
Quincey Koziol
a9808853d5 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into develop_merge_cache_image_06 2017-01-28 23:27:08 -08:00
Quincey Koziol
3ad18072eb Add missing files 2017-01-28 23:21:22 -08:00
Quincey Koziol
1949d5756d Cache image feature and testing 2017-01-28 23:19:37 -08:00
Quincey Koziol
5a776edb81 Bring change to use array of metadata cache entry classes for H5C_create
instead of array of class names from cache image branch.
2017-01-27 23:18:47 -08:00
Quincey Koziol
349f41d974 Switch list lengths to unsigned integers (to align better w/cache image merge) 2017-01-27 20:35:14 -08:00
Dana Robinson
081955fdd1 Additional minor changes from revise_chunks->develop. 2017-01-27 16:46:09 -05:00
Dana Robinson
2cac1c0637 Moved remaining SWMR-related test files to develop. 2017-01-26 14:34:12 -05:00
Allen Byrne
6b9a7cece4 Disable SHARED cache test on windows 2017-01-24 09:28:11 -06:00
Allen Byrne
556ff06975 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit 'bd85e57904f1f943774bb99ea5e8b0d074db0edb':
  tools/test/h5diff/testh5diff.sh.in:          Add code to delete copies of test .h5 files copied from          tools/testfiles/vds to tools/test/h5diff/testfiles when running          "make check" for an in-source build.
2017-01-13 10:57:26 -06:00
lrknox
8452976de5 tools/test/h5diff/testh5diff.sh.in:
Add code to delete copies of test .h5 files copied from
         tools/testfiles/vds to tools/test/h5diff/testfiles when running
         "make check" for an in-source build.

    tools/test/misc/testh5repart.sh.in:
         Change the name of the temporary test directory from "testfiles"
         which is also used for 2 h5mkgrp test files under source control to
         "testrepart" to avoid deleting source controlled files when running
         "make check" in the source directory.

    Fix test scripts that remove source-controlled files during "make check" when run in-source.
    Fix test scripts that don't remove test files because they add files to test directories but don't remove them if the build directories are the same as the source directory.

    Fix any test source file or Makefile.am files for tests that leave data files are removed by neither "make check" nor "make distclean".
2017-01-12 22:20:11 -06:00
Allen Byrne
1bca531a73 Remove duplicate command 2017-01-12 09:04:10 -06:00
Allen Byrne
244faaa0d0 HDFFV-10094: upgrade cmake command conventions
Also converted tests to use macros
2017-01-11 14:02:34 -06:00
Allen Byrne
e1bcae8a6a Update cmake min version and cmake command convention 2017-01-10 17:10:03 -06:00
Quincey Koziol
8d4c84eae2 Bring changes to metadata cache "get entry status" call and new
"child serialized / unserialized" messages and support from the cache image
branch.
2017-01-06 11:37:17 -08:00
Quincey Koziol
7568dcaf15 Add "image up to date" flag / parameter to metadata cache entry status calls. 2017-01-06 07:51:40 -08:00
Quincey Koziol
afd7e5b476 Bring file shutdown code from cache image branch 2017-01-04 21:36:14 -08:00
Quincey Koziol
e73a32b852 Remove 'const' from cache client pre_serialize callback, to reduce warnings,
and correspondingly remove 'const' from some internal routines.  Also rename
some H5MF* routines to reflect their static/package usage.
2017-01-03 08:28:05 -08:00
Dana Robinson
105b6f0f65 Merge pull request #227 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:flush_vfd_fix to develop
flush1/2 fix w/ non-SWMR VFDs + code overhaul

* commit 'fcb7e204b568efb785f47153172147b83da5418a':
  Updated the flush1/2 test to not be a mess and to handle non-SWMR-enabled VFDs.
2016-12-31 07:20:09 -06:00
Dana Robinson
fcb7e204b5 Updated the flush1/2 test to not be a mess and to handle
non-SWMR-enabled VFDs.
2016-12-30 14:41:37 -05:00
Dana Robinson
3908d10f93 Merge pull request #226 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:flush_vfd_fix to develop
Added a missing "vfd check" test file for use in scripts (should have been brought from revise_chunks).

* commit 'e41a15b7f07dc39b7d68f4635965d0b8b440d299':
  Added a missing test file needed for the flushrefresh script.
2016-12-30 10:37:58 -06:00
Dana Robinson
e41a15b7f0 Added a missing test file needed for the flushrefresh script. 2016-12-30 10:06:07 -05:00
Quincey Koziol
0610c41c75 Bring support for doubly-linked list of all entries in cache from incoming
cache image branch.
2016-12-30 02:22:48 -08:00
Quincey Koziol
da2f1610d1 More normalizations against incoming cache image branch. 2016-12-29 02:40:26 -08:00
Quincey Koziol
eefddf2813 Bring new/updated tests from revise+chunks branch: flush1/flush2 update and
flushrefresh test.  (Also refactor of message send / wait code)
2016-12-19 20:44:54 -08:00
Quincey Koziol
f8cfbee05e Merge SWMR-related testing to existing tests. 2016-12-18 02:19:26 -08:00
Quincey Koziol
fc482f126f Bring over tweak for missing environment variable. 2016-12-18 00:59:36 -08:00
Quincey Koziol
104253575d Update CMake configuration files with SWMR accumulator changes. 2016-12-18 00:58:46 -08:00
Quincey Koziol
f01ee3c004 Add missing accumulator test. 2016-12-18 00:42:15 -08:00
Quincey Koziol
b772be716d Merge SWMR-oriented accumulator tests from revise_chunks to develop. 2016-12-18 00:41:13 -08:00
Quincey Koziol
feb9d1f676 Bring over changes from revise_chunks that cleanup recent SWMR changes from
code review feedback.
2016-12-11 00:28:05 -08:00
Dana Robinson
cc6eb939f4 Merge pull request #191 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:eoc_h5fs_fix to develop
Final fix for evict-on-close issues with new-style groups.

* commit '02879bc4b2ba161dc259bf28a71e7a99b44be0e3':
  Updated the H5FS cache code to grab the correct tag and modified the freespace test to use dxpls that have been tagged with the H5AC__FREESPACE_TAG global tag instead of H5AC_ind_read_dxpl_id. The library code now expects the owner of the free space manager to tag it so the owner-less free space managers in the freespace tag had to be tagged with *something* to avoid cache errors.
  Changed the name of H5MF_alloc_close() to H5MF__alloc_close() to make the FUNC_ENTER macro happy.
  Reverted the freespace test. The TAG macros in H5MF_get_freespace() make the changes unnecessary.
  Added missing TAG macros to H5MF_get_freespace().
  Updated the freespace test so functions take a common, tagged dxpl so the test will work with the new H5FS tag changes.
  Updated the cache_tagging test so it correctly analyzes the new fractal heap free space manager behavior.
  Added TAG macros to the H5MF package code so H5AC__FREESPACE_TAG gets set on the dxpl correctly. Also restored the TAG macro to H5FS__cache_hdr_pre_serialize() since otherwise the tag won't be initialized.
  First cut at fixing the tagging issue with the free space managers. * Converted the tag macros to regular ones so the parent's tag   is used via the dxpl.
2016-12-10 08:09:40 -06:00
Dana Robinson
02879bc4b2 Updated the H5FS cache code to grab the correct tag and modified
the freespace test to use dxpls that have been tagged with
the H5AC__FREESPACE_TAG global tag instead of H5AC_ind_read_dxpl_id.
The library code now expects the owner of the free space manager
to tag it so the owner-less free space managers in the freespace
tag had to be tagged with *something* to avoid cache errors.
2016-12-07 21:16:12 -05:00
Dana Robinson
16a5c7c8f3 Merge branch 'develop' into eoc_h5fs_fix 2016-12-07 17:21:07 -05:00
Allen Byrne
fa6488d785 Reduce timeout as normal run time on windows is less then 10 min 2016-12-05 09:40:57 -06:00
Quincey Koziol
64a339183f Bring SWMR support in to the main development branch. (Finally!) More tests
and the tool and API wrappers will be coming in over the weekend.
2016-12-02 08:07:04 -08:00
Dana Robinson
3186fd93dc Reverted the freespace test. The TAG macros in H5MF_get_freespace()
make the changes unnecessary.
2016-11-30 03:26:34 -05:00
Dana Robinson
6744963c61 Updated the freespace test so functions take a common, tagged
dxpl so the test will work with the new H5FS tag changes.
2016-11-30 02:38:34 -05:00
Dana Robinson
916eaa4ab8 Updated the cache_tagging test so it correctly analyzes
the new fractal heap free space manager behavior.

Also converted the debug output of the cache_tagging
test to use the revised H5AC_dump_cache() function.
2016-11-30 02:20:09 -05:00
Dana Robinson
a046ac31d2 First cut at fixing the tagging issue with the free space managers.
* Converted the tag macros to regular ones so the parent's tag
  is used via the dxpl.

* Updated the tag sanity check logic so for free space managers.

* Turned new-style group checks back on in test/evict_on_close.c
2016-11-29 23:53:52 -05:00
Dana Robinson
63bcd73f1f Merge pull request #179 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:eoc_valgrind_bugfix to develop
Fix for some evict-on-close issues revealed by Valgrind.

* commit 'd3dbe9d90ee71b231b3264e1d62d3efbae3242ef':
  Tentative fix for valgrind issues related to EoC.
2016-11-29 20:22:47 -06:00
Dana Robinson
d3dbe9d90e Merge branch 'develop' into eoc_valgrind_bugfix 2016-11-29 15:06:37 -05:00
Dana Robinson
30bd7247d8 Merge branch 'develop' into expanded_eoc_test 2016-11-29 12:52:46 -05:00
Quincey Koziol
b092dbcdfd Bring over another batch (hopefully the last) of non-SWMR "normalization"
changes from the revise_chunks branch.
2016-11-28 16:34:36 -08:00
Dana Robinson
978e83fb5b Merge branch 'develop' into eoc_valgrind_bugfix 2016-11-28 13:01:10 -05:00
Dana Robinson
4ddc4a602f Merge pull request #171 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:develop_memfix to develop
Fix for tfile.c valgrind issues.

* commit '191cc19ae0a2feb72d2171a0808fdc71a3b862dd':
  Added vlen reclaim functions to a test in tfile.c that leaks memory.
2016-11-28 09:37:23 -06:00
Dana Robinson
d79c73320e Updated the evict-on-close test to create a large number of subgroups
in a containing group. This will ensure that many cache entries will
be used, making a better test of EoC functionality.
2016-11-27 23:42:08 -05:00
Dana Robinson
2c3d02e6e5 Merge branch 'develop' into eoc_valgrind_bugfix 2016-11-26 13:11:08 -05:00
Dana Robinson
a8d1aff235 Tentative fix for valgrind issues related to EoC.
Adds /*out*/ parameters to H5O_close() and H5F_try_close() so
that H5D/G_close() will know when H5O_close() has triggered a
file close and thus the file struct is not reliable.

Also removes the H5F_CLOSING() macro and related which were
formerly used to check if the file was closing.
2016-11-26 10:47:32 -05:00
Dana Robinson
191cc19ae0 Added vlen reclaim functions to a test in tfile.c
that leaks memory.
2016-11-23 16:45:47 -05:00
Dana Robinson
b0321f7e7d Merge pull request #170 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:develop_memfix to develop
Fixed a few memory issues in the tests.

* commit 'e2863ca880a6983558d3ba3e5219164a57287f04':
  Fixed a few memory problems in test/cache_logging.c and test/h5test.c (which show up in test/vfd.c).
2016-11-23 12:58:12 -06:00
Dana Robinson
e2863ca880 Fixed a few memory problems in test/cache_logging.c and
test/h5test.c (which show up in test/vfd.c).
2016-11-22 20:55:03 -05:00
Allen Byrne
c2e60909e3 Add test to memcheck ignore and increase cache test timeout 2016-11-22 15:04:34 -06:00
Quincey Koziol
3b0c2b24da Bring over support for retrying metadata cache entry loads, along with all the
supporting metadata cache callback changes, etc.
2016-11-20 04:24:57 -08:00
Quincey Koziol
eba0ff43c9 Remove 'clear' callback from metadata cache client interface, using the
new 'entry cleaned' notify action innstead.
2016-11-17 17:48:21 -08:00
Quincey Koziol
be27e1ad79 Remove explicit support within the metadata cache for tracked compressed
entries, they are now completely handled in the clients.
2016-11-17 12:58:04 -08:00
Quincey Koziol
d4591ff54d Bring over new 'notify' metadata cache client callback actions for when an
entry is cleaned / dirtied or its [flush dependency] child entry is cleaned /
dirtied.
2016-11-12 14:01:30 -08:00
Quincey Koziol
d183e9a1a2 Merge pull request #155 in HDFFV/hdf5 from ~KOZIOL/hdf5:develop_merge_revise_chunks_05 to develop
* commit '1c6924f18bd8fdad63c4f191c00605092c17fa6e':
  Refactor H5O code to clean up message allocation, align cache deserialize code with revise_chunks changes, and remove unused "message locking" code.
2016-11-12 13:01:58 -06:00
Quincey Koziol
1c6924f18b Refactor H5O code to clean up message allocation, align cache deserialize code
with revise_chunks changes, and remove unused "message locking" code.
2016-11-12 01:05:47 -08:00
Quincey Koziol
80ac4343cd Align w/minor cleanups in revise_chunks branch. 2016-11-11 23:14:58 -08:00
Quincey Koziol
9504d3f337 Switch to new, more scalable, metadata cache entry tagging. 2016-11-06 23:14:10 -08:00
Quincey Koziol
bb6cb6adb0 Code style and warning cleanups, from revise_chunks branch. 2016-11-06 12:08:19 -08:00
Quincey Koziol
0d12ce44b2 Add cache logging test. 2016-11-03 23:27:10 -07:00
Frank Willmore
b2948d9f97 Merge pull request #100 in HDFFV/hdf5 from ~FRANK.WILLMORE/hdf5:HDFFV-8882-replace-numeric-exit-code-with to develop
* commit '526bcc242d03a6fc4fd28d3a0f68cc31f4f2a850':
  Changes to be committed: 	modified:   bittests.c 	modified:   cmpd_dset.c 	modified:   dsets.c 	modified:   dt_arith.c 	modified:   dtypes.c 	modified:   extend.c 	modified:   fillval.c 	modified:   filter_fail.c 	modified:   flush1.c 	modified:   gen_cross.c 	modified:   hyperslab.c 	modified:   istore.c 	modified:   links.c 	modified:   links_env.c 	modified:   objcopy.c 	modified:   plugin.c 	modified:   tcheck_version.c 	modified:   unlink.c 	modified:   unregister.c
2016-11-03 16:28:29 -05:00
Quincey Koziol
7ac8215cbf Merge pull request #123 in HDFFV/hdf5 from ~KOZIOL/hdf5:develop to develop
* commit 'b2878dec04c120e4e3cdf00e943283e359862c84':
  Move cache debugging routines into separate module.
  Change file memory type for extending to default, and correct error in backward compatibility for multi VFD.
  Warning and whitespace cleanup.
  Improvements to the log VFD, for helping with SWMR debugging.
  Minor code cleanups.
  Uncomment line in test/objcopy.c, clean up whitespace and POSIX call wrapping in tools code.
2016-11-03 16:15:26 -05:00
Quincey Koziol
84787e3085 Uncomment line in test/objcopy.c, clean up whitespace and POSIX call wrapping
in tools code.
2016-11-03 08:46:53 -07:00
Dana Robinson
30f5f7d9f9 Removed commented-out lines that clean up test files. 2016-11-03 11:31:02 -04:00
Dana Robinson
c41321a9f7 Added a test for old- and new-style groups under evict-on-close. 2016-10-29 21:12:07 -04:00
Dana Robinson
de3abdf5d7 Evict-on-close now works with groups.
test/evict_on_close.c has been updated to generate
groups but does not yet check EoC works correctly.
2016-10-28 15:58:18 -04:00