Commit Graph

21142 Commits

Author SHA1 Message Date
mainzer
33f35183cb When flushing, the metadata cache attempts to flush entries in increasing
address order.  To facilitate this, the metadata cache needs a list of
of dirty entries in increasing address order.  This is implemented via a
skip list of all dirty entries in the cache.

To date this skip list has been maintained at all times.

However, profiling indicates that we can avoid significant overhead by
constructing the skip list of dirty entries just before a flush, taking
it down afterwareds, and not maintaining it during normal operation.

This commit implements this optimization for both serial and parallel.

Tested serial and parallel, debug and production on charis and jelly.
2020-08-05 15:39:49 -05:00
Dana Robinson
50f404c887 Merge pull request #2720 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:develop_minor to develop
* commit 'c8ba30d965f0342e5725a11cb562dfb22c8a8d24':
  Minor normalization with 1.10 branch
2020-07-30 20:16:17 -05:00
Dana Robinson
c8ba30d965 Merge branch 'develop' into develop_minor 2020-07-30 15:23:54 -07:00
Allen Byrne
88aa0d5c76 Merge pull request #2718 in HDFFV/hdf5 from ~BYRN/hdf5_adb:develop to develop
* commit 'ff4a9d0ec2e85cf15f769006e92e756a79869ab8':
  Warnings reduction in tools
2020-07-30 11:46:39 -05:00
Allen Byrne
ff4a9d0ec2 Warnings reduction in tools 2020-07-30 11:26:14 -05:00
Allen Byrne
2ece66e999 Merge pull request #2716 in HDFFV/hdf5 from ~BYRN/hdf5_adb:develop to develop
* commit '6a1cf090c6a2f970084c714a2f75a31d68a0a6a8':
  Correct reference file
  Correct h5diff filenames
2020-07-29 15:40:05 -05:00
Allen Byrne
6a1cf090c6 Correct reference file 2020-07-29 12:56:03 -05:00
Allen Byrne
1f7fa50137 Correct h5diff filenames 2020-07-29 08:24:30 -05:00
Binh-Minh Ribler
cf4e78b3c9 Merge pull request #2711 in HDFFV/hdf5 from bmr-HDFFV-11120 to develop
Fix HDFFV-11120 and HDFFV-11121 (CVE-2018-13870 and CVE-2018-13869)

* commit '707e30c6be1954c0027374124207e46caae68cbc':
  Fixed typos in error messages.
  Fix HDFFV-11120 and HDFFV-11121 (CVE-2018-13870 and CVE-2018-13869)
2020-07-28 19:08:19 -05:00
Binh-Minh Ribler
707e30c6be Fixed typos in error messages. 2020-07-28 12:20:13 -05:00
Dana Robinson
b201a78b33 Merge pull request #2714 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:h5s_set_extent_none to develop
* commit 'e4603ff8a171e82de6373ba6eddcf59d5eeab5aa':
  Updated the Java tests to deal with H5Sset_extent_none changes
  Fixed additional typos in tselect.c comments.
  Fixes typo in tselect.c
  Updates H5Sset_extent_none() to set H5S_NULL
2020-07-28 11:29:04 -05:00
Dana Robinson
e4603ff8a1 Updated the Java tests to deal with H5Sset_extent_none changes 2020-07-28 08:28:38 -07:00
Allen Byrne
10e59f6ba4 Merge pull request #2697 in HDFFV/hdf5 from ~BYRN/hdf5_adb:develop to develop
* commit '5857bb817019b8513f796c9564de3f1845f9a44a':
  Only one line needed to compare
  Revert err file
  revert error-stack enable
  Add release notes for tools
  add test file ext
  Correct file name in test
  Tools refactor
2020-07-28 07:13:18 -05:00
Dana Robinson
c78326eb92 Fixed additional typos in tselect.c comments. 2020-07-27 17:40:25 -07:00
Dana Robinson
dbba42c306 Fixes typo in tselect.c 2020-07-27 17:38:41 -07:00
Dana Robinson
91f5320ff0 Updates H5Sset_extent_none() to set H5S_NULL
The API call used to set the internal H5S_NO_CLASS value which
produced errors when such a dataspace was passed to many other API
calls.

Fixes HDFFV-11027
2020-07-27 17:32:01 -07:00
Allen Byrne
5857bb8170 Only one line needed to compare 2020-07-27 16:15:50 -05:00
Allen Byrne
799f2891c2 Revert err file 2020-07-27 15:47:10 -05:00
Dana Robinson
6e740457cc Minor normalization with 1.10 branch 2020-07-27 11:09:05 -07:00
Binh-Minh Ribler
303be6d7de Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into bmr-HDFFV-11120 2020-07-22 16:15:47 -05:00
Binh-Minh Ribler
88e458ac43 Fix HDFFV-11120 and HDFFV-11121 (CVE-2018-13870 and CVE-2018-13869)
Description:
    When a buffer overflow occurred because a name length was corrupted
    and became very large, h5dump produced a segfault on one file and a
    memcpy parameter overlap on another file.  This commit added checks
    that detect a read pass the end of the buffer to prevent these error
    conditions.
Platforms tested:
    Linux/64 (jelly)
    SunOS 5.11 (emu)
2020-07-22 16:13:26 -05:00
Allen Byrne
e0b86594c2 revert error-stack enable 2020-07-20 16:46:49 -05:00
Allen Byrne
c0bc9d0e60 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '74e09473c825610d203565e5ed9deb156256d6ea':
  Minor normalization with 1.10.
2020-07-20 14:08:55 -05:00
Dana Robinson
74e09473c8 Merge pull request #2703 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:develop_minor to develop
* commit '242459231b40a88741cbc85df4272914b5d8bed3':
  Minor normalization with 1.10.
2020-07-20 14:04:41 -05:00
Dana Robinson
242459231b Minor normalization with 1.10. 2020-07-20 11:50:10 -07:00
Allen Byrne
bae8107621 Add release notes for tools 2020-07-20 10:26:17 -05:00
Allen Byrne
a498b4ee79 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '9d3ea366d4bf3278e8260f4d5cdcc88cb4b61c56':
  Update MANIFEST. Add new test to Cmake.
  Fix for jira issue HDFFV-11080: (1) Patch up the file pointer when reading attribute of variable length datatype (2) Test to verify the fix when doing multiple threads
2020-07-17 14:39:25 -05:00
Vailin Choi
9d3ea366d4 Merge pull request #2693 in HDFFV/hdf5 from ~VCHOI/my_third_fork:bugfix/HDFFV-11080-heap-use-after-free-by-the-call to develop
* commit '3023b34272cc6ada84aebfa7441a6a55afa3621c':
  Update MANIFEST. Add new test to Cmake.
  Fix for jira issue HDFFV-11080: (1) Patch up the file pointer when reading attribute of variable length datatype (2) Test to verify the fix when doing multiple threads
2020-07-17 14:27:12 -05:00
Allen Byrne
a2f2887675 add test file ext 2020-07-17 13:22:26 -05:00
Allen Byrne
93c3d7b2f3 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit 'e36047b9b9839d1fd0b4009b7066b8b50b95bc6d':
  Minor normalization with 1.10
2020-07-17 10:40:49 -05:00
Dana Robinson
e36047b9b9 Merge pull request #2700 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:develop_minor to develop
* commit '6388a6750ff1dc476b8631d26cd1742730df9fe8':
  Minor normalization with 1.10
2020-07-17 10:23:24 -05:00
Dana Robinson
6388a6750f Minor normalization with 1.10 2020-07-17 05:17:48 -07:00
vchoi
3023b34272 Update MANIFEST.
Add new test to Cmake.
2020-07-16 16:40:11 -05:00
Allen Byrne
247e2dc546 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '58d37c2295044ad1a201d6671cdd67ae380b84f8':
  H5R: fix type conversion of references within vlen type
2020-07-16 08:57:28 -05:00
Allen Byrne
e5145030a4 Correct file name in test 2020-07-16 08:31:48 -05:00
Jerome Soumagne
58d37c2295 Merge pull request #2694 in HDFFV/hdf5 from ~JSOUMAGNE/hdf5:fix_vlen_ref to develop
* commit 'aa9d2bd9bcce7a1a53c9d9cf519df5401041bb74':
  H5R: fix type conversion of references within vlen type
2020-07-15 21:02:14 -05:00
Jerome Soumagne
aa9d2bd9bc H5R: fix type conversion of references within vlen type
Update trefer.c with vlen test case
2020-07-15 21:00:36 -05:00
Allen Byrne
bac0fc6309 Tools refactor
Move error_stack setup to tools library
Add structure for h5dump properties
Eliminate duplicated h5diff functions
Use properties structure in place of argument lists.
2020-07-15 11:20:16 -05:00
Allen Byrne
900e75544e Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '806f5b5b2a280072f62f58c48e93acdc84f2b3ae':
  Removed POSIX #defines from configure on Solaris.
  Fallback to older time functions in new timer code.
2020-07-14 12:02:43 -05:00
Dana Robinson
806f5b5b2a Merge pull request #2692 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:develop_minor to develop
* commit 'b8013380792a3503003f0a2bd60d7541656b03c7':
  Removed POSIX #defines from configure on Solaris.
  Fallback to older time functions in new timer code.
2020-07-14 11:46:33 -05:00
vchoi
4bc49d8b2b Fix for jira issue HDFFV-11080:
(1) Patch up the file pointer when reading attribute of variable length datatype
(2) Test to verify the fix when doing multiple threads
2020-07-14 10:56:33 -05:00
Dana Robinson
b801338079 Removed POSIX #defines from configure on Solaris. 2020-07-14 07:49:00 -05:00
Dana Robinson
4305d9677b Fallback to older time functions in new timer code. 2020-07-14 05:01:22 -07:00
Allen Byrne
9224c7abaa Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit 'b480d0a38b1e265c55c74262bf384d255a33e07b':
  Remove path to szip header file from AM_CPPFLAGS when configure check of libsz fails. Fix for HDFFV-10830.
  HDFFV-11116 Add option for install location of Fortran MOD files
2020-07-11 12:53:11 -05:00
Larry Knox
b480d0a38b Merge pull request #2686 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:develop to develop
* commit '800f93f7c994f88dfa716746153ded4b1e690e3a':
  Remove path to szip header file from AM_CPPFLAGS when configure check of libsz fails. Fix for HDFFV-10830.
2020-07-11 08:07:10 -05:00
Allen Byrne
e3f5f59a61 Merge pull request #2684 in HDFFV/hdf5 from ~BYRN/hdf5_adb:feature/HDFFV-11116-trilabs-mod-location to develop
* commit '07a52ecb2ffa1d8d32f06db23470833899db3deb':
  HDFFV-11116 Add option for install location of Fortran MOD files
2020-07-11 07:10:31 -05:00
Larry Knox
800f93f7c9 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~lrknox/hdf5_lrk into develop 2020-07-10 13:44:08 -05:00
Larry Knox
0f3fd8d53d Remove path to szip header file from AM_CPPFLAGS when configure check of libsz fails.
Fix for HDFFV-10830.
2020-07-10 13:39:59 -05:00
Allen Byrne
b63c9e81dd Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit 'f5da9b1befeb81aa9f4d97df3ec606f801807420':
  Minor changes from normalization with 1.10
2020-07-09 16:14:34 -05:00
Dana Robinson
f5da9b1bef Merge pull request #2683 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:develop_normalization to develop
* commit '127d1cfdf06e9c1cd264595c496a063848358a0a':
  Minor changes from normalization with 1.10
2020-07-09 15:50:43 -05:00