Commit Graph

3266 Commits

Author SHA1 Message Date
Larry Knox
8008294578 Squashed commit of the following:
Merge changes from update_merged_S3_HDFS branch into develop.

commit d5034315aea88629929ac0c9c59ebfafd5f21a31
Merge: 9c48823 d3fdcd8
Author: Larry Knox <lrknox@hdfgroup.org>
Date:   Thu Jul 25 08:24:53 2019 -0500

    Merge branch 'develop' into update_merged_S3_HDFS
2019-07-25 11:47:12 -05:00
Allen Byrne
6a932198aa HDFFV-10529 Update CMake tests to use test fixtures 2019-07-23 16:24:56 -05:00
Jacob Smith
ded30348ae Add copyright/disclaimer to repack gentest.
Minor formatting tweaks.
2019-07-22 17:34:01 -05:00
Jacob Smith
c7f8cb011d Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~jake.smith/hdf5 into bugfix/repack_external_storage 2019-07-22 17:09:54 -05:00
Jacob Smith
7ff3a63d5f Minor formatting adjustments. 2019-07-22 16:55:37 -05:00
Allen Byrne
ac8a1f02ab Add missing target root path, check other export macro 2019-07-17 11:58:05 -05:00
Allen Byrne
741ef0aaef HDFFV-10845 skip test if CMake command doesn't support ignore EOL 2019-07-17 11:58:05 -05:00
Allen Byrne
3ea746014a HDFFV-10845 copy files with windows EOL 2019-07-17 11:58:05 -05:00
Allen Byrne
95d3b41af0 HDFFV-10845 separate test used by copy windows files 2019-07-17 11:58:04 -05:00
Allen Byrne
0eb050d481 HDFFV-10845 mingw needs special windows reference files 2019-07-17 11:58:04 -05:00
Allen Byrne
f724e86727 HDFFV-10845 use of TARGETFILE disables auto emulator 2019-07-17 11:58:04 -05:00
Allen Byrne
7ba562a820 HDFFV-10845 fix configure check and PATHs 2019-07-17 11:58:03 -05:00
Allen Byrne
9655485f6a non-windows test must be skipped on windows 2019-07-11 10:20:04 -05:00
Quincey Koziol
73df02f13c Merge branch 'develop' into feature/update_gcc_flags 2019-07-09 22:17:22 -05:00
Quincey Koziol
db7c43d375 Updates to warnhist script, along with a few cleanups, and add some comments
to warning cleanups that are a bit obscure.
2019-07-09 22:15:36 -05:00
Allen Byrne
c424877351 perf program can only be built on unix platforms 2019-07-08 15:37:58 -05:00
Allen Byrne
c88caec5ae Declaration needed for parallel standalone 2019-07-08 15:33:31 -05:00
Allen Byrne
ae0c16d1f2 HDFFV-10581 fix locations for static pdb files 2019-07-08 12:21:11 -05:00
Quincey Koziol
78e13fe8b0 Merge branch 'develop' into feature/update_gcc_flags 2019-07-05 13:33:13 -05:00
Allen Byrne
1c7420d1e3 perf test is parallel only 2019-07-03 15:30:41 -05:00
Allen Byrne
ce92a64716 Add missing test 2019-07-03 10:15:07 -05:00
Quincey Koziol
3b594992d6 Add support for GCC9, update warnhist script, and clean up warnings. 2019-07-02 23:43:45 -05:00
Quincey Koziol
2042c775f0 Merge branch 'develop' into feature/update_gcc_flags 2019-06-28 16:13:36 -05:00
Allen Byrne
13d951d372 Merge pull request #1779 in HDFFV/hdf5 from ~BYRN/hdf5_adb:develop to develop
* commit '0405935654e2c3a8e414da6f66a46ab0ef7ef9ec':
  Remove unused code
2019-06-28 10:10:55 -05:00
Dana Robinson
8f0aacce8f Merge pull request #1782 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:warning_fixes to develop
* commit '1c4101c6a77dd9223483dd9e3f7bec0f9e92f435':
  Fixed various warnings in testhdf5 files.
  Warning fixes: * fix for static/const flipped in H5Dint.c * Removed missing done target in H5Rint.c * Fixed misleading indentation in h5format_convert.c
2019-06-28 10:03:45 -05:00
Quincey Koziol
d5130bb573 Updated configure & CMake compiler flags for GCC 8.x, along with corresponding
changes to warnhist script (and some extra improvements for condensing C++
and Java warnings), and fixed a bunch of warnings.
2019-06-28 09:10:43 -05:00
Dana Robinson
de2440ae5f Warning fixes:
* fix for static/const flipped in H5Dint.c
* Removed missing done target in H5Rint.c
* Fixed misleading indentation in h5format_convert.c
2019-06-27 22:38:20 -07:00
Allen Byrne
e7dd793f42 Remove unused code 2019-06-27 08:00:31 -05:00
Vailin Choi
9cafc8d4de Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~vchoi/my_third_fork into bugfix/new_shutdown_fsm 2019-06-26 15:47:54 -05:00
Allen Byrne
f6ddfd7dda Add test cleanup 2019-06-24 16:10:20 -05:00
Allen Byrne
1891324419 Need to shift off two arguments 2019-06-24 15:45:38 -05:00
Vailin Choi
7485981bca Merge pull request #8 in ~VCHOI/my_third_fork from develop to bugfix/new_shutdown_fsm
* commit '35fd0ec8ceffe96cee352187154da15c967fb990':
  Updated H5Tcopy() to get the dataset's datatype through the VOL when that is passed in as the object ID.
  Fix fortran test and test library linking
  Update GCC 6 & 7 flags for CMake builds
  Move -Wnormalized down into GCC 6.x flags
  Put the memcpy overlap check back into H5MM.
  Fixed the heap overflow in t_filters_parallel
  Fixed some low-hanging fruit from -fsanitize in t_filters_parallel.
  Add lib dir for testing plugins
  Add support for GCC 7.x warnings, update warnhist script to account for them, clean up warnings.
  Add H5_HLDLL prefix for windows link
  HDFFV-10805 cleanup examples and test code
  Fix minor typo in H5S_select_iter_release
  Add missing fortran mods
  HDFFV-10805 Fix test of library libinfo
  Cleanup possible CMake target conflicts
  Correct CMake issues
2019-06-24 15:04:38 -05:00
Allen Byrne
ee07744966 HDFFV-9407 add autotools test 2019-06-24 15:04:07 -05:00
Allen Byrne
a03d35dc22 HDFFV-9407 Add test 2019-06-24 07:28:39 -05:00
Quincey Koziol
128853b58a Merge branch 'develop' into feature/update_gcc_flags 2019-06-22 11:39:25 -05:00
Allen Byrne
468b130be3 Add lib dir for testing plugins 2019-06-21 13:46:46 -05:00
Jacob Smith
9103fd5953 Undo some formatting modifications.
Update MANIFEST.
2019-06-21 12:05:54 -05:00
Quincey Koziol
1ccbdfee58 Add support for GCC 7.x warnings, update warnhist script to account for them,
clean up warnings.
2019-06-21 10:05:34 -05:00
Vailin Choi
61b2dddc25 Checkin for new shutting down free-space manager. 2019-06-20 16:14:33 -05:00
Jacob Smith
27d5742b39 Merge branch 'develop' into bugfix/repack_external_storage 2019-06-20 12:37:27 -05:00
Jacob Smith
63fc460c13 Add forgotten files to include from h5repack testfiles for tests. 2019-06-20 12:30:20 -05:00
Jacob Smith
2ba7b8f5e0 Fix failing automated reporting.
Add 8-bit big-endian testfile.
2019-06-19 17:15:05 -05:00
Allen Byrne
aa9a979fbb Add missing fortran mods 2019-06-19 15:42:35 -05:00
Allen Byrne
562cd7f3ab Cleanup possible CMake target conflicts 2019-06-19 10:35:55 -05:00
Allen Byrne
e7f16c6f2c HDFFV-10805 Add option to only build shared targets
HDFFV-10805 Add ONLY_SHARED_LIBS option and prefer shared over static
HDFFV-10803 Update FindSZIP.cmake find module
Remove unneeded modules and update java modules
2019-06-18 07:46:25 -05:00
Larry Knox
3dfc80a28a Moved h5cc.in and h5redeploy.in from tools/misc to bin directory. They
should always be built and installed whether tools are enabled or
disabled.  Also added Makefile.am to bin to build h5redeploy and
to install and uninstall them.  h5cc is created from h5cc.in by
configure.
2019-06-14 10:47:09 -05:00
Larry Knox
5385f032dd Modified Makefile.ams for h5repack, h5watch and gif2h5 to allow
disabling tests.
Moved h5cc.in from tools/src/misc to src directory to always create h5cc
whether or not tools are enabled.
Added configuration status of tools and tests to libhdf5.settings.
2019-06-12 10:21:41 -05:00
Larry Knox
54a07f556c Add options to enable or disable building tools and tests. The default
is enabled for each.
2019-06-05 23:20:01 -05:00
Jacob Smith
ad2a7f4789 First pass at repack consolidating external storage if layout given.
Checks out manually, tests still complain.
2019-05-31 17:24:26 -05:00
Jacob Smith
cb6029196a Formatting changes, mostly. 2019-05-30 15:46:39 -05:00
Jacob Smith
a648183cc3 Add initial failing test for h5repack copying external data into new file. 2019-05-30 15:45:53 -05:00
Jacob Smith
449d6807d5 Modify code comments for clarity.
Some minor reformatting.
2019-05-28 14:31:42 -05:00
Jacob Smith
ea8c99cc56 refactor h5repack_main.c -- remove some redundancy, formatting 2019-05-24 16:57:47 -05:00
Jacob Smith
d1ed626845 Fit help message to 80 character width and clarify some details. 2019-05-14 13:53:28 -05:00
hdftest
1cec61a775 Snapshot version 1.11 release 5.
Update version to 1.11.6.
2019-05-02 10:42:53 -05:00
Allen Byrne
68bddef35d HDFFV-10741 - remove SKIP_ERRSTK_TESTS option 2019-04-15 14:46:23 -05:00
Quincey Koziol
fc945999d0 Merge remote-tracking branch 'origin/develop' into env_vol_load 2019-04-03 17:27:10 -05:00
Allen Byrne
cb0ff923de Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '844f38c6462f42a9a3a6002de6a4acb0520b8fb7':
  Correct entry
  Snapshot version 1.11 release 4. Update version to 1.11.5.
  Allow option to select NAMESPACE
  Correct examples for packaging
2019-04-03 08:56:57 -05:00
hdftest
7a0ac416c9 Snapshot version 1.11 release 4.
Update version to 1.11.5.
2019-04-01 16:11:28 -05:00
Allen Byrne
6e2e461087 HDFFV-10741 - add option to skip tool error stack tests 2019-03-26 14:00:34 -05:00
Dana Robinson
0c20c65e2f Added the HDopen work-around on windows to pio_engine.c 2019-03-15 08:45:38 -07:00
Dana Robinson
750b5c2930 Adds _wopen support on Windows so that files with UTF-8 names
can be opened.

Fixes: HDFFV-2714, HDFFV-3914, HDFFV-3895, HDFFV-8237, HDFFV-10413, HDFFV-10691
2019-03-15 00:41:39 -07:00
Quincey Koziol
deeb302747 Specify the default VOL connector to use with an environment variable.
This implicitly adds support for changing the VOL connector for command-line
tools or any application linked with the library.

Also, add 'make check-vol' support for all directories, clearing up necessary
issues in testing scripts, etc.
2019-03-09 21:41:38 -06:00
Allen Byrne
122b63a5a2 HDFFV-10703 Update CMake commands to latest standard 2019-02-15 15:48:24 -06:00
Allen Byrne
6f71b2cdcf Refactor mpi test names 2019-02-11 12:42:46 -06:00
Allen Byrne
d98faf5703 Rename parallel tests to start with PAR_ 2019-02-11 11:09:43 -06:00
Allen Byrne
f7bab665bb Distinguish parallel performance tests 2019-02-11 10:53:13 -06:00
Allen Byrne
36e3d1600a Refactor out timeout values 2019-02-06 16:41:58 -06:00
Allen Byrne
18e875f75f Fail configure if SZIP/ZLIB is requested but not found 2019-02-05 15:25:30 -06:00
Allen Byrne
ad906dc708 Invalid position of declaration fixed. 2019-01-25 15:21:11 -06:00
Binh-Minh Ribler
2fe69e7639 Merge pull request #1479 in HDFFV/hdf5 from ~BMRIBLER/hdf5_bmr_fixbug:develop to develop
HDFFV-10586 and HDFFV-10588

* commit '25cd1ab02b9ddaf58a4f5422f4ab4fde411e050a':
  Added test for HDFFV-10588
  Fixed HDFFV-10684
  Fixed HDFFV-10586 and HDFFV-10588 Description:     HDFFV-10586 CVE-2018-17434  Divide by zero inh5repack_filters         Added a check for zero value     HDFFV-10588 CVE-2018-17437  Memory leak in H5O_dtype_decode_helper         This is actually an Invalid read issue.  It was found that the         attribute name length in an attribute message was corrupted,         which caused the buffer pointer to be advanced too far and later         caused an invalid read.         Added a check to detect attribute name and its length mismatch.  The         fix is not perfect, but it'll reduce the chance of this issue when a         name length is corrupted or the attribute name is corrupted. Platforms tested:     Linux/64 (jelly)     Linux/64 (platypus)     Darwin (osx1010test)
2019-01-24 17:07:57 -06:00
Allen Byrne
2cf543b294 Fix typo 2019-01-18 12:57:15 -06:00
Allen Byrne
13d1568c3f Fix missing depends for last repart test 2019-01-18 12:27:21 -06:00
Binh-Minh Ribler
43bd93c4f5 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~bmribler/hdf5_bmr_fixbug into develop 2019-01-15 11:49:07 -06:00
Binh-Minh Ribler
bc3d878add Fixed HDFFV-10586 and HDFFV-10588
Description:
    HDFFV-10586 CVE-2018-17434  Divide by zero inh5repack_filters
        Added a check for zero value
    HDFFV-10588 CVE-2018-17437  Memory leak in H5O_dtype_decode_helper
        This is actually an Invalid read issue.  It was found that the
        attribute name length in an attribute message was corrupted,
        which caused the buffer pointer to be advanced too far and later
        caused an invalid read.
        Added a check to detect attribute name and its length mismatch.  The
        fix is not perfect, but it'll reduce the chance of this issue when a
        name length is corrupted or the attribute name is corrupted.
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1010test)
2019-01-15 11:48:31 -06:00
Binh-Minh Ribler
56b32278c4 Merge pull request #1441 in HDFFV/hdf5 from ~BMRIBLER/hdf5_bmr_fixbug:develop to develop
* commit '90d13bef33f9e2e80b23996a0c39f16f7c34ecf8':
  Fixed typo Platforms tested:     Darwin (osx1010test)
  Refixed HDFFV-10578 Description:     Applied Neil's fix for this issue after removing previous     attempt.  The resources are now released in init_objs() when     failure occurs there.     Neil will fix HDFFV-10676 separately. Platforms tested:     Linux/64 (jelly)     Linux/64 (platypus)     Darwin (osx1010test)
  Removed previous change in H5O__chunk_deserialize().
  Removed the previous change in H5O__chunk_deserialize()
  Removed previous change in table_list_add().
  Removed the previous change in table_list_add()
  Updated per review Description:     HDFFV-10676 - CVE-2018-13873         Changed the new assert to if statement, per Dana's comment. Platforms tested:     Linux/64 (jelly)
  HDFFV-10578 and HDFFV-10676 Description:     HDFFV-10578 - CVE-2018-17234         The file has some issue, however, there was a bug in h5dump that caused         memory leaks after the problem in the file was encountered. The bug         was that an if statement was missing in the function table_list_add()         resulting in the memory not being freed at a later time.         After the fix had been applied, there were no more leaks after h5dump         detected the issue in the file and reported the error.
2019-01-08 18:59:22 -06:00
Binh-Minh Ribler
90d13bef33 Fixed typo
Platforms tested:
    Darwin (osx1010test)
2019-01-07 20:46:55 -06:00
Binh-Minh Ribler
78d0564c2a Refixed HDFFV-10578
Description:
    Applied Neil's fix for this issue after removing previous
    attempt.  The resources are now released in init_objs() when
    failure occurs there.
    Neil will fix HDFFV-10676 separately.
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1010test)
2019-01-07 20:36:34 -06:00
Binh-Minh Ribler
5b721a060e Removed previous change in table_list_add(). 2019-01-07 14:03:22 -06:00
Binh-Minh Ribler
463515f7db Removed the previous change in table_list_add() 2019-01-07 14:00:56 -06:00
Binh-Minh Ribler
aa62951a5f Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~bmribler/hdf5_bmr_fixbug into develop 2019-01-06 01:45:15 -06:00
Binh-Minh Ribler
e1b59919bb HDFFV-10578 and HDFFV-10676
Description:
    HDFFV-10578 - CVE-2018-17234
        The file has some issue, however, there was a bug in h5dump that caused
        memory leaks after the problem in the file was encountered. The bug
        was that an if statement was missing in the function table_list_add()
        resulting in the memory not being freed at a later time.
        After the fix had been applied, there were no more leaks after h5dump
        detected the issue in the file and reported the error.

        In H5O__chunk_deserialize, replaced an assert with an if statement
        and reporting error, per Neil's recommendation

    HDFFV-10676 - CVE-2018-13873
        Also in H5O__chunk_deserialize, added an assertion to detect
        out of bound ids
2019-01-06 01:44:40 -06:00
Quincey Koziol
8ecd9cc1a1 Updated error stacks for h5dump testing. 2019-01-05 23:41:43 -06:00
Allen Byrne
028d47d707 Fix runtest case for data to stdout in tool 2018-12-31 12:38:09 -06:00
Allen Byrne
991996b251 Remove unused CMake files 2018-12-21 12:22:32 -06:00
Allen Byrne
7e8923957f HDFFV-10656 Add CHECK_VOL support to CMake 2018-12-20 15:11:27 -06:00
Allen Byrne
8f3a777a8a Use H5I_INVALID_HID 2018-12-15 06:25:02 -06:00
Allen Byrne
11b4c1f3af HDFFV-10632 update autotools test scripts to match cmake 2018-12-06 12:48:39 -06:00
Allen Byrne
eb1b3f42c4 HDFFV=10632 test cleanup 2018-12-06 07:54:19 -06:00
Allen Byrne
307749cc71 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '55e87e5b86aacbb37c57614cd2e869541291caad': (51 commits)
  Correct Windows build.
  Refactor allocating & releasing pass through wrapper objects.
  Fix typo
  Remove duplicated comment.
  Add a couple of missing prototypes for static routines, along with updating a comment from VOL plugin -> connector.
  Switch H5VL_class_value_t from unsigned to int, and add "invalid" ID value.
  Correct "make check-vol" regression test target.
  Switch H5VL_class_value_t from enum to unsigned integer
  Add support for "make check-vol", along with a few minor cleanups, etc.
  Refactor infrastructure for setting FAPL information from environment variables during testing, including connecting native, pass-through, and dynamically loaded VOL connectors.   Also bring native and pass-through VOL connectors into alignment, removing the "H5VLnative_private.h" header.
  Remove example VOL connector, since it's been superceded by the pass-through VOL connector in the src subdirectory.
  Add pass-through VOL connector
  Update API tracing info for VOL functions and regenerate trace macros for VOL API calls.
  Added 'notify' callback for async requests; switched VOL class and info comparison to return comparison value as parameter, so they can return error values; "cancelled" -> "canceled"; switched order of 'wrap_object' and 'free_wrap_ctx' management callbacks.
  Add using FAPL from h5_fileaccess() to more tests.
  Remove unused test for unimplemented routine.
  Switch loc_params to VOL callbacks to pass struct by pointer instead of by value.
  Add info_to_str and str_to_info "management" callbacks for serializing and deserializing a connector's info object.
  Revert some of the changes to support the original property list value of metadata read attempts.
  Add VOL connector info to the flie access property list returned from H5Fget_access_plist().  Also, other misc. cleanups, etc.
  ...
2018-12-05 14:25:58 -06:00
Allen Byrne
a68fd29208 Fix special macro stdout as empty 2018-12-05 09:44:13 -06:00
Allen Byrne
d6de299f01 Need nonexistent file to skip stdout compare 2018-12-04 23:16:03 -06:00
Allen Byrne
e3e24a2989 HDFFV-10632 - split stderr and stdout in cmake 2018-12-03 09:35:59 -06:00
Quincey Koziol
d0d360ff2d Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into stackable_vol 2018-11-18 00:39:27 -06:00
Songyu Lu
d25afb294c HDFFV-10601: Some compiler doesn't support clock_gettime(). So I changed to use gettimeofday (HDgettimeofday in the HDF5 library). 2018-11-16 12:06:30 -06:00
Ray Lu
e07d097da1 Merge pull request #1316 in HDFFV/hdf5 from ~SONGYULU/hdf5_ray:bugfix/HDFFV-10601-issues-with-chunk-cache-hash to develop
* commit 'cd13d24e5140578a880aebe4e2d8b899179d0870':
  HDFFV-10601: I added error checking to the HDF5 functions.
  HDFFV10601: Adding performance test to verify the improvement.
  HDFFV-10601: I changed to a better way to calculate the number of chunks in a dataset.
  HDFFV-10601 Issues with chunk cache hash value calcuation:
2018-11-15 09:43:46 -06:00
Songyu Lu
cd13d24e51 HDFFV-10601: I added error checking to the HDF5 functions. 2018-11-14 17:30:23 -06:00
Dana Robinson
3edd97731e Renamed h5repart's -family_to_sec2 to -family_to_single. 2018-11-14 13:37:31 -08:00
Songyu Lu
2fe10c647c HDFFV10601: Adding performance test to verify the improvement. 2018-11-13 11:37:29 -06:00
Quincey Koziol
e2ed2edf6e Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into stackable_vol 2018-11-12 16:23:54 -06:00