Commit Graph

1571 Commits

Author SHA1 Message Date
Allen Byrne
f9a3ae5081 TRILAB-244 separate CXX warnings and errors from C 2020-04-23 14:32:29 -05:00
Allen Byrne
e68848f397 TRILAB-192 - cleanup edits and match CMake CXX to autotools 2020-04-13 08:16:57 -05:00
Jacob Smith
075e74e9b2 Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~jake.smith/hdf5 into feature/vfd_splitter_mirror_a 2020-04-07 10:29:00 -05:00
Jacob Smith
d97c00013d Tidying of Mirror VFD.
* Rename server-stop utility to mirror_server_stop.
* Remove external dependency on bzero().
* Modify test/use_common to use only the public API.
* Rename internal bitswap macro to follow convention.
2020-04-07 10:20:44 -05:00
Allen Byrne
0d701b9ff8 Move intel warnings to subfolder, update autotools files 2020-04-06 09:35:43 -05:00
Allen Byrne
2c2627e7f6 Flag construction must be after compiler detection 2020-04-05 13:38:04 -05:00
Allen Byrne
57f5d00a43 TRILAB-192 add c++ and fortran warnings build systems one file 2020-04-05 10:50:08 -05:00
Allen Byrne
0da53027d9 TRILAB-192 add comparable clang flags 2020-03-27 17:10:53 -05:00
David Young
43e5a89697 Update the MANIFEST for the autoconf/cmake shared warnings files. 2020-03-24 10:24:51 -05:00
Jacob Smith
b65405439d Add Splitter VFD to library.
* "Simultaneous and equivalent" Read-Write and Write-Only channels for
  file I/O.
* Only supports drivers with the H5FD_FEAT_DEFAULT_VFD_COMPATIBLE flag for
  now, preventing issues with multi-file drivers.

Add Mirror VFD to library.

* Write-only operations over a network.
* Uses TCP/IP sockets.
* Server and auxiliary server-shutdown programs provided in a new directory,
  `utils/mirror_vfd`.
* Automated testing via loopback ("remote" of localhost).
2020-03-13 17:13:17 -05:00
Allen Byrne
1885c3e7d7 Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit 'd24afb233383aa6c945fad430eb1b66edc4d6c63':
  Add missing MANIFEST entry, ./doc/code-conventions.md .
2020-02-24 13:00:25 -06:00
Allen Byrne
f7212df030 HDFFV-11036 add file compare test process 2020-02-24 12:45:22 -06:00
David Young
921278c604 Add missing MANIFEST entry, ./doc/code-conventions.md . 2020-02-20 13:34:33 -06:00
David Young
00608a8756 Merge pull request #2341 in HDFFV/hdf5 from ~DYOUNG/werror:thread_id to develop
* commit 'cefacee21b5ad569e2394b32793648e1b80f3d6b':
  src/H5Eint.c: #include H5TSprivate.h for H5TS_thread_id() definitions.
  Change thread IDs to uint64_t from unsigned long, per Quincey's suggestion.
  Add thread_id.c to the MANIFEST and the CMakeLists.txt per Allen's request.
  Replace pthread_self_ulong() with H5TS_thread_id().  The POSIX Threads implementation ought to be portable to any system that has POSIX Threads.  On Windows, I use the same API call as before.
2020-02-05 13:34:00 -06:00
Allen Byrne
00d68851fc HDFFV-10996 - add java vol tests 2020-02-03 13:45:14 -06:00
David Young
65600cbd72 Add thread_id.c to the MANIFEST and the CMakeLists.txt per Allen's
request.
2020-02-03 12:59:52 -06:00
Allen Byrne
3e9980d20d Merging in latest from upstream (HDFFV/hdf5:refs/heads/develop)
* commit '66a94df13b3801547523f1ea90bc84fe2ee6824b':
  Removed UIUC from copyright, added helpful comment.
  Updated MANIFEST
  Added a config file for ARM (Tested on a Raspberry Pi 3B+ w/ gcc 6).
  Tidying from code review.
  Fixed stack and frame size warnings. Not complete, but fixes most of the easier cases.
2020-01-19 07:45:15 -06:00
Allen Byrne
cf84882b3c Create common build system files for warnings 2020-01-17 13:14:06 -06:00
Dana Robinson
b51ae5f35e Updated MANIFEST 2020-01-17 05:00:23 -08:00
Dana Robinson
a92c735c9b Squashed commit of the token_refactoring branch: 2020-01-16 13:29:34 -08:00
Jerome Soumagne
a7648879d7 Add test for reference shutdown issue 2020-01-15 17:23:33 -06:00
Allen Byrne
f679b7495d Merge pull request #2173 in HDFFV/hdf5 from ~BYRN/hdf5_adb:develop to develop
* commit '576e713924b1802a64ae2f6655d2389d70fadfd1':
  Add new h5diff files
  Fix compile statement order
  HDFFV-10976,-10980 Init obj_type before calling H5Rget_obj_type3
  Add unknown define for h5dump ref objects
  Standalone doesn't use h5test implementation.
  HDFFV-10980 - h5diff uses new ref APIs
2019-12-21 07:49:22 -06:00
Allen Byrne
576e713924 Add new h5diff files 2019-12-21 07:48:23 -06:00
Quincey Koziol
58cf795321 Refactor all the 'H5VL_*_optional' callbacks to move the type of operation out
of the va_list, so it's at least possible for another connector to know what
the operation is and decide whether to implement it or not.

Added a new VOL sub-class called "introspect" where callbacks that report
information about the connector or container can be placed.  Added an
'opt_query' callback to this sub-class, for a connector to report back
to the library whether a particular optional callback operation is supported.
Also added a 'get_conn_cls' introspection callback, to retrieve the H5VL_class_t
of a connector (either the "current" connector, H5VL_GET_CONN_LVL_CURR, or
the terminal connector, H5VL_GET_CONN_LVL_TERM).

Moved the "post open" operation from a file 'specific' operation to a file
'optional' operation, now that it's possible to detect (with the 'opt_query'
introspection callback) whether a VOL connector implements an optional
operation, without just returning an error.

Added new internal VOL helper routines: H5VL_object_is_native, to determine
if an object is in (or is a) native file, and H5VL_file_is_same, to determine
if two objects are in (or are) the same terminal VOL connector's container.
(And moved the special handling for FILE_IS_EQUAL operation out of internal VOL
callback routine into H5VL_file_is_same)

Made new dataset 'get' operation for H5Dvlen_get_buf_size, aligning it better
with other 'get' operations in API.

Fixed several issues with pass-through connectors, which are now passing the
'make check-passthrough-vol' tests again.

A bunch of warning and style cleanups as well.
2019-12-19 22:41:37 -06:00
Allen Byrne
628c63f125 HDFFV-10980 - h5diff uses new ref APIs 2019-12-19 16:29:43 -06:00
Allen Byrne
74524c5a35 TRILABS-135 Add clang analyzers 2019-12-11 16:49:34 -06:00
Allen Byrne
96cb94a95c Add missing filenames 2019-12-09 14:14:39 -06:00
Allen Byrne
349656ecba Add new test reference 2019-12-08 13:58:35 -06:00
David Young
f1b39ad80e Apparently, + has no special meaning, and neither does \+, in so-called
"obsolete" / POSIX "basic" regular expressions.  Also, not every version of
`sed` out there supports the `-E` option.  So delete the -E flag and use
the regex `[^/][^/]*` instead of `[^/]+`.

Add config/netbsd to the MANIFEST.
2019-11-12 10:43:26 -06:00
Jerome Soumagne
ae490016b9 Add new H5R API that abstracts object, region and attribute reference types
Also support references to external files

Add new H5T_REF type and type conversion routines

Support conversion from H5T_REF_OBJ/DSET_REG to H5T_REF

Add H5Treclaim() API to reclaim memory of vlen/reference types

Deprecate H5Dvlen_reclaim()

Fix H5T_vlen_reclaim() and H5T_reclaim() to use private callback

Add H5T_ref_reclaim()

Move previous H5R APIs to H5Rdeprec.c

Clean up H5Ocopy

Separate H5O_copy_expand_ref() to H5Ocopy_ref()

Add support for copying new reference types

Clean up deprecated routines to go through VOL and same code path

Fix return codes in existing trefer.c test

Rename trefer.c to trefer_deprec.c

trefer.c is for new references

Add performance test for trefer

Add additional obj_copy_ref test

Make use of tokens and blobs to store references

Skip blob encoding for object references

Start adding new reference examples
2019-10-08 14:30:24 -05:00
Quincey Koziol
eaa65c862b Add 'blob' callbacks to VOL, along with a native implementation to store them
in the global heap, and changed the VL datatype conversion code to use blobs.

Move encode/decode of sequence lengths into VL datatype callbacks, from native
VOL blob routines.
2019-10-08 14:30:23 -05:00
Allen Byrne
09e6dcbfbb Add new toolchain file 2019-10-01 10:09:16 -05:00
Jacob Smith
35a9e9c50f Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~jake.smith/hdf5 into bugfix/repack_external_storage 2019-09-23 17:23:02 -05:00
Jacob Smith
aa797af31b Refactor h5repackgentest to have more modular internals, for easier
addition of new tests.
Update manifest and test script for new hdf5 external-storage .h5 files.
2019-09-23 16:21:37 -05:00
Binh-Minh Ribler
a8af828bbf Merge pull request #1908 in HDFFV/hdf5 from ~BMRIBLER/hdf5-bmr:develop to develop
- Added new chunk query functions: H5Dget_num_chunks(), H5Dget_chunk_info(), and H5Dget_chunk_info_by_coord()
- Fixed iterator issue in H5EA_iterate(), H5FA_iterate(), and H5D__none_idx_iterate()

* commit '96dab622e3bbe75d85b0cef3ca2bd5288cf4ac32':
  Fixed error code and error messages Tested on Jelly (very minor)
  Changed H5VL_NATIVE_DATASET_GET_CHUNK_INFO_BY_COOR to H5VL_NATIVE_DATASET_GET_CHUNK_INFO_BY_COORD per a review comment.
  Updated comments
  Updated based on reviews and refactored test code
  Code cleanup and refactor
  Re-ordered items w.r.t. date.
  Fixed typo
  Added notes about HDFFV-10677 and HDFFV-10661
  HDFFV-10677 and HDFFV-10661
2019-09-03 11:08:42 -05:00
Allen Byrne
fa540a3af8 Add 32-bit cross-compile toolchain 2019-08-29 10:36:12 -05:00
Binh-Minh Ribler
e06bf9d263 HDFFV-10677 and HDFFV-10661
Description:
    - Added functions to query chunk information:
        H5Dget_num_chunks(dset_id, fspace_id, *nchunks)
            Gets the number of written chunks that intersect with the given
            dataspace.  However, in this version, the intersection is not
            yet completed.  Thus, the number of all written chunks will be
            returned.
        H5Dget_chunk_info_by_coord(dset_id, *offset, *filter_mask, *addr, *size)
            Given a chunk's logical coordinates, returns the chunk's filter,
            address, and size.
        H5Dget_chunk_info(dset_id, fspace_id, index, *offset, *filter_mask, *addr, *size)
            Given a chunk's index, returns the chunk's logical coordinates, filter,
            address, and size.  The chunk belongs to a set of chunks that have
            nonempty intersection with the specified dataspace.  However, in
            this version, the intersection is not yet completed, and the index
            is of all the written chunks.
        These functions comply with VOL.

    - Fixed some oversights found in the library for the tests in chunk_info.c
      to work correctly.  The returned value from a callback function was not
      checked in H5EA_iterate(), H5FA_iterate(), and H5D__none_idx_iterate().
      This oversight caused a callback function to continue iterating even though
      it's supposed to stop.
Platforms tested:
    Linux/64 (jelly)
    Linux/64 (platypus)
    Darwin (osx1011test)
2019-08-28 10:08:12 -05:00
Dana Robinson
bd8da502cf Some refactoring prior to implementing new H5P MPI functions.
* Macro cleanup and obvious warning fixes in parallel code.
* Moved H5FD comm and info dup/free wrapper code to a new H5mpi.c file
  and separated it to deal with each MPI type separately.
2019-08-14 16:22:06 -07:00
Dana Robinson
5b6bd7c922 Added the map (H5M) API 2019-08-13 01:36:42 -07:00
Allen Byrne
c60ceb8c08 Merge reconciliation and VFD test fixes 2019-08-04 15:19:51 -05:00
Dana Robinson
4227b8948d Reverts the removal of config/conclude_fc.am while keeping
the changes to config/gnu-fflags.
2019-07-31 22:51:53 -05:00
Dana Robinson
99a6a5a16e Fixed the MANIFEST 2019-07-30 07:29:37 -07:00
Dana Robinson
d169391529 Merge pull request #1808 in HDFFV/hdf5 from ~DEROBINS/hdf5_der:gnu-flags-work to develop
* commit '7d291d059da6287f769951c4854c566da0b93ce8':
  Parallel Fortran tests now use the MPI module instead of including mpif.h.
  First stab at gfortran versions. Tested on gfortran 7-9 so far.
  Moved the -std=f2008 option to gfortran 9 for now.
  Updated gfortran options. * Removed conclude_fc.am, which was causing verbose builds. Need to   investigate more, though. * Added gfortran 9 warnings and flags. This will need to be separated   into other version blocks based on when the options first appeared. * The standard is now explicitly set at 2008.
2019-07-29 16:13:24 -05:00
Larry Knox
8008294578 Squashed commit of the following:
Merge changes from update_merged_S3_HDFS branch into develop.

commit d5034315aea88629929ac0c9c59ebfafd5f21a31
Merge: 9c48823 d3fdcd8
Author: Larry Knox <lrknox@hdfgroup.org>
Date:   Thu Jul 25 08:24:53 2019 -0500

    Merge branch 'develop' into update_merged_S3_HDFS
2019-07-25 11:47:12 -05:00
Jacob Smith
c7f8cb011d Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/~jake.smith/hdf5 into bugfix/repack_external_storage 2019-07-22 17:09:54 -05:00
Allen Byrne
7bb020fcec HDFFV-10845 add support for mingw toolchain 2019-07-17 11:56:48 -05:00
Dana Robinson
8bd2c81cd5 Updated gfortran options.
* Removed conclude_fc.am, which was causing verbose builds. Need to
  investigate more, though.
* Added gfortran 9 warnings and flags. This will need to be separated
  into other version blocks based on when the options first appeared.
* The standard is now explicitly set at 2008.
2019-07-17 01:23:44 -07:00
Quincey Koziol
3b594992d6 Add support for GCC9, update warnhist script, and clean up warnings. 2019-07-02 23:43:45 -05:00
Allen Byrne
a03d35dc22 HDFFV-9407 Add test 2019-06-24 07:28:39 -05:00
Jacob Smith
9103fd5953 Undo some formatting modifications.
Update MANIFEST.
2019-06-21 12:05:54 -05:00