* hbool_t --> bool in src
* Does not remove TRUE/FALSE
* Public header files are unchanged
* Public API calls are unchanged
* TRUE/FALSE --> true/false in src
* Add deprecation notice for hbool_t
These are meaningless noise.
Removes the "Programmer" lines on comment start lines:
/* Programmer: John Smith
These complicate my sed script that will rip out the rest of
the comments.
Moves the file-independent encode and decode macros to a new
H5encode.h header that is itself included in H5private.h.
Removes UINT64ENCODE_VARLEN and UINT64DECODE_VARLEN, which
were unused.
Fixes include statements in files where H5VMprivate.h and
H5MMprivate.h were included via H5Fprivate.h.
* Updated source file copyright headers to remove "Copyright by the Board of Trustees
of the University of Illinois", which is kept in the top-level COPYING file.
* Fix bug with cross platform compatibility of references within vlens.
No testing yet.
* Merge from multi_rd_wd_coll_io to a more recent branch from develop.
Untested, probably does not work yet.
* Committing clang-format changes
* Committing clang-format changes
* Fix many bugs in multi dataset branch. Mostly works, some issues in
SWMR tests.
* Committing clang-format changes
* Disable test in swmr.c that was failing due to bug in HDF5 unrelated to
multi dataset.
* Committing clang-format changes
* Fixed fortran multi-dataset tests
* Fixed xlf errors
* Added benchmark code for multi-datasets
* loops over datasets
* added missing error arg.
* Added gnuplot formatting
* Jonathan Kim original MD benchmarking code
* updated MD benchmarking code
* code clean-up
* Only make files in feature test mode
* misc clean-up
* removed TEST_MDSET_NO_LAST_DSET_2ND_PROC option
* Committing clang-format changes
* Change multi dataset API to use arrays of individual parameters instead
of the parameter struct.
* Committing clang-format changes
* Update to new multi dataset Fortran API and tests. (#1724)
* Update to new multi dataset Fortran API and tests.
* Sync Fortran with develop.
* skipping h5pget_mpio_actual_io_mode_f for now
* Fixed issue with dxpl_id, changed to variable size dim. (#1770)
* Remove "is_coll_broken" field from H5D_io_info_t struct
* Committing clang-format changes
* Minor cleanup in multi dataset code.
* Committing clang-format changes
* Clean up in multi dataset code.
* Committing clang-format changes
* Committing clang-format changes
* Fix speeling
* Fix bug in parallel compression. Switch base_maddr in io_info to be a
union.
* Committing clang-format changes
* Implement selection I/O support with multi dataset. Will be broken in
parallel until PR 1803 is merged to develop then the MDS branch.
* Committing clang-format changes
* Spelling
* Fix bug in multi dataset that could cause errors when only some of the
datasets in the multi dataset I/O used type conversion.
* Committing clang-format changes
* Integrate multi dataset APIs with VOL layer. Add async versions of
multi dataset APIs.
* Committing clang-format changes
* Spelling fixes
* Fix bug in non-parallel HDF5 compilation.
* Committing clang-format changes
* Fix potential memory/free list error. Minor performance fix. Other minor
changes.
* Committing clang-format changes
* Fix memory leak with memory dataspace for I/O.
* Committing clang-format changes
* Fix stack variables too large. Rename H5D_dset_info_t to
H5D_dset_io_info_t.
* Committing clang-format changes
* Remove mem_space_alloc field from H5D_dset_io_info_t. Each function is
now responsible for freeing any spaces it adds to dset_info.
* Committing clang-format changes
* fixed _multi Fortran declaration
* Refactor various things in (mostly) the serial I/O code path to make
things more maintainable.
* Committing clang-format changes
* updated to array based, doxygen, and examples
* Reinstate H5D_chunk_map_t, stored (via pointer) inside
H5D_dset_io_info_t.
* Change from calloc to malloc for H5D_dset_io_info_t and H5D_chunk_map_t.
Switch temporary dset_infos to be local stack variables.
* Committing clang-format changes
* format cleanup
* format cleanup
* added coll and ind
* Modify all parallel I/O paths to take dset_info instead of assuming
dset_info[0].
* Committing clang-format changes
* fixed output
* Rework parallel I/O code to work properly with multi dataset in more
cases. Fix bug in parallel compression.
* Committing clang-format changes
* Prevent H5D__multi_chunk_collective_io() from messing up collective opt
property for other datasets in I/O. Other minor cleanup. Add new test
case to t_pmulti_dset.c for H5FD_MPIO_INDIVIDUAL_IO, disabled for now
due to failures apparently unrelated to multi dataset code.
* Fix spelling
* Committing clang-format changes
* Replace N log N algorithm for finding chunk in
H5D__multi_chunk_collective_io() with O(N) algorithm, and remove use of
io_info->sel_pieces in that function.
* Committing clang-format changes
* Replace sel_pieces skiplist in io_info with flat array of pointers, use
qsort in I/O routine only when necessary.
* Committing clang-format changes
* Add new test case to mdset.c
* Committing clang-format changes
* Fix spelling
* Very minor fix in H5VL__native_dataset_read()
* Fix bug that could affect filtered parallel multi-dataset I/O.
* Add RM entries for H5Dread_multi(), H5Dread_multi_async(),
H5Dwrite_multi(), and H5Dwrite_multi_async()
* Unskip test in swmr.c
* Committing clang-format changes
* Eliminate H5D__pre_read and H5D__pre_write
* Remove examples/ph5mdsettest.c. Will fix and re-add as a test.
* Enable hyperslab combinations in mdset test
* Committing clang-format changes
* Clarify H5Dread/write_multi documentation.
* Fix bugs in multi-dataset I/O. Expand serial multi dataset test.
Update macro in parallel multi dataset test.
* Committing clang-format changes
* Spelling
* Remove obsolete entry in bin/trace
* Rework type conversion buffer allocation. Only one buffer is shared
between datasets in mdset mode, and it is malloced instead of calloced.
* Committing clang-format changes
* Fix bug in error handling in H5D__read/write
* added multi-dataset fortran check with optional dataset creation id (#2150)
* removed dup. dll entry
* Address comments from code review.
* Remove spurious changes in H5Fmpi.c
* Fix issue with reading unallocated datasets in multi-dataset mode.
Address other comments from code review.
* Committing clang-format changes
* Delay chunk index lookup from io_init to mdio_init so it doesn't add
overhead to single dataset I/O.
* Committing clang-format changes
* Fix inappropriate use of piece_count
* updated copyright on new file, removed benchmark from testing dir.
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: M. Scot Breitenfeld <brtnfld@hdfgroup.org>
Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com>
Hyperslabs can be reworked inside several H5S callbacks, making H5S_t
non-const in some places where it is marked const. This change switches
these incorrectly const H5S_t pointer parameters and variables to
non-const where appropriate.
* Modify temporary rpath for testing in java example scripts.
* Update URL in source file Copyright headers for web copy of COPYING
file - src and test directories.
Description:
- Added parameter validation (HDFFV-10210)
- Added detection of division by zero (HDFFV-10587 - CVE-2018-17438)
- Fixed typos in various tests
Platforms tested:
Linux/64 (jelly)
Linux/64 (platypus)
Darwin (osx1011test)
* commit '54957d37f5aa73912763dbb6e308555e863c43f4':
Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes.
Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh.
Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1
Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
Clean up more warnings: drop the warning count from ~1310 down to ~940,
with only 31 types of warnings in 148 files (down from 38 types in 167 files).
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial & parallel
(h5committest forthcoming)
Complete revamp of package initialization/shutdown mechanism in the library.
Each package now has a single init/term routine.
This new way should avoid packages being re-initialized during library
shutdown and is also be _much_ more proactive about giving feedback for
resource leaks internal to the library.
Introduces a new "module" header file for packages in the library
(e.g src/H5Fmodule.h) which sets up some necessary package configuration macros
for the FUNC_ENTER/LEAVE macros. (The VFL drivers have their own slightly
modified version of this header, src/H5FDdrvr_module.h)
Also cleaned up a bunch of resources leaks all across the library and tests,
along with addressing many warnings, as I encountered them.
Tested on:
MacOSX/64 10.10.5 (amazon) w/serial & parallel
Linux/64 3.10.x (kituo) w/serial & parallel
Linux/64 2.6.x (ostrich) w/serial
Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward
the final design (as exemplified by the H5EA & H5FA code).
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & parallel
Refactor function name macros and simplify the FUNC_ENTER macros, to clear
away the cruft and prepare for further cleanups.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
Bring revision 16278 back from revise_chunks branch:
Update layout information in DCPL to unify all information in one
underlying property and switch to using H5O_layout_t for storing it, which
simplifies things considerably.
Also, fix many compiler warnings.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(Original patch tested on many machines)
Omnibus raw data I/O revisions, with wide-ranging changes and
refactoring, in order to prepare for implementing "fast append" feature.
These changes remove the majority of the code duplication for raw data
I/O which has crept in over the last ten years and introduces a more object-
oriented design for operating on different types of dataset storage.
Chunked storage no longer has it's own I/O routines, it is now handled
as either contiguous (if chunk is not pulled into the cache) or compact (if the
chunk is cached in memory).
No bug or feature changes, at least intentionally... :-)
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.2 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
- Extracted common code from H5D_select_read/H5D_select_write into single
routine (H5D_select_io) and made H5D_select_read/H5D_select_write
"gateway" routines, passing I/O buffer to operate on in "op" struct.
- Create optimized pathway in H5D_select_io for I/O operations on single
elements.
- Make "get linear offset of first element" callback for each type of
selection (used in "optimized pathway", above)
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
Refactor generic property list initialization code to put property list
specific routines in property list modules, instead of scattered to the four
winds. Also, introduce property list class initialization objects, to make
adding new property list classes in the library easier.
Fix daily test failure by using H5Pget_elink_prefix() API routine instead
of looking at the "raw" generic property list information.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/C++ & FORTRAN
Linux/64 2.4 (mir) w/build-all & 1.6 compat
Code cleanup
Description:
Clean up some compiler warnings (esp. those flagged on Windows builds)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Enhanced collective chunk IO supports
Description:
1. When using collective IO with chunking storage without any tuning, performance may become worse under some circumstances.
2. Current HDF5 handles raw-data IO per chunk. So for many small chunks, many small IOs will be passed into MPI-IO. That may cause bad performance.
3. For one IO per chunk case, sometimes performance with collective is worse than performance with independent. An obvious case is when only one process is doing IO and all other process are not doing IO, the collective IO will only add overheads for communication. We want to avoid this case. Some management inside our library needs to be done.
Solution:
- Added managements of collective IO supports for chunking storage
inside parallel HDF5
1) Implemented One IO with collective mode for all chunks in the
application by building one MPI derived datatype accross all
chunks.
2) Implemented the decision-making support to do collective IO inside
MPI-IO per chunk.
3) Added the decision-making support to do one IO accross all chunks
or to do multiple IOs with each IO per chunk.
4) Added the support to handle the case some processes won't do any IOs in
collectively.
5) Some MPI-IO package(mpich 1.2.6 or lower, e.g.) cannot handle
collective IO correctly for the case when some processes have no
contributions to IOs, a special macro is added to change
collective IO mode to independent IO mode inside HDF5 library.
Platforms tested:
Parallel:
IBM AIX 5.2(copper)
Linux (heping) mpich-1.2.6
SDSC Teragrid mpich-1.2.5
Linux(Tungsten) mpich-1.2.6
Altix(NCSA cobalt)
Seq:
Linux(heping)
Misc. update:
Code cleanup
Description:
Clean up & standardize a bit in preparation for coding standards
discussion.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Description: Removed PABLO from the source
Solution:
Platforms tested: arabica with 64-bit, copper with parallel,
heping with GNU C and C++ and PGI fortran (but
I disabled hl, there is some weird problem only
on heping: F9XMODFLAG is not
propagated to the Makefile files
Misc. update:
Bug fix & code cleanup
Description:
More dataset cleanups to get to a point where we can fix the chunked I/O
bug.
Also fix a couple of errors in the recent file object resurrection changes
which should hopefully address the recent daily test failres (H5T.c)
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
h5committest
