mirror of
https://github.com/HDFGroup/hdf5.git
synced 2024-12-09 02:39:55 +08:00
4394 lines
177 KiB
Plaintext
4394 lines
177 KiB
Plaintext
HDF5 History
|
|
============
|
|
|
|
This file contains development history of the HDF5 1.10 branch
|
|
|
|
06. Release Information for hdf5-1.10.4
|
|
05. Release Information for hdf5-1.10.3
|
|
04. Release Information for hdf5-1.10.2
|
|
03. Release Information for hdf5-1.10.1
|
|
02. Release Information for hdf5-1.10.0-patch1
|
|
01. Release Information for hdf5-1.10.0
|
|
|
|
[Search on the string '%%%%' for section breaks of each release.]
|
|
|
|
%%%%1.10.4%%%%
|
|
|
|
HDF5 version 1.10.4 released on 2018-10-05
|
|
================================================================================
|
|
|
|
|
|
INTRODUCTION
|
|
|
|
This document describes the differences between this release and the previous
|
|
HDF5 release. It contains information on the platforms tested and known
|
|
problems in this release. For more details check the HISTORY*.txt files in the
|
|
HDF5 source.
|
|
|
|
Note that documentation in the links below will be updated at the time of each
|
|
final release.
|
|
|
|
Links to HDF5 documentation can be found on The HDF5 web page:
|
|
|
|
https://portal.hdfgroup.org/display/HDF5/HDF5
|
|
|
|
The official HDF5 releases can be obtained from:
|
|
|
|
https://www.hdfgroup.org/downloads/hdf5/
|
|
|
|
Changes from Release to Release and New Features in the HDF5-1.10.x release series
|
|
can be found at:
|
|
|
|
https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide
|
|
|
|
If you have any questions or comments, please send them to the HDF Help Desk:
|
|
|
|
help@hdfgroup.org
|
|
|
|
|
|
CONTENTS
|
|
|
|
- Bug Fixes since HDF5-1.10.3
|
|
- Supported Platforms
|
|
- Tested Configuration Features Summary
|
|
- More Tested Platforms
|
|
- Known Problems
|
|
- CMake vs. Autotools installations
|
|
|
|
|
|
New Features
|
|
============
|
|
|
|
Configuration:
|
|
-------------
|
|
- Add toolchain and cross-compile support
|
|
|
|
Added info on using a toolchain file to INSTALL_CMAKE.txt. A
|
|
toolchain file is also used in cross-compiling, which requires
|
|
CMAKE_CROSSCOMPILING_EMULATOR to be set. To help with cross-compiling
|
|
the fortran configure process, the HDF5UseFortran.cmake file macros
|
|
were improved. Fixed a Fortran configure file issue that incorrectly
|
|
used #cmakedefine instead of #define.
|
|
|
|
(ADB - 2018/10/04, HDFFV-10594)
|
|
|
|
- Add warning flags for Intel compilers
|
|
|
|
Identified Intel compiler specific warnings flags that should be used
|
|
instead of GNU flags.
|
|
|
|
(ADB - 2018/10/04, TRILABS-21)
|
|
|
|
- Add default rpath to targets
|
|
|
|
Default rpaths should be set in shared executables and
|
|
libraries to allow the use of loading dependent libraries
|
|
without requiring LD_LIBRARY_PATH to be set. The default
|
|
path should be relative using @rpath on osx and $ORIGIN
|
|
on linux. Windows is not affected.
|
|
|
|
(ADB - 2018/09/26, HDFFV-10594)
|
|
|
|
Library:
|
|
--------
|
|
- Allow pre-generated H5Tinit.c and H5make_libsettings.c to be used.
|
|
|
|
Rather than always running H5detect and generating H5Tinit.c and
|
|
H5make_libsettings.c, supply a location for those files.
|
|
|
|
(ADB - 2018/09/18, HDFFV-10332)
|
|
|
|
|
|
Bug Fixes since HDF5-1.10.3 release
|
|
==================================
|
|
|
|
Library
|
|
-------
|
|
- Allow H5detect and H5make_libsettings to take a file as an argument.
|
|
|
|
Rather than only writing to stdout, add a command argument to name
|
|
the file that H5detect and H5make_libsettings will use for output.
|
|
Without an argument, stdout is still used, so backwards compatibility
|
|
is maintained.
|
|
|
|
(ADB - 2018/09/05, HDFFV-9059)
|
|
|
|
- A bug was discovered in the parallel library where an application
|
|
would hang if a collective read/write of a chunked dataset occurred
|
|
when collective metadata reads were enabled and some of the ranks
|
|
had no selection in the dataset's dataspace. The ranks which had no
|
|
selection in the dataset's dataspace called H5D__chunk_addrmap() to
|
|
retrieve the lowest chunk address in the dataset. This is because we
|
|
require reads/writes to be performed in strictly non-decreasing order
|
|
of chunk address in the file.
|
|
|
|
When the chunk index used was a version 1 or 2 B-tree, these
|
|
non-participating ranks would issue a collective MPI_Bcast() call
|
|
that the participating ranks would not issue, causing the hang. Since
|
|
the non-participating ranks are not actually reading/writing anything,
|
|
the H5D__chunk_addrmap() call can be safely removed and the address used
|
|
for the read/write can be set to an arbitrary number (0 was chosen).
|
|
|
|
(JTH - 2018/08/25, HDFFV-10501)
|
|
|
|
Java Library:
|
|
----------------
|
|
- JNI native library dependencies
|
|
|
|
The build for the hdf5_java native library used the wrong
|
|
hdf5 target library for CMake builds. Correcting the hdf5_java
|
|
library to build with the shared hdf5 library required testing
|
|
paths to change also.
|
|
|
|
(ADB - 2018/08/31, HDFFV-10568)
|
|
|
|
- Java iterator callbacks
|
|
|
|
Change global callback object to a small stack structure in order
|
|
to fix a runtime crash. This crash was discovered when iterating
|
|
through a file with nested group members. The global variable
|
|
visit_callback is overwritten when recursion starts. When recursion
|
|
completes, visit_callback will be pointing to the wrong callback method.
|
|
|
|
(ADB - 2018/08/15, HDFFV-10536)
|
|
|
|
- Java HDFLibraryException class
|
|
|
|
Change parent class from Exception to RuntimeException.
|
|
|
|
(ADB - 2018/07/30, HDFFV-10534)
|
|
|
|
- JNI Read and Write
|
|
|
|
Refactored variable-length functions, H5DreadVL and H5AreadVL,
|
|
to correct dataset and attribute reads. New write functions,
|
|
H5DwriteVL and H5AwriteVL, are under construction.
|
|
|
|
(ADB - 2018/06/02, HDFFV-10519)
|
|
|
|
|
|
Supported Platforms
|
|
===================
|
|
|
|
Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
#1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
(ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
IBM XL C/C++ V13.1
|
|
IBM XL Fortran V15.1
|
|
|
|
Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
Version 4.9.3, Version 5.2.0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.0.098 Build 20160721
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc
|
|
(emu) Sun Fortran 95 8.6 SunOS_sparc
|
|
Sun C++ 5.12 SunOS_sparc
|
|
|
|
Windows 7 Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
|
|
Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
Visual Studio 2015 w/ Intel C, Fortran 2017 (cmake)
|
|
Visual Studio 2015 w/ MSMPI 8 (cmake)
|
|
|
|
Windows 10 Visual Studio 2015 w/ Intel Fortran 18 (cmake)
|
|
|
|
Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake)
|
|
Visual Studio 2017 w/ Intel Fortran 18 (cmake)
|
|
|
|
Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3
|
|
64-bit gfortran GNU Fortran (GCC) 5.2.0
|
|
(osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2
|
|
|
|
Mac OS Sierra 10.12.6 Apple LLVM version 8.1.0 (clang/clang++-802.0.42)
|
|
64-bit gfortran GNU Fortran (GCC) 7.1.0
|
|
(kite) Intel icc/icpc/ifort version 17.0.2
|
|
|
|
|
|
Tested Configuration Features Summary
|
|
=====================================
|
|
|
|
In the tables below
|
|
y = tested
|
|
n = not tested in this release
|
|
C = Cluster
|
|
W = Workstation
|
|
x = not working in this release
|
|
dna = does not apply
|
|
( ) = footnote appears below second table
|
|
<blank> = testing incomplete on this feature or platform
|
|
|
|
Platform C F90/ F90 C++ zlib SZIP
|
|
parallel F2003 parallel
|
|
Solaris2.11 32-bit n y/y n y y y
|
|
Solaris2.11 64-bit n y/n n y y y
|
|
Windows 7 y y/y n y y y
|
|
Windows 7 x64 y y/y y y y y
|
|
Windows 7 Cygwin n y/n n y y y
|
|
Windows 7 x64 Cygwin n y/n n y y y
|
|
Windows 10 y y/y n y y y
|
|
Windows 10 x64 y y/y n y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y
|
|
Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y
|
|
Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y
|
|
Mac OS Sierra 10.12.6 64-bit n y/y n y y y
|
|
CentOS 7.2 Linux 3.10.0 x86_64 PGI n y/y n y y y
|
|
CentOS 7.2 Linux 3.10.0 x86_64 GNU y y/y y y y y
|
|
CentOS 7.2 Linux 3.10.0 x86_64 Intel n y/y n y y y
|
|
Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y
|
|
|
|
|
|
Platform Shared Shared Shared Thread-
|
|
C libs F90 libs C++ libs safe
|
|
Solaris2.11 32-bit y y y y
|
|
Solaris2.11 64-bit y y y y
|
|
Windows 7 y y y y
|
|
Windows 7 x64 y y y y
|
|
Windows 7 Cygwin n n n y
|
|
Windows 7 x64 Cygwin n n n y
|
|
Windows 10 y y y y
|
|
Windows 10 x64 y y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit y n y y
|
|
Mac OS X Yosemite 10.10.5 64-bit y n y y
|
|
Mac OS X El Capitan 10.11.6 64-bit y n y y
|
|
Mac OS Sierra 10.12.6 64-bit y n y y
|
|
CentOS 7.2 Linux 3.10.0 x86_64 PGI y y y n
|
|
CentOS 7.2 Linux 3.10.0 x86_64 GNU y y y y
|
|
CentOS 7.2 Linux 3.10.0 x86_64 Intel y y y n
|
|
Linux 2.6.32-573.18.1.el6.ppc64 y y y n
|
|
|
|
Compiler versions for each platform are listed in the preceding
|
|
"Supported Platforms" table.
|
|
|
|
|
|
More Tested Platforms
|
|
=====================
|
|
The following platforms are not supported but have been tested for this release.
|
|
|
|
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(mayll/platypus) Version 4.4.7 20120313
|
|
Version 4.9.3, 5.3.0, 6.2.0
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86-64;
|
|
Version 17.10-0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.4.196 Build 20170411
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers
|
|
#1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
(jelly) with NAG Fortran Compiler Release 6.1(Tozai)
|
|
GCC Version 7.1.0
|
|
OpenMPI 3.0.0-GCC-7.2.0-2.29,
|
|
3.1.0-GCC-7.2.0-2.29
|
|
Intel(R) C (icc) and C++ (icpc) compilers
|
|
Version 17.0.0.098 Build 20160721
|
|
with NAG Fortran Compiler Release 6.1(Tozai)
|
|
|
|
Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0
|
|
#1 SMP x86_64 GNU/Linux
|
|
(moohan)
|
|
|
|
Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with
|
|
#1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1
|
|
(ostrich) and IBM XL Fortran for Linux, V15.1
|
|
|
|
Debian 8.4 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux
|
|
gcc, g++ (Debian 4.9.2-10) 4.9.2
|
|
GNU Fortran (Debian 4.9.2-10) 4.9.2
|
|
(cmake and autotools)
|
|
|
|
Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
|
|
gcc, g++ (GCC) 6.1.1 20160621
|
|
(Red Hat 6.1.1-3)
|
|
GNU Fortran (GCC) 6.1.1 20160621
|
|
(Red Hat 6.1.1-3)
|
|
(cmake and autotools)
|
|
|
|
Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux
|
|
gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2)
|
|
5.4.0 20160609
|
|
GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2)
|
|
5.4.0 20160609
|
|
(cmake and autotools)
|
|
|
|
|
|
Known Problems
|
|
==============
|
|
|
|
At present, metadata cache images may not be generated by parallel
|
|
applications. Parallel applications can read files with metadata cache
|
|
images, but since this is a collective operation, a deadlock is possible
|
|
if one or more processes do not participate.
|
|
|
|
Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29:
|
|
testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio)
|
|
t_shapesame (sscontig2)
|
|
t_pflush1/fails on exit
|
|
The first two tests fail attempting collective writes.
|
|
|
|
Known problems in previous releases can be found in the HISTORY*.txt files
|
|
in the HDF5 source. Please report any new problems found to
|
|
help@hdfgroup.org.
|
|
|
|
|
|
CMake vs. Autotools installations
|
|
=================================
|
|
While both build systems produce similar results, there are differences.
|
|
Each system produces the same set of folders on linux (only CMake works
|
|
on standard Windows); bin, include, lib and share. Autotools places the
|
|
COPYING and RELEASE.txt file in the root folder, CMake places them in
|
|
the share folder.
|
|
|
|
The bin folder contains the tools and the build scripts. Additionally, CMake
|
|
creates dynamic versions of the tools with the suffix "-shared". Autotools
|
|
installs one set of tools depending on the "--enable-shared" configuration
|
|
option.
|
|
build scripts
|
|
-------------
|
|
Autotools: h5c++, h5cc, h5fc
|
|
CMake: h5c++, h5cc, h5hlc++, h5hlcc
|
|
|
|
The include folder holds the header files and the fortran mod files. CMake
|
|
places the fortran mod files into separate shared and static subfolders,
|
|
while Autotools places one set of mod files into the include folder. Because
|
|
CMake produces a tools library, the header files for tools will appear in
|
|
the include folder.
|
|
|
|
The lib folder contains the library files, and CMake adds the pkgconfig
|
|
subfolder with the hdf5*.pc files used by the bin/build scripts created by
|
|
the CMake build. CMake separates the C interface code from the fortran code by
|
|
creating C-stub libraries for each Fortran library. In addition, only CMake
|
|
installs the tools library. The names of the szip libraries are different
|
|
between the build systems.
|
|
|
|
The share folder will have the most differences because CMake builds include
|
|
a number of CMake specific files for support of CMake's find_package and support
|
|
for the HDF5 Examples CMake project.
|
|
|
|
%%%%1.10.3%%%%
|
|
|
|
HDF5 version 1.10.3 released on 2018-08-21
|
|
================================================================================
|
|
|
|
|
|
INTRODUCTION
|
|
|
|
This document describes the differences between this release and the previous
|
|
HDF5 release. It contains information on the platforms tested and known
|
|
problems in this release. For more details check the HISTORY*.txt files in the
|
|
HDF5 source.
|
|
|
|
Note that documentation in the links below will be updated at the time of each
|
|
final release.
|
|
|
|
Links to HDF5 documentation can be found on The HDF5 web page:
|
|
|
|
https://portal.hdfgroup.org/display/HDF5/HDF5
|
|
|
|
The official HDF5 releases can be obtained from:
|
|
|
|
https://www.hdfgroup.org/downloads/hdf5/
|
|
|
|
Changes from Release to Release and New Features in the HDF5-1.10.x release series
|
|
can be found at:
|
|
|
|
https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide
|
|
|
|
If you have any questions or comments, please send them to the HDF Help Desk:
|
|
|
|
help@hdfgroup.org
|
|
|
|
|
|
CONTENTS
|
|
|
|
- New Features
|
|
- Bug Fixes since HDF5-1.10.2
|
|
- Supported Platforms
|
|
- Tested Configuration Features Summary
|
|
- More Tested Platforms
|
|
- Known Problems
|
|
- CMake vs. Autotools installations
|
|
|
|
|
|
New Features
|
|
============
|
|
|
|
Library
|
|
-------
|
|
- Moved the H5DOread/write_chunk() API calls to H5Dread/write_chunk()
|
|
|
|
The functionality of the direct chunk I/O calls in the high-level
|
|
library has been moved to the H5D package in the main library. This
|
|
will allow using those functions without building the high-level
|
|
library. The parameters and functionality of the H5D calls are
|
|
identical to the H5DO calls.
|
|
|
|
The original H5DO high-level API calls have been retained, though
|
|
they are now just wrappers for the H5D calls. They are marked as
|
|
deprecated and are only available when the library is built with
|
|
deprecated functions. New code should use the H5D calls for this
|
|
reason.
|
|
|
|
As a part of this work, the following symbols from H5Dpublic.h are no
|
|
longer used:
|
|
|
|
H5D_XFER_DIRECT_CHUNK_WRITE_FLAG_NAME
|
|
H5D_XFER_DIRECT_CHUNK_WRITE_FILTERS_NAME
|
|
H5D_XFER_DIRECT_CHUNK_WRITE_OFFSET_NAME
|
|
H5D_XFER_DIRECT_CHUNK_WRITE_DATASIZE_NAME
|
|
H5D_XFER_DIRECT_CHUNK_READ_FLAG_NAME
|
|
H5D_XFER_DIRECT_CHUNK_READ_OFFSET_NAME
|
|
H5D_XFER_DIRECT_CHUNK_READ_FILTERS_NAME
|
|
|
|
And properties with these names are no longer stored in the dataset
|
|
transfer property lists. The symbols are still defined in H5Dpublic.h,
|
|
but only when the library is built with deprecated symbols.
|
|
|
|
(DER - 2018/05/04)
|
|
|
|
Configuration:
|
|
-------------
|
|
- Add missing USE_110_API_DEFAULT option.
|
|
|
|
Option USE_110_API_DEFAULT sets the default version of
|
|
versioned APIs. The bin/makevers perl script did not set
|
|
the maxidx variable correctly when the 1.10 branch was
|
|
created. This caused the versioning process to always use
|
|
the latest version of any API.
|
|
|
|
(ADB - 2018/08/17, HDFFV-10552)
|
|
|
|
- Added configuration checks for the following MPI functions:
|
|
|
|
MPI_Mprobe - Used for the Parallel Compression feature
|
|
MPI_Imrecv - Used for the Parallel Compression feature
|
|
|
|
MPI_Get_elements_x - Used for the "big Parallel I/O" feature
|
|
MPI_Type_size_x - Used for the "big Parallel I/O" feature
|
|
|
|
(JTH - 2018/08/02, HDFFV-10512)
|
|
|
|
- Added section to the libhdf5.settings file to indicate
|
|
the status of the Parallel Compression and "big Parallel I/O"
|
|
features.
|
|
|
|
(JTH - 2018/08/02, HDFFV-10512)
|
|
|
|
- Add option to execute swmr shell scripts from CMake.
|
|
|
|
Option TEST_SHELL_SCRIPTS redirects processing into a
|
|
separate ShellTests.cmake file for UNIX types. The tests
|
|
execute the shell scripts if a SH program is found.
|
|
|
|
(ADB - 2018/07/16)
|
|
|
|
|
|
C++ Library:
|
|
------------
|
|
- New wrappers
|
|
|
|
Added the following items:
|
|
|
|
+ Class DSetAccPropList for the dataset access property list.
|
|
|
|
+ Wrapper for H5Dget_access_plist to class DataSet
|
|
// Gets the access property list of this dataset.
|
|
DSetAccPropList getAccessPlist() const;
|
|
|
|
+ Wrappers for H5Pset_chunk_cache and H5Pget_chunk_cache to class DSetAccPropList
|
|
// Sets the raw data chunk cache parameters.
|
|
void setChunkCache(size_t rdcc_nslots, size_t rdcc_nbytes, double rdcc_w0)
|
|
|
|
// Retrieves the raw data chunk cache parameters.
|
|
void getChunkCache(size_t &rdcc_nslots, size_t &rdcc_nbytes, double &rdcc_w0)
|
|
|
|
+ New operator!= to class DataType (HDFFV-10472)
|
|
// Determines whether two datatypes are not the same.
|
|
bool operator!=(const DataType& compared_type)
|
|
|
|
+ Wrappers for H5Oget_info2, H5Oget_info_by_name2, and H5Oget_info_by_idx2
|
|
(HDFFV-10458)
|
|
|
|
// Retrieves information about an HDF5 object.
|
|
void getObjinfo(H5O_info_t& objinfo, unsigned fields = H5O_INFO_BASIC) const;
|
|
|
|
// Retrieves information about an HDF5 object, given its name.
|
|
void getObjinfo(const char* name, H5O_info_t& objinfo,
|
|
unsigned fields = H5O_INFO_BASIC,
|
|
const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) const;
|
|
void getObjinfo(const H5std_string& name, H5O_info_t& objinfo,
|
|
unsigned fields = H5O_INFO_BASIC,
|
|
const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) const;
|
|
|
|
// Retrieves information about an HDF5 object, given its index.
|
|
void getObjinfo(const char* grp_name, H5_index_t idx_type,
|
|
H5_iter_order_t order, hsize_t idx, H5O_info_t& objinfo,
|
|
unsigned fields = H5O_INFO_BASIC,
|
|
const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) const;
|
|
void getObjinfo(const H5std_string& grp_name, H5_index_t idx_type,
|
|
H5_iter_order_t order, hsize_t idx, H5O_info_t& objinfo,
|
|
unsigned fields = H5O_INFO_BASIC,
|
|
const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) const;
|
|
|
|
(BMR - 2018/07/22, HDFFV-10150, HDFFV-10458, HDFFV-1047)
|
|
|
|
|
|
Java Library:
|
|
----------------
|
|
- Java HDFLibraryException class
|
|
|
|
Change parent class from Exception to RuntimeException.
|
|
|
|
(ADB - 2018/07/30, HDFFV-10534)
|
|
|
|
- JNI Read and Write
|
|
|
|
Refactored variable-length functions, H5DreadVL and H5AreadVL,
|
|
to correct dataset and attribute reads. New write functions,
|
|
H5DwriteVL and H5AwriteVL, are under construction.
|
|
|
|
(ADB - 2018/06/02, HDFFV-10519)
|
|
|
|
|
|
Bug Fixes since HDF5-1.10.2 release
|
|
==================================
|
|
|
|
Library
|
|
-------
|
|
- Performance issue with H5Oget_info
|
|
|
|
H5Oget_info family of routines retrieves information for an object such
|
|
as object type, access time, number of attributes, and storage space etc.
|
|
Retrieving all such information regardless is an overkill and causes
|
|
performance issue when doing so for many objects.
|
|
|
|
Add an additional parameter "fields" to the the H5Oget_info family of routines
|
|
indicating the type of information to be retrieved. The same is done to
|
|
the H5Ovisit family of routines which recursively visits an object
|
|
returning object information in a callback function. Both sets of routines
|
|
are versioned and the corresponding compatibility macros are added.
|
|
|
|
The version 2 names of the two sets of routines are:
|
|
(1) H5Oget_info2, H5Oget_info_by_idx2, H5Oget_info_by_name2
|
|
(2) H5Ovisit2, H5Ovisit_by_name2
|
|
|
|
(VC - 2018/08/15, HDFFV-10180)
|
|
|
|
- Test failure due to metadata size in test/vds.c
|
|
|
|
The size of metadata from test_api_get_ex_dcpl() in test/vds.c is not as expected
|
|
because the latest format should be used when encoding the layout for VDS.
|
|
|
|
Set the latest format in a temporary fapl and pass the setting to the routines that
|
|
encode the dataset selection for VDS.
|
|
|
|
(VC - 2018/08/14 HDFFV-10469)
|
|
|
|
- Java HDF5LibraryException class
|
|
|
|
The error minor and major values would be lost after the
|
|
constructor executed.
|
|
|
|
Created two local class variables to hold the values obtained during
|
|
execution of the constructor. Refactored the class functions to retrieve
|
|
the class values rather then calling the native functions.
|
|
The native functions were renamed and called only during execution
|
|
of the constructor.
|
|
Added error checking to calling class constructors in JNI classes.
|
|
|
|
(ADB - 2018/08/06, HDFFV-10544)
|
|
|
|
- Added checks of the defined MPI_VERSION to guard against usage of
|
|
MPI-3 functions in the Parallel Compression and "big Parallel I/O"
|
|
features when HDF5 is built with MPI-2. Previously, the configure
|
|
step would pass but the build itself would fail when it could not
|
|
locate the MPI-3 functions used.
|
|
|
|
As a result of these new checks, HDF5 can again be built with MPI-2,
|
|
but the Parallel Compression feature will be disabled as it relies
|
|
on the MPI-3 functions used.
|
|
|
|
(JTH - 2018/08/02, HDFFV-10512)
|
|
|
|
- User's patches: CVEs
|
|
|
|
The following patches have been applied:
|
|
|
|
CVE-2018-11202 - NULL pointer dereference was discovered in
|
|
H5S_hyper_make_spans in H5Shyper.c (HDFFV-10476)
|
|
https://security-tracker.debian.org/tracker/CVE-2018-11202
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11202
|
|
|
|
CVE-2018-11203 - A division by zero was discovered in
|
|
H5D__btree_decode_key in H5Dbtree.c (HDFFV-10477)
|
|
https://security-tracker.debian.org/tracker/CVE-2018-11203
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11203
|
|
|
|
CVE-2018-11204 - A NULL pointer dereference was discovered in
|
|
H5O__chunk_deserialize in H5Ocache.c (HDFFV-10478)
|
|
https://security-tracker.debian.org/tracker/CVE-2018-11204
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11204
|
|
|
|
CVE-2018-11206 - An out of bound read was discovered in
|
|
H5O_fill_new_decode and H5O_fill_old_decode in H5Ofill.c
|
|
(HDFFV-10480)
|
|
https://security-tracker.debian.org/tracker/CVE-2018-11206
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11206
|
|
|
|
CVE-2018-11207 - A division by zero was discovered in
|
|
H5D__chunk_init in H5Dchunk.c (HDFFV-10481)
|
|
https://security-tracker.debian.org/tracker/CVE-2018-11207
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11207
|
|
|
|
(BMR - 2018/7/22, PR#s: 1134 and 1139,
|
|
HDFFV-10476, HDFFV-10477, HDFFV-10478, HDFFV-10480, HDFFV-10481)
|
|
|
|
- H5Adelete
|
|
|
|
H5Adelete failed when deleting the last "large" attribute that
|
|
is stored densely via fractal heap/v2 b-tree.
|
|
|
|
After removing the attribute, update the ainfo message. If the
|
|
number of attributes goes to zero, remove the message.
|
|
|
|
(VC - 2018/07/20, HDFFV-9277)
|
|
|
|
- A bug was discovered in the parallel library which caused partial
|
|
parallel reads of filtered datasets to return incorrect data. The
|
|
library used the incorrect dataspace for each chunk read, causing
|
|
the selection used in each chunk to be wrong.
|
|
|
|
The bug was not caught during testing because all of the current
|
|
tests which do parallel reads of filtered data read all of the data
|
|
using an H5S_ALL selection. Several tests were added which exercise
|
|
partial parallel reads.
|
|
|
|
(JTH - 2018/07/16, HDFFV-10467)
|
|
|
|
- A bug was discovered in the parallel library which caused parallel
|
|
writes of filtered datasets to trigger an assertion failure in the
|
|
file free space manager.
|
|
|
|
This occurred when the filter used caused chunks to repeatedly shrink
|
|
and grow over the course of several dataset writes. The previous chunk
|
|
information, such as the size of the chunk and the offset in the file,
|
|
was being cached and not updated after each write, causing the next write
|
|
to the chunk to retrieve the incorrect cached information and run into
|
|
issues when reallocating space in the file for the chunk.
|
|
|
|
(JTH - 2018/07/16, HDFFV-10509)
|
|
|
|
- A bug was discovered in the parallel library which caused the
|
|
H5D__mpio_array_gatherv() function to allocate too much memory.
|
|
|
|
When the function is called with the 'allgather' parameter set
|
|
to a non-true value, the function will receive data from all MPI
|
|
ranks and gather it to the single rank specied by the 'root'
|
|
parameter. However, the bug in the function caused memory for
|
|
the received data to be allocated on all MPI ranks, not just the
|
|
singular rank specified as the receiver. In some circumstances,
|
|
this would cause an application to fail due to the large amounts
|
|
of memory being allocated.
|
|
|
|
(JTH - 2018/07/16, HDFFV-10467)
|
|
|
|
- Error checks in h5stat and when decoding messages
|
|
|
|
h5stat exited with seg fault/core dumped when
|
|
errors are encountered in the internal library.
|
|
|
|
Add error checks and --enable-error-stack option to h5stat.
|
|
Add range checks when decoding messages: old fill value, old
|
|
layout and refcount.
|
|
|
|
(VC - 2018/07/11, HDFFV-10333)
|
|
|
|
- If an HDF5 file contains a malformed compound datatype with a
|
|
suitably large offset, the type conversion code can run off
|
|
the end of the type conversion buffer, causing a segmentation
|
|
fault.
|
|
|
|
This issue was reported to The HDF Group as issue #CVE-2017-17507.
|
|
|
|
NOTE: The HDF5 C library cannot produce such a file. This condition
|
|
should only occur in a corrupt (or deliberately altered) file
|
|
or a file created by third-party software.
|
|
|
|
THE HDF GROUP WILL NOT FIX THIS BUG AT THIS TIME
|
|
|
|
Fixing this problem would involve updating the publicly visible
|
|
H5T_conv_t function pointer typedef and versioning the API calls
|
|
which use it. We normally only modify the public API during
|
|
major releases, so this bug will not be fixed at this time.
|
|
|
|
(DER - 2018/02/26, HDFFV-10356)
|
|
|
|
|
|
Configuration
|
|
-------------
|
|
- Applied patches to address Cywin build issues
|
|
|
|
There were three issues for Cygwin builds:
|
|
- Shared libs were not built.
|
|
- The -std=c99 flag caused a SIG_SETMASK undeclared error.
|
|
- Undefined errors when buildbing test shared libraries.
|
|
|
|
Patches to address these issues were received and incorporated in this version.
|
|
|
|
(LRK - 2018/07/18, HDFFV-10475)
|
|
|
|
- The --enable-debug/production configure flags are listed as 'deprecated'
|
|
when they should really be listed as 'removed'.
|
|
|
|
In the autotools overhaul several years ago, we removed these flags and
|
|
implemented a new --enable-build-mode= flag. This was done because we
|
|
changed the semantics of the modes and didn't want users to silently
|
|
be exposed to them. The newer system is also more flexible and us to
|
|
add other modes (like 'clean').
|
|
|
|
The --enable-debug/production flags are now listed as removed.
|
|
|
|
(DER - 2018/05/31, HDFFV-10505)
|
|
|
|
- Moved the location of gcc attribute.
|
|
|
|
The gcc attribute(no_sanitize), named as the macro HDF_NO_UBSAN,
|
|
was located after the function name. Builds with GCC 7 did not
|
|
indicate any problem, but GCC 8 issued errors. Moved the
|
|
attribute before the function name, as required.
|
|
|
|
(ADB - 2018/05/22, HDFFV-10473)
|
|
|
|
- Reworked java test suite into individual JUnit tests.
|
|
|
|
Testing the whole suite of java unit tests in a single JUnit run
|
|
made it difficult to determine actual failures when tests would fail.
|
|
Running each file set of tests individually, allows individual failures
|
|
to be diagnosed easier. A side benefit is that tests for optional components
|
|
of the library can be disabled if not configured.
|
|
|
|
(ADB - 2018/05/16, HDFFV-9739)
|
|
|
|
- Converted CMake global commands ADD_DEFINITIONS and INCLUDE_DIRECTORIES
|
|
to use target_* type commands. This change modernizes the CMake usage
|
|
in the HDF5 library.
|
|
|
|
In addition, there is the intention to convert to generator expressions,
|
|
where possible. The exception is Fortran FLAGS on Windows Visual Studio.
|
|
The HDF macros TARGET_C_PROPERTIES and TARGET_FORTRAN_PROPERTIES have
|
|
been removed with this change in usage.
|
|
|
|
The additional language (C++ and Fortran) checks have also been localized
|
|
to only be checked when that language is enabled.
|
|
|
|
(ADB - 2018/05/08)
|
|
|
|
|
|
Performance
|
|
-------------
|
|
- Revamped internal use of DXPLs, improving performance
|
|
|
|
(QAK - 2018/05/20)
|
|
|
|
|
|
Fortran
|
|
--------
|
|
- Fixed issue with h5fget_obj_count_f and using a file id of H5F_OBJ_ALL_F not
|
|
returning the correct count.
|
|
|
|
(MSB - 2018/5/15, HDFFV-10405)
|
|
|
|
|
|
C++ APIs
|
|
--------
|
|
- Adding default arguments to existing functions
|
|
|
|
Added the following items:
|
|
+ Two more property list arguments are added to H5Location::createDataSet:
|
|
const DSetAccPropList& dapl = DSetAccPropList::DEFAULT
|
|
const LinkCreatPropList& lcpl = LinkCreatPropList::DEFAULT
|
|
|
|
+ One more property list argument is added to H5Location::openDataSet:
|
|
const DSetAccPropList& dapl = DSetAccPropList::DEFAULT
|
|
|
|
(BMR - 2018/07/21, PR# 1146)
|
|
|
|
- Improvement C++ documentation
|
|
|
|
Replaced the table in main page of the C++ documentation from mht to htm format
|
|
for portability.
|
|
|
|
(BMR - 2018/07/17, PR# 1141)
|
|
|
|
|
|
Supported Platforms
|
|
===================
|
|
|
|
Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
#1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
(ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
IBM XL C/C++ V13.1
|
|
IBM XL Fortran V15.1
|
|
|
|
Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
Version 4.9.3, Version 5.2.0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.0.098 Build 20160721
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc
|
|
(emu) Sun Fortran 95 8.6 SunOS_sparc
|
|
Sun C++ 5.12 SunOS_sparc
|
|
|
|
Windows 7 Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
|
|
Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
Visual Studio 2015 w/ Intel C, Fortran 2017 (cmake)
|
|
Visual Studio 2015 w/ MSMPI 8 (cmake)
|
|
|
|
Windows 10 Visual Studio 2015 w/ Intel Fortran 18 (cmake)
|
|
|
|
Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake)
|
|
Visual Studio 2017 w/ Intel Fortran 18 (cmake)
|
|
|
|
Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3
|
|
64-bit gfortran GNU Fortran (GCC) 5.2.0
|
|
(osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2
|
|
|
|
Mac OS Sierra 10.12.6 Apple LLVM version 8.1.0 (clang/clang++-802.0.42)
|
|
64-bit gfortran GNU Fortran (GCC) 7.1.0
|
|
(swallow/kite) Intel icc/icpc/ifort version 17.0.2
|
|
|
|
Tested Configuration Features Summary
|
|
=====================================
|
|
|
|
In the tables below
|
|
y = tested
|
|
n = not tested in this release
|
|
C = Cluster
|
|
W = Workstation
|
|
x = not working in this release
|
|
dna = does not apply
|
|
( ) = footnote appears below second table
|
|
<blank> = testing incomplete on this feature or platform
|
|
|
|
Platform C F90/ F90 C++ zlib SZIP
|
|
parallel F2003 parallel
|
|
Solaris2.11 32-bit n y/y n y y y
|
|
Solaris2.11 64-bit n y/n n y y y
|
|
Windows 7 y y/y n y y y
|
|
Windows 7 x64 y y/y y y y y
|
|
Windows 7 Cygwin n y/n n y y y
|
|
Windows 7 x64 Cygwin n y/n n y y y
|
|
Windows 10 y y/y n y y y
|
|
Windows 10 x64 y y/y n y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y
|
|
Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y
|
|
Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y
|
|
Mac OS Sierra 10.12.6 64-bit n y/y n y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 PGI n y/y n y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 GNU y y/y y y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 Intel n y/y n y y y
|
|
Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y
|
|
|
|
|
|
Platform Shared Shared Shared Thread-
|
|
C libs F90 libs C++ libs safe
|
|
Solaris2.11 32-bit y y y y
|
|
Solaris2.11 64-bit y y y y
|
|
Windows 7 y y y y
|
|
Windows 7 x64 y y y y
|
|
Windows 7 Cygwin n n n y
|
|
Windows 7 x64 Cygwin n n n y
|
|
Windows 10 y y y y
|
|
Windows 10 x64 y y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit y n y y
|
|
Mac OS X Yosemite 10.10.5 64-bit y n y y
|
|
Mac OS X El Capitan 10.11.6 64-bit y n y y
|
|
Mac OS Sierra 10.12.6 64-bit y n y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 PGI y y y n
|
|
CentOS 7.2 Linux 2.6.32 x86_64 GNU y y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 Intel y y y n
|
|
Linux 2.6.32-573.18.1.el6.ppc64 y y y n
|
|
|
|
Compiler versions for each platform are listed in the preceding
|
|
"Supported Platforms" table.
|
|
|
|
|
|
More Tested Platforms
|
|
=====================
|
|
The following platforms are not supported but have been tested for this release.
|
|
|
|
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(mayll/platypus) Version 4.4.7 20120313
|
|
Version 4.9.3, 5.3.0, 6.2.0
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86-64;
|
|
Version 17.10-0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.4.196 Build 20170411
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers
|
|
#1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
(jelly) with NAG Fortran Compiler Release 6.1(Tozai)
|
|
GCC Version 7.1.0
|
|
OpenMPI 3.0.0-GCC-7.2.0-2.29,
|
|
3.1.0-GCC-7.2.0-2.29
|
|
Intel(R) C (icc) and C++ (icpc) compilers
|
|
Version 17.0.0.098 Build 20160721
|
|
with NAG Fortran Compiler Release 6.1(Tozai)
|
|
|
|
Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0
|
|
#1 SMP x86_64 GNU/Linux
|
|
(moohan)
|
|
|
|
Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with
|
|
#1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1
|
|
(ostrich) and IBM XL Fortran for Linux, V15.1
|
|
|
|
Debian 8.4 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux
|
|
gcc, g++ (Debian 4.9.2-10) 4.9.2
|
|
GNU Fortran (Debian 4.9.2-10) 4.9.2
|
|
(cmake and autotools)
|
|
|
|
Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
|
|
gcc, g++ (GCC) 6.1.1 20160621
|
|
(Red Hat 6.1.1-3)
|
|
GNU Fortran (GCC) 6.1.1 20160621
|
|
(Red Hat 6.1.1-3)
|
|
(cmake and autotools)
|
|
|
|
Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux
|
|
gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2)
|
|
5.4.0 20160609
|
|
GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2)
|
|
5.4.0 20160609
|
|
(cmake and autotools)
|
|
|
|
|
|
Known Problems
|
|
==============
|
|
|
|
At present, metadata cache images may not be generated by parallel
|
|
applications. Parallel applications can read files with metadata cache
|
|
images, but since this is a collective operation, a deadlock is possible
|
|
if one or more processes do not participate.
|
|
|
|
Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29:
|
|
testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio)
|
|
t_shapesame (sscontig2)
|
|
t_pflush1/fails on exit
|
|
The first two tests fail attempting collective writes.
|
|
|
|
Known problems in previous releases can be found in the HISTORY*.txt files
|
|
in the HDF5 source. Please report any new problems found to
|
|
help@hdfgroup.org.
|
|
|
|
|
|
CMake vs. Autotools installations
|
|
=================================
|
|
While both build systems produce similar results, there are differences.
|
|
Each system produces the same set of folders on linux (only CMake works
|
|
on standard Windows); bin, include, lib and share. Autotools places the
|
|
COPYING and RELEASE.txt file in the root folder, CMake places them in
|
|
the share folder.
|
|
|
|
The bin folder contains the tools and the build scripts. Additionally, CMake
|
|
creates dynamic versions of the tools with the suffix "-shared". Autotools
|
|
installs one set of tools depending on the "--enable-shared" configuration
|
|
option.
|
|
build scripts
|
|
-------------
|
|
Autotools: h5c++, h5cc, h5fc
|
|
CMake: h5c++, h5cc, h5hlc++, h5hlcc
|
|
|
|
The include folder holds the header files and the fortran mod files. CMake
|
|
places the fortran mod files into separate shared and static subfolders,
|
|
while Autotools places one set of mod files into the include folder. Because
|
|
CMake produces a tools library, the header files for tools will appear in
|
|
the include folder.
|
|
|
|
The lib folder contains the library files, and CMake adds the pkgconfig
|
|
subfolder with the hdf5*.pc files used by the bin/build scripts created by
|
|
the CMake build. CMake separates the C interface code from the fortran code by
|
|
creating C-stub libraries for each Fortran library. In addition, only CMake
|
|
installs the tools library. The names of the szip libraries are different
|
|
between the build systems.
|
|
|
|
The share folder will have the most differences because CMake builds include
|
|
a number of CMake specific files for support of CMake's find_package and support
|
|
for the HDF5 Examples CMake project.
|
|
|
|
|
|
%%%%1.10.2%%%%
|
|
|
|
HDF5 version 1.10.2 released on 2018-03-29
|
|
================================================================================
|
|
|
|
|
|
INTRODUCTION
|
|
|
|
This document describes the differences between this release and the previous
|
|
HDF5 release. It contains information on the platforms tested and known
|
|
problems in this release. For more details check the HISTORY*.txt files in the
|
|
HDF5 source.
|
|
|
|
Note that documentation in the links below will be updated at the time of each
|
|
final release.
|
|
|
|
Links to HDF5 documentation can be found on The HDF5 web page:
|
|
|
|
https://portal.hdfgroup.org/display/HDF5/HDF5
|
|
|
|
The official HDF5 releases can be obtained from:
|
|
|
|
https://www.hdfgroup.org/downloads/hdf5/
|
|
|
|
Changes from Release to Release and New Features in the HDF5-1.10.x release series
|
|
can be found at:
|
|
|
|
https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide
|
|
|
|
If you have any questions or comments, please send them to the HDF Help Desk:
|
|
|
|
help@hdfgroup.org
|
|
|
|
|
|
CONTENTS
|
|
|
|
- New Features
|
|
- Support for new platforms and languages
|
|
- Bug Fixes since HDF5-1.10.1
|
|
- Supported Platforms
|
|
- Tested Configuration Features Summary
|
|
- More Tested Platforms
|
|
- Known Problems
|
|
|
|
|
|
New Features
|
|
============
|
|
|
|
Configuration and Build Systems:
|
|
--------------------------------
|
|
- CMake builds
|
|
--------------
|
|
|
|
- Changed minimum CMake required version to 3.10.
|
|
|
|
This change removed the need to support a copy of the FindMPI.cmake module,
|
|
which has been removed, along with its subfolder in the config/cmake_ext_mod
|
|
location.
|
|
|
|
(ADB - 2018/03/09)
|
|
|
|
- Added pkg-config file generation
|
|
|
|
Added pkg-config file generation for the C, C++, HL, and HL C++ libraries.
|
|
In addition, builds on Linux will create h5cc, h5c++, h5hlcc, and h5hlc++ scripts in the bin
|
|
directory that use the pkg-config files. The scripts can be used to build HDF5 C and C++
|
|
applications (i.e, similar to the compiler scripts produced by the Autotools builds).
|
|
|
|
(ADB - 2018/03/08, HDFFV-4359)
|
|
|
|
- Refactored use of CMAKE_BUILD_TYPE for new variable, which understands
|
|
the type of generator in use.
|
|
|
|
Added new configuration macros to use new HDF_BUILD_TYPE variable. This
|
|
variable is set correctly for the type of generator being used for the build.
|
|
|
|
(ADB - 2018/01/08, HDFFV-10385, HDFFV-10296)
|
|
|
|
- Autotools builds
|
|
------------------
|
|
|
|
- Removed version-specific gcc/gfortran flags for version 4.0 (inclusive)
|
|
and earlier.
|
|
|
|
The config/gnu-flags file, which is sourced as a part of the configure
|
|
process, adds version-specific flags for use when building HDF5. Most of
|
|
these flags control warnings and do not affect the final product.
|
|
|
|
Flags for older versions of the compiler were consolidated into the
|
|
common flags section. Moving these flags simplifies maintenance of
|
|
the file.
|
|
|
|
The upshot of this is that building with ancient versions of gcc
|
|
(<= 4.0) will possibly no longer work without hand-hacking the file
|
|
to remove the flags not understood by that version of the compiler.
|
|
Nothing should change when building with gcc >= 4.1.
|
|
|
|
(DER - 2017/05/31, HDFFV-9937)
|
|
|
|
- -fno-omit-frame-pointer was added when building with debugging symbols
|
|
enabled.
|
|
|
|
Debugging symbols can be enabled independently of the overall build
|
|
mode in both the autotools and CMake. This allows (limited) debugging
|
|
of optimized code. Since many debuggers rely on the frame pointer,
|
|
we've disabled this optimization when debugging symbols are requested
|
|
(e.g.: via building with --enable-symbols).
|
|
|
|
(DER - 2017/05/31, HDFFV-10226)
|
|
|
|
|
|
Library:
|
|
--------
|
|
- Added an enumerated value to H5F_libver_t for H5Pset_libver_bounds().
|
|
|
|
Currently, the library defines two values for H5F_libver_t and supports
|
|
only two pairs of (low, high) combinations as derived from these values.
|
|
Thus the bounds setting via H5Pset_libver_bounds() is rather restricted.
|
|
|
|
Added an enumerated value (H5F_LIBVER_V18) to H5F_libver_t and
|
|
H5Pset_libver_bounds() now supports five pairs of (low, high) combinations
|
|
as derived from these values. This addition provides the user more
|
|
flexibility in setting bounds for object creation.
|
|
|
|
(VC - 2018/03/14)
|
|
|
|
- Added prefix option to VDS files.
|
|
|
|
Currently, VDS source files must be in the active directory to be
|
|
found by the virtual file. Adding the option of a prefix to be set
|
|
on the virtual file, using a data access property list (DAPL),
|
|
allows the source files to locate at an absolute or relative path
|
|
to the virtual file.
|
|
Private utility functions in H5D and H5L packages merged into single
|
|
function in H5F package.
|
|
|
|
New public APIs:
|
|
herr_t H5Pset_virtual_prefix(hid_t dapl_id, const char* prefix);
|
|
ssize_t H5Pget_virtual_prefix(hid_t dapl_id, char* prefix /*out*/, size_t size);
|
|
The prefix can also be set with an environment variable, HDF5_VDS_PREFIX.
|
|
|
|
(ADB - 2017/12/12, HDFFV-9724, HDFFV-10361)
|
|
|
|
- H5FDdriver_query() API call added to the C library.
|
|
|
|
This new library call allows the user to query a virtual file driver
|
|
(VFD) for the feature flags it supports (listed in H5FDpublic.h).
|
|
This can be useful to determine if a VFD supports SWMR, for example.
|
|
|
|
Note that some VFDs have feature flags that may only be present
|
|
after a file has been created or opened (e.g.: the core VFD will
|
|
have the H5FD_FEAT_POSIX_COMPAT_HANDLE flag set if the backing
|
|
store is switched on). Since the new API call queries a generic VFD
|
|
unassociated with a file, these flags will never be returned.
|
|
|
|
(DER - 2017/05/31, HDFFV-10215)
|
|
|
|
- H5FD_FEAT_DEFAULT_VFD_COMPATIBLE VFD feature flag added to the C library.
|
|
|
|
This new feature flag indicates that the VFD is compatible with the
|
|
default VFD. VFDs that set this flag create single files that follow
|
|
the canonical HDF5 file format.
|
|
|
|
(DER - 2017/05/31, HDFFV-10214)
|
|
|
|
- The H5I_REFERENCE value in the H5I_type_t enum (defined in H5Ipublic.h)
|
|
has been marked as deprecated.
|
|
|
|
This ID type value is not used in the C library. i.e.: There are no
|
|
hid_t values that are of ID type H5I_REFERENCE.
|
|
|
|
This enum value will be removed in a future major version of the library.
|
|
The code will remain unchanged in the HDF5 1.10.x releases and branches.
|
|
|
|
(DER - 2017/04/05, HDFFV-10252)
|
|
|
|
|
|
Parallel Library:
|
|
-----------------
|
|
- Enabled compression for parallel applications.
|
|
|
|
With this release parallel applications can create and write compressed
|
|
datasets (or the datasets with the filters such as Fletcher32 applied).
|
|
|
|
(EIP - 2018/03/29)
|
|
|
|
- Addressed slow file close on some Lustre file systems.
|
|
|
|
Slow file close has been reported on some Lustre file systems.
|
|
While the ultimate cause is not understood fully, the proximate
|
|
cause appears to be long delays in MPI_File_set_size() calls at
|
|
file close and flush.
|
|
|
|
To minimize this problem pending a definitive diagnosis and fix,
|
|
PHDF5 has been modified to avoid MPI_File_set_size() calls when
|
|
possible. This is done by comparing the library's EOA (End of
|
|
Allocation) with the file systems EOF, and skipping the
|
|
MPI_File_set_size() call if the two match.
|
|
|
|
(JRM - 2018/03/29)
|
|
|
|
- Optimized parallel open/location of the HDF5 super-block.
|
|
|
|
Previous releases of PHDF5 required all parallel ranks to
|
|
search for the HDF5 superblock signature when opening the
|
|
file. As this is accomplished more or less as a synchronous
|
|
operation, a large number of processes can experience a
|
|
slowdown in the file open due to filesystem contention.
|
|
|
|
As a first step in improving the startup/file-open performance,
|
|
we allow MPI rank 0 of the associated MPI communicator to locate
|
|
the base offset of the super-block and then broadcast that result
|
|
to the remaining ranks in the parallel group. Note that this
|
|
approach is utilized ONLY during file opens which employ the MPIO
|
|
file driver in HDF5 by previously having called H5Pset_fapl_mpio().
|
|
|
|
HDF5 parallel file operations which do not employ multiple ranks
|
|
e.g. specifiying MPI_COMM_SELF (whose MPI_Comm_size == 1)
|
|
as opposed to MPI_COMM_WORLD, will not be affected by this
|
|
optimization. Conversely, parallel file operations on subgroups
|
|
of MPI_COMM_WORLD are allowed to be run in parallel with each
|
|
subgroup operating as an independant collection of processes.
|
|
|
|
(RAW - 2017/10/10, HDFFV-10294)
|
|
|
|
- Added large (>2GB) MPI-IO transfers.
|
|
|
|
Previous releases of PHDF5 would fail when attempting to
|
|
read or write greater than 2GB of data in a single IO operation.
|
|
This issue stems principally from an MPI API whose definitions
|
|
utilize 32 bit integers to describe the number of data elements
|
|
and datatype that MPI should use to effect a data transfer.
|
|
Historically, HDF5 has invoked MPI-IO with the number of
|
|
elements in a contiguous buffer represented as the length
|
|
of that buffer in bytes.
|
|
|
|
Resolving the issue and thus enabling larger MPI-IO transfers
|
|
is accomplished first, by detecting when a user IO request would
|
|
exceed the 2GB limit as described above. Once a transfer request
|
|
is identified as requiring special handling, PHDF5 now creates a
|
|
derived datatype consisting of a vector of fixed sized blocks
|
|
which is in turn wrapped within a single MPI_Type_struct to
|
|
contain the vector and any remaining data. The newly created
|
|
datatype is then used in place of MPI_BYTE and can be used to
|
|
fulfill the original user request without encountering API
|
|
errors.
|
|
|
|
(RAW - 2017/09/10, HDFFV-8839)
|
|
|
|
|
|
C++ Library:
|
|
------------
|
|
- The following C++ API wrappers have been added to the C++ Library:
|
|
+ H5Lcreate_soft:
|
|
// Creates a soft link from link_name to target_name.
|
|
void link(const char *target_name, const char *link_name,...)
|
|
void link(const H5std_string& target_name,...)
|
|
|
|
+ H5Lcreate_hard:
|
|
// Creates a hard link from new_name to curr_name.
|
|
void link(const char *curr_name, const Group& new_loc,...)
|
|
void link(const H5std_string& curr_name, const Group& new_loc,...)
|
|
|
|
// Creates a hard link from new_name to curr_name in same location.
|
|
void link(const char *curr_name, const hid_t same_loc,...)
|
|
void link(const H5std_string& curr_name, const hid_t same_loc,...)
|
|
|
|
Note: previous version of H5Location::link will be deprecated.
|
|
|
|
+ H5Lcopy:
|
|
// Copy an object from a group of file to another.
|
|
void copyLink(const char *src_name, const Group& dst,...)
|
|
void copyLink(const H5std_string& src_name, const Group& dst,...)
|
|
|
|
// Copy an object from a group of file to the same location.
|
|
void copyLink(const char *src_name, const char *dst_name,...)
|
|
void copyLink(const H5std_string& src_name,...)
|
|
|
|
+ H5Lmove:
|
|
// Rename an object in a group or file to a new location.
|
|
void moveLink(const char* src_name, const Group& dst,...)
|
|
void moveLink(const H5std_string& src_name, const Group& dst,...)
|
|
|
|
// Rename an object in a group or file to the same location.
|
|
void moveLink(const char* src_name, const char* dst_name,...)
|
|
void moveLink(const H5std_string& src_name,...)
|
|
|
|
Note: previous version H5Location::move will be deprecated.
|
|
|
|
+ H5Ldelete:
|
|
// Removes the specified link from this location.
|
|
void unlink(const char *link_name,
|
|
const LinkAccPropList& lapl = LinkAccPropList::DEFAULT)
|
|
void unlink(const H5std_string& link_name,
|
|
const LinkAccPropList& lapl = LinkAccPropList::DEFAULT)
|
|
|
|
Note: additional parameter is added to previous H5Location::unlink.
|
|
|
|
+ H5Tencode and H5Tdecode:
|
|
// Creates a binary object description of this datatype.
|
|
void DataType::encode() - C API H5Tencode()
|
|
|
|
// Returns the decoded type from the binary object description.
|
|
DataType::decode() - C API H5Tdecode()
|
|
ArrayType::decode() - C API H5Tdecode()
|
|
CompType::decode() - C API H5Tdecode()
|
|
DataType::decode() - C API H5Tdecode()
|
|
EnumType::decode() - C API H5Tdecode()
|
|
FloatType::decode() - C API H5Tdecode()
|
|
IntType::decode() - C API H5Tdecode()
|
|
StrType::decode() - C API H5Tdecode()
|
|
VarLenType::decode() - C API H5Tdecode()
|
|
|
|
+ H5Lget_info:
|
|
// Returns the information of the named link.
|
|
H5L_info_t getLinkInfo(const H5std_string& link_name,...)
|
|
|
|
(BMR - 2018/03/11, HDFFV-10149)
|
|
|
|
- Added class LinkCreatPropList for link create property list.
|
|
|
|
(BMR - 2018/03/11, HDFFV-10149)
|
|
|
|
- Added overloaded functions H5Location::createGroup to take a link
|
|
creation property list.
|
|
Group createGroup(const char* name, const LinkCreatPropList& lcpl)
|
|
Group createGroup(const H5std_string& name, const LinkCreatPropList& lcpl)
|
|
|
|
(BMR - 2018/03/11, HDFFV-10149)
|
|
|
|
- A document is added to the HDF5 C++ API Reference Manual to show the
|
|
mapping from a C API to C++ wrappers. It can be found from the main
|
|
page of the C++ API Reference Manual.
|
|
|
|
(BMR - 2017/10/17, HDFFV-10151)
|
|
|
|
|
|
Java Library:
|
|
----------------
|
|
- Wrapper added for enabling the error stack.
|
|
|
|
H5error_off would disable the error stack reporting. In order
|
|
to re-enable the reporting, the error stack info needs to be
|
|
saved so that H5error_on can revert state.
|
|
|
|
(ADB - 2018/03/13, HDFFV-10412)
|
|
|
|
- Wrappers were added for the following C APIs:
|
|
H5Pset_evict_on_close
|
|
H5Pget_evict_on_close
|
|
H5Pset_chunk_opts
|
|
H5Pget_chunk_opts
|
|
H5Pset_efile_prefix
|
|
H5Pget_efile_prefix
|
|
H5Pset_virtual_prefix
|
|
H5Pget_virtual_prefix
|
|
|
|
(ADB - 2017/12/20)
|
|
|
|
- The H5I_REFERENCE value in the H5I_type_t enum (defined in H5Ipublic.h)
|
|
has been marked as deprectated.
|
|
|
|
JNI code which refers to this value will be removed in a future
|
|
major version of the library. The code will remain unchanged in the
|
|
1.10.x releases and branches.
|
|
|
|
See the C library section, above, for further information.
|
|
|
|
(HDFFV-10252, DER, 2017/04/05)
|
|
|
|
|
|
Tools:
|
|
------
|
|
- h5diff has a new option to display error stack.
|
|
|
|
Updated h5diff with the --enable-error-stack argument, which
|
|
enables the display of the hdf5 error stack. This completes the
|
|
improvement to the main tools: h5copy, h5diff, h5dump, h5ls and
|
|
h5repack.
|
|
|
|
(ADB - 2017/08/30, HDFFV-9774)
|
|
|
|
|
|
Support for new platforms, languages and compilers.
|
|
=======================================
|
|
- None
|
|
|
|
Bug Fixes since HDF5-1.10.1 release
|
|
==================================
|
|
|
|
Library
|
|
-------
|
|
- The data read after a direct chunk write to a chunked dataset with
|
|
one chunk was incorrect.
|
|
|
|
The problem was due to the passing of a null dataset pointer to
|
|
the insert callback for the chunk index in the routine
|
|
H5D__chunk_direct_write() in H5Dchunk.c
|
|
The dataset was a single-chunked dataset which will use the
|
|
single chunk index when latest format was enabled on file creation.
|
|
The single chunk index was the only index that used this pointer
|
|
in the insert callback.
|
|
|
|
Passed the dataset pointer to the insert callback for the chunk
|
|
index in H5D__chunk_direct_write().
|
|
|
|
(VC - 2018/03/20, HDFFV-10425)
|
|
|
|
- Added public routine H5DOread_chunk to the high-level C library.
|
|
|
|
The patch for H5DOwrite_chunk() to write an entire chunk to the file
|
|
directly was contributed by GE Healthcare and integrated by The HDF Group
|
|
developers.
|
|
|
|
(VC - 2017/05/19, HDFFV-9934)
|
|
|
|
- Freeing of object header after failed checksum verification.
|
|
|
|
It was discovered that the object header (in H5Ocache.c) was not released properly
|
|
when the checksum verification failed and a re-load of the object
|
|
header was needed.
|
|
|
|
Freed the object header that failed the chksum verification only
|
|
after the new object header is reloaded, deserialized and set up.
|
|
|
|
(VC - 2018/03/14, HDFFV-10209)
|
|
|
|
- Updated H5Pset_evict_on_close in H5Pfapl.c
|
|
|
|
Changed the minor error number from H5E_CANTSET to H5E_UNSUPPORTED for
|
|
parallel library.
|
|
|
|
(ADB - 2018/03/06, HDFFV-10414)
|
|
|
|
- Fixed the problems with the utility function that could not handle lowercase
|
|
Windows drive letters.
|
|
|
|
Added call to upper function for drive letter.
|
|
|
|
(ADB - 2017/12/18, HDFFV-10307)
|
|
|
|
- Fixed H5Sencode() bug when the number of elements selected was > 2^32.
|
|
|
|
H5Sencode() incorrectly encodes dataspace selection with number of
|
|
elements exceeding 2^32. When decoding such selection via H5Sdecode(),
|
|
the number of elements in the decoded dataspace is not the same as
|
|
what is encoded. This problem exists for H5S_SEL_HYPER and
|
|
H5S_SEL_POINTS encoding.
|
|
|
|
The cause of the problem is due to the fact that the library uses 32 bits to
|
|
encode counts and block offsets for the selection.
|
|
The solution is to use the original 32 bit encodings if possible,
|
|
but use a different way to encode selection if more that 32 bits is needed.
|
|
See details in the RFC: H5Sencode/H5Sdecode Format Change i
|
|
https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5doc/browse/RFCs/HDF5_Library/H5SencodeFormatChange.
|
|
|
|
(VC - 2017/11/28, HDFFV-9947)
|
|
|
|
- Fixed filter plugin handling in H5PL.c and H5Z.c to not require i availability of
|
|
dependent libraries (e.g., szip or zlib).
|
|
|
|
It was discovered that the dynamic loading process used by
|
|
filter plugins had issues with library dependencies.
|
|
|
|
CMake build process changed to use LINK INTERFACE keywords, which
|
|
allowed HDF5 C library to make dependent libraries private. The
|
|
filter plugin libraries no longer require dependent libraries
|
|
(such as szip or zlib) to be available.
|
|
|
|
(ADB - 2017/11/16, HDFFV-10328)
|
|
|
|
- Fixed rare object header corruption bug.
|
|
|
|
In certain cases, such as when converting large attributes to dense
|
|
storage, an error could occur which would either fail an assertion or
|
|
cause file corruption. Fixed and added test.
|
|
|
|
(NAF - 2017/11/14, HDFFV-10274)
|
|
|
|
- Updated H5Zfilter_avail in H5Z.c.
|
|
|
|
The public function checked for plugins, while the private
|
|
function did not.
|
|
|
|
Modified H5Zfilter_avail and private function, H5Z_filter_avail.
|
|
Moved check for plugin from public to private function. Updated
|
|
H5P__set_filter due to change in H5Z_filter_avail. Updated tests.
|
|
|
|
(ADB - 2017/10/10, HDFFV-10297, HDFFV-10319)
|
|
|
|
- h5dump produced SEGFAULT when dumping corrypted file.
|
|
|
|
The behavior was due to the error in the internal function H5HL_offset_into().
|
|
|
|
(1) Fixed H5HL_offset_into() to return error when offset exceeds heap data
|
|
block size.
|
|
(2) Fixed other places in the library that call this routine to detect
|
|
error routine.
|
|
|
|
(VC - 2017/08/30, HDFFV-10216)
|
|
|
|
- Fixes for paged aggregation feature.
|
|
|
|
Skip test in test/fheap.c when:
|
|
(1) multi/split drivers and
|
|
(2) persisting free-space or using paged aggregation strategy
|
|
|
|
(VC, 2017/07/10)
|
|
|
|
Changes made based on RFC review comments:
|
|
(1) Added maximum value for file space page size
|
|
(2) Dropped check for page end metadata threshold
|
|
(3) Removed "can_shrink" and "shrink" callbacks for small section class
|
|
|
|
(VC - 2017/06/09)
|
|
|
|
- Fixed for infinite loop in H5VM_power2up().
|
|
|
|
The function H5VM_power2up() returns the next power of 2
|
|
for n. When n exceeds 2^63, it overflows and becomes 0 causing
|
|
the infinite looping.
|
|
|
|
The fix ensures that the function checks for n >= 2^63
|
|
and returns 0.
|
|
|
|
(VC - 2017/07/10, HDFFV-10217)
|
|
|
|
- Fixed for H5Ocopy doesn't work with open identifiers.
|
|
|
|
Changes made so that raw data for dataset objects are copied from
|
|
cached info when possible instead of flushing objects to file and
|
|
read them back in again.
|
|
|
|
(VC - 2017/07/05, HDFFV-7853)
|
|
|
|
- An uninitialized struct could cause a memory access error when using
|
|
variable-length or reference types in a compressed, chunked dataset.
|
|
|
|
A struct containing a callback function pointer and a pointer to some
|
|
associated data was used before initialization. This could cause a
|
|
memory access error and system crash. This could only occur under
|
|
unusual conditions when using variable-lenth and reference types in
|
|
a compressed, chunked dataset.
|
|
|
|
On recent versions of Visual Studio, when built in debug mode, the
|
|
debug heap will complain and cause a crash if the code in question
|
|
is executed (this will cause the objcopy test to fail).
|
|
|
|
(DER - 2017/11/21, HDFFV-10330)
|
|
|
|
- Fixed collective metadata writes on file close.
|
|
|
|
It was discovered that metadata was being written twice as part of
|
|
the parallel file close behavior, once independently and once
|
|
collectively.
|
|
|
|
A fix for this error was included as part of the parallel compression
|
|
feature but remained undocumented here.
|
|
|
|
(RAW - 2017/12/01, HDFFV-10272)
|
|
|
|
- If an HDF5 file contains a filter pipeline message with a 'number of
|
|
filters' field that exceeds the maximum number of allowed filters,
|
|
the error handling code will attempt to dereference a NULL pointer.
|
|
|
|
This issue was reported to The HDF Group as issue #CVE-2017-17505.
|
|
https://security-tracker.debian.org/tracker/CVE-2017-17505
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17505
|
|
|
|
NOTE: The HDF5 C library cannot produce such a file. This condition
|
|
should only occur in a corrupt (or deliberately altered) file
|
|
or a file created by third-party software.
|
|
|
|
This problem arose because the error handling code assumed that
|
|
the 'number of filters' field implied that a dynamic array of that
|
|
size had already been created and that the cleanup code should
|
|
iterate over that array and clean up each element's resources. If
|
|
an error occurred before the array has been allocated, this will
|
|
not be true.
|
|
|
|
This has been changed so that the number of filters is set to
|
|
zero on errors. Additionally, the filter array traversal in the
|
|
error handling code now requires that the filter array not be NULL.
|
|
|
|
(DER - 2018/02/06, HDFFV-10354)
|
|
|
|
- If an HDF5 file contains a filter pipeline message which contains
|
|
a 'number of filters' field that exceeds the actual number of
|
|
filters in the message, the HDF5 C library will read off the end of
|
|
the read buffer.
|
|
|
|
This issue was reported to The HDF Group as issue #CVE-2017-17506.
|
|
https://security-tracker.debian.org/tracker/CVE-2017-17506
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17506
|
|
|
|
NOTE: The HDF5 C library cannot produce such a file. This condition
|
|
should only occur in a corrupt (or deliberately altered) file
|
|
or a file created by third-party software.
|
|
|
|
The problem was fixed by passing the buffer size with the buffer
|
|
and ensuring that the pointer cannot be incremented off the end
|
|
of the buffer. A mismatch between the number of filters declared
|
|
and the actual number of filters will now invoke normal HDF5
|
|
error handling.
|
|
|
|
(DER - 2018/02/26, HDFFV-10355)
|
|
|
|
- If an HDF5 file contains a malformed compound datatype with a
|
|
suitably large offset, the type conversion code can run off
|
|
the end of the type conversion buffer, causing a segmentation
|
|
fault.
|
|
|
|
This issue was reported to The HDF Group as issue #CVE-2017-17507.
|
|
https://security-tracker.debian.org/tracker/CVE-2017-17506
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17506
|
|
|
|
NOTE: The HDF5 C library cannot produce such a file. This condition
|
|
should only occur in a corrupt (or deliberately altered) file
|
|
or a file created by third-party software.
|
|
|
|
THE HDF GROUP WILL NOT FIX THIS BUG AT THIS TIME
|
|
|
|
Fixing this problem would involve updating the publicly visible
|
|
H5T_conv_t function pointer typedef and versioning the API calls
|
|
which use it. We normally only modify the public API during
|
|
major releases, so this bug will not be fixed at this time.
|
|
|
|
(DER - 2018/02/26, HDFFV-10356)
|
|
|
|
- If an HDF5 file contains a malformed compound type which contains
|
|
a member of size zero, a division by zero error will occur while
|
|
processing the type.
|
|
|
|
This issue was reported to The HDF Group as issue #CVE-2017-17508.
|
|
https://security-tracker.debian.org/tracker/CVE-2017-17508
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17508
|
|
|
|
NOTE: The HDF5 C library cannot produce such a file. This condition
|
|
should only occur in a corrupt (or deliberately altered) file
|
|
or a file created by third-party software.
|
|
|
|
Checking for zero before dividing fixes the problem. Instead of the
|
|
division by zero, the normal HDF5 error handling is invoked.
|
|
|
|
(DER - 2018/02/26, HDFFV-10357)
|
|
|
|
- If an HDF5 file contains a malformed symbol table node that declares
|
|
it contains more symbols than it actually contains, the library
|
|
can run off the end of the metadata cache buffer while processing
|
|
the symbol table node.
|
|
|
|
This issue was reported to The HDF Group as issue #CVE-2017-17509.
|
|
https://security-tracker.debian.org/tracker/CVE-2017-17509
|
|
https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17509
|
|
|
|
NOTE: The HDF5 C library cannot produce such a file. This condition
|
|
should only occur in a corrupt (or deliberately altered) file
|
|
or a file created by third-party software.
|
|
|
|
Performing bounds checks on the buffer while processing fixes the
|
|
problem. Instead of the segmentation fault, the normal HDF5 error
|
|
handling is invoked.
|
|
|
|
(DER - 2018/03/12, HDFFV-10358)
|
|
|
|
- Fixed permissions passed to open(2) on file create.
|
|
|
|
On Windows, the POSIX permissions passed to open(2) when creating files
|
|
were only incidentally correct. They are now set to the correct value of
|
|
(_S_IREAD | _S_IWRITE).
|
|
|
|
On other platforms, the permissions were set to a mix of 666, 644, and
|
|
000. They are now set uniformly to 666.
|
|
|
|
(DER - 2017/04/28, HDFFV-9877)
|
|
|
|
- The H5FD_FEAT_POSIX_COMPAT_HANDLE flag is no longer used to determine
|
|
if a virtual file driver (VFD) is compatible with SWMR.
|
|
|
|
Use of this VFD feature flag was not in line with the documentation in
|
|
the public H5FDpublic.h file. In particular, it was being used as a
|
|
proxy for determining if SWMR I/O is allowed. This is unecessary as we
|
|
already have a feature flag for this (H5FD_SUPPORTS_SWMR_IO).
|
|
|
|
(DER - 2017/05/31, HDFFV-10214)
|
|
|
|
|
|
Configuration
|
|
-------------
|
|
- CMake changes
|
|
|
|
- Updated CMake commands configuration.
|
|
|
|
A number of improvements were made to the CMake commands. Most
|
|
changes simplify usage or eliminate unused constructs. Also,
|
|
some changes support better cross-platform support.
|
|
|
|
(ADB - 2018/02/01, HDFFV-10398)
|
|
|
|
- Corrected usage of CMAKE_BUILD_TYPE variable.
|
|
|
|
The use of the CMAKE_BUILD_TYPE is incorrect for multi-config
|
|
generators (Visual Studio and XCode) and is optional for single
|
|
config generators. Created a new macro to check
|
|
GLOBAL PROPERTY -> GENERATOR_IS_MULTI_CONFIG
|
|
Created two new HDF variable, HDF_BUILD_TYPE and HDF_CFG_BUILD_TYPE.
|
|
Defaults for these variables is "Release".
|
|
|
|
(ADB - 2018/01/10, HDFFV-10385)
|
|
|
|
- Added replacement of fortran flags if using static CRT.
|
|
|
|
Added TARGET_STATIC_CRT_FLAGS call to HDFUseFortran.cmake file in
|
|
config/cmake_ext_mod folder.
|
|
|
|
(ADB - 2018/01/08, HDFFV-10334)
|
|
|
|
|
|
- The hdf5 library used shared szip and zlib, which needlessly required
|
|
applications to link with the same szip and zlib libraries.
|
|
|
|
Changed the target_link_libraries commands to use the static libs.
|
|
Removed improper link duplication of szip and zlib.
|
|
Adjusted the link dependencies and the link interface values of
|
|
the target_link_libraries commands.
|
|
|
|
(ADB - 2017/11/14, HDFFV-10329)
|
|
|
|
- CMake MPI
|
|
|
|
CMake implementation for MPI was problematic and would create incorrect
|
|
MPI library references in the hdf5 libraries.
|
|
|
|
Reworked the CMake MPI code to properly create CMake targets. Also merged
|
|
the latest CMake FindMPI.cmake changes to the local copy. This is necessary
|
|
until HDF changes the CMake minimum to 3.9 or greater.
|
|
|
|
(ADB - 2017/11/02, HDFFV-10321)
|
|
|
|
- Corrected FORTRAN_HAVE_C_LONG_DOUBLE processing in the autotools.
|
|
|
|
A bug in the autotools Fortran processing code always set the
|
|
FORTRAN_HAVE_C_LONG_DOUBLE variable to be true regardless of
|
|
whether or not a C long double type was present.
|
|
|
|
This would cause compilation failures on platforms where a C
|
|
long double type was not available and the Fortran wrappers
|
|
were being built.
|
|
|
|
(DER - 2017/07/05, HDFFV-10247)
|
|
|
|
- The deprecated --enable-production and --enable-debug configure options
|
|
failed to emit errors when passed an empty string
|
|
(e.g.: --enable-debug="").
|
|
|
|
Due to the way we checked for these options being set, it was possible
|
|
to avoid the error message and continue configuration if an empty string
|
|
was passed to the option.
|
|
|
|
Any use of --enable-production or --enable-debug will now halt the
|
|
configuration step and emit a helpful error message
|
|
(use --enable-build-mode=debug|production instead).
|
|
|
|
(DER - 2017/07/05, HDFFV-10248)
|
|
|
|
- CMake
|
|
|
|
Too many commands for POST_BUILD step caused command line to be
|
|
too big on windows.
|
|
|
|
Changed foreach of copy command to use a custom command with the
|
|
use of the HDFTEST_COPY_FILE macro.
|
|
|
|
(ADB - 2017/07/12, HDFFV-10254)
|
|
|
|
- CMake test execution environment
|
|
|
|
The parallel HDF5 test: 't_pread' assumed the use of autotools
|
|
and the directory structure associated with that testing approach.
|
|
Modified the test code to check whether the 'h5jam' utility can be
|
|
found in the same directory as the test executable (which is
|
|
preferred directory structure utilized by cmake) and if found
|
|
will invoke the tool directly rather than utilizing a relative path.
|
|
|
|
(RAW - 2017/11/03, HDFFV-10318)
|
|
|
|
- Fortran compilation fails for xlf and CMake builds.
|
|
|
|
Fixed CMake shared library build for H5match_types and modules
|
|
|
|
(MSB - 2017/12/19, HDFFV-10363)
|
|
|
|
- Shared libraries fail test on OSX with Fortran enabled with CMake.
|
|
|
|
Fixed by removing the F77 use of EQUIVALENCE and COMMON, replaced
|
|
using MODULES. Updated CMake.
|
|
|
|
(MSB - 2017/12/07, HDFFV-10223)
|
|
|
|
- The bin/trace script now emits an error code on problems and autogen.sh
|
|
will fail if bin/trace fails.
|
|
|
|
The bin/trace script adds tracing functionality to public HDF5 API calls.
|
|
It is only of interest to developers who modify the HDF5 source code.
|
|
Previously, bin/trace just wrote an error message to stdout when it
|
|
encountered problems, so autogen.sh processing did not halt and a broken
|
|
version of the library could be built. The script will now return an
|
|
error code when it encounters problems, and autogen.sh will fail.
|
|
|
|
This only affects users who run autogen.sh to rebuild the Autotools files,
|
|
which is not necessary to build HDF5 from source in official releases of the
|
|
library. CMake users are unaffected as bin/trace is not run via CMake
|
|
at this time.
|
|
|
|
(DER - 2017/04/25, HDFFV-10178)
|
|
|
|
- FC_BASENAME was changed from gfortran40 to gfortran in a few places.
|
|
|
|
In the autotools, FC_BASENAME was set to gfortran40 in a few locations
|
|
(config/gnu-fflags and config/freebsd). This was probably a historical
|
|
artifact and did not seem to affect many users.
|
|
|
|
The value is now correctly set to gfortran.
|
|
|
|
(DER - 2017/05/26, HDFFV-10249)
|
|
|
|
- The ar flags were changed to -cr (was: -cru)
|
|
|
|
The autotools set the flags for ar to -cru by default. The -u flag,
|
|
which allows selective replacement of only the members which have
|
|
changed, raises warnings on some platforms, so the flags are now set to
|
|
-cr via AR_FLAGS in configure.ac. This causes the static library to
|
|
always be completely recreated from the object files on each build.
|
|
|
|
(DER - 2017/11/15, HDFFV-10428)
|
|
|
|
|
|
Fortran
|
|
--------
|
|
- Fixed compilation errors when using Intel 18 Fortran compilers
|
|
(MSB - 2017/11/3, HDFFV-10322)
|
|
|
|
Tools
|
|
-----
|
|
- h5clear
|
|
|
|
An enhancement to the tool in setting a file's stored EOA.
|
|
|
|
It was discovered that a crashed file's stored EOA in the superblock
|
|
was smaller than the actual file's EOF. When the file was reopened
|
|
and closed, the library truncated the file to the stored EOA.
|
|
|
|
Added an option to the tool in setting the file's stored EOA in the
|
|
superblock to the maximum of (EOA, EOF) + increment.
|
|
An option was also added to print the file's EOA and EOF.
|
|
|
|
(VC - 2018/03/14, HDFFV-10360)
|
|
|
|
- h5repack
|
|
|
|
h5repack changes the chunk parameters when a change of layout is not
|
|
specified and a filter is applied.
|
|
|
|
HDFFV-10297, HDFFV-10319 reworked code for h5repack and h5diff code
|
|
in the tools library. The check for an existing layout was incorrectly
|
|
placed into an if block and not executed. The check was moved into
|
|
the normal path of the function.
|
|
|
|
(ADB - 2018/02/21, HDFFV-10412)
|
|
|
|
- h5dump
|
|
|
|
The tools library will hide the error stack during file open.
|
|
|
|
While this is preferable almost always, there are reasons to enable
|
|
display of the error stack when a tool will not open a file. Adding an
|
|
optional argument to the --enable-error-stack will provide this use case.
|
|
As an optional argument it will not affect the operation of the
|
|
--enable-error-stack. h5dump is the only tool to implement this change.
|
|
|
|
(ADB - 2018/02/15, HDFFV-10384)
|
|
|
|
- h5dump
|
|
|
|
h5dump would output an indented blank line in the filters section.
|
|
|
|
h5dump overused the h5tools_simple_prefix function, which is a
|
|
function intended to account for the data index (x,y,z) option.
|
|
Removed the function call for header information.
|
|
|
|
(ADB - 2018/01/25, HDFFV-10396)
|
|
|
|
- h5repack
|
|
|
|
h5repack incorrectly searched internal object table for name.
|
|
|
|
h5repack would search the table of objects for a name, if the
|
|
name did not match it tried to determine if the name without a
|
|
leading slash would match. The logic was flawed! The table
|
|
stored names(paths) without a leading slash and did a strstr
|
|
of the table path to the name.
|
|
The assumption was that if there was a difference of one then
|
|
it was a match, however "pressure" would match "/pressure" as
|
|
well as "/pressure1", "/pressure2", etc. Changed logic to remove
|
|
any leading slash and then do a full compare of the name.
|
|
|
|
(ADB - 2018/01/18, HDFFV-10393)
|
|
|
|
- h5repack
|
|
|
|
h5repack failed to handle command line parameters for customer filters.
|
|
|
|
User defined filter parameter conversions would fail when integers were
|
|
represented on the command line with character string
|
|
larger then 9 characters. Increased local variable array for storing
|
|
the current command line parameter to prevent buffer overflows.
|
|
|
|
(ADB - 2018/01/17, HDFFV-10392)
|
|
|
|
- h5diff
|
|
|
|
h5diff seg faulted if comparing VL strings against fixed strings.
|
|
|
|
Reworked solution for HDFFV-8625 and HDFFV-8639. Implemented the check
|
|
for string objects of same type in the diff_can_type function by
|
|
adding an if(tclass1 == H5T_STRING) block. This "if block" moves the
|
|
same check that was added for attributes to this function, which is
|
|
used by all object types. This function handles complex type structures.
|
|
Also added a new test file in h5diffgenttest for testing this issue
|
|
and removed the temporary files used in the test scripts.
|
|
|
|
(ADB - 2018/01/04, HDFFV-8745)
|
|
|
|
- h5repack
|
|
|
|
h5repack failed to copy a dataset with existing filter.
|
|
|
|
Reworked code for h5repack and h5diff code in the tools library. Added
|
|
improved error handling, cleanup of resources and checks of calls.
|
|
Modified H5Zfilter_avail and private function, H5Z_filter_avail.
|
|
Moved check for plugin from public to private function. Updated
|
|
H5P__set_filter due to change in H5Z_filter_avail. Updated tests.
|
|
Note, h5repack output display has changed to clarify the individual
|
|
steps of the repack process. The output indicates if an operation
|
|
applies to all objects. Lines with notation and no information
|
|
have been removed.
|
|
|
|
(ADB - 2017/10/10, HDFFV-10297, HDFFV-10319)
|
|
|
|
- h5repack
|
|
|
|
h5repack always set the User Defined filter flag to H5Z_FLAG_MANDATORY.
|
|
|
|
Added another parameter to the 'UD=' option to set the flag by default
|
|
to '0' or H5Z_FLAG_MANDATORY, the other choice is '1' or H5Z_FLAG_OPTIONAL.
|
|
|
|
(ADB - 2017/08/31, HDFFV-10269)
|
|
|
|
- h5ls
|
|
|
|
h5ls generated error on stack when it encountered a H5S_NULL
|
|
dataspace.
|
|
|
|
Adding checks for H5S_NULL before calling H5Sis_simple (located
|
|
in the h5tools_dump_mem function) fixed the issue.
|
|
|
|
(ADB - 2017/08/17, HDFFV-10188)
|
|
|
|
- h5repack
|
|
|
|
Added tests to h5repack.sh.in to verify options added for paged
|
|
aggregation work as expected.
|
|
|
|
(VC - 2017/08/03)
|
|
|
|
- h5dump
|
|
|
|
h5dump segfaulted on output of XML file.
|
|
|
|
Function that escape'd strings used the full buffer length
|
|
instead of just the length of the replacement string in a
|
|
strncpy call. Using the correct length fixed the issue.
|
|
|
|
(ADB - 2017/08/01, HDFFV-10256)
|
|
|
|
- h5diff
|
|
|
|
h5diff segfaulted on compare of a NULL variable length string.
|
|
|
|
Improved h5diff compare of strings by adding a check for
|
|
NULL strings and setting the lengths to zero.
|
|
|
|
(ADB - 2017/07/25, HDFFV-10246)
|
|
|
|
- h5import
|
|
|
|
h5import crashed trying to import data from a subset of a dataset.
|
|
|
|
Improved h5import by adding the SUBSET keyword. h5import understands
|
|
to use the Count times the Block as the size of the dimensions.
|
|
Added INPUT_B_ORDER keyword to old-style configuration files.
|
|
The import from h5dump function expects the binary files to use native
|
|
types (FILE '-b' option) in the binary file.
|
|
|
|
(ADB - 2017/06/15, HDFFV-10219)
|
|
|
|
- h5repack
|
|
|
|
h5repack did not maintain the creation order flag of the root
|
|
group.
|
|
|
|
Improved h5repack by reading the creation order and applying the
|
|
flag to the new root group. Also added arguments to set the
|
|
order and index direction, which applies to the traversing of the
|
|
original file, on the command line.
|
|
|
|
(ADB - 2017/05/26, HDFFV-8611)
|
|
|
|
- h5diff
|
|
|
|
h5diff failed to account for strpad type and null terminators
|
|
of char strings. Also, h5diff failed to account for string length
|
|
differences and would give a different result depending on file
|
|
order in the command line.
|
|
|
|
Improved h5diff compare of strings and arrays by adding a check for
|
|
string lengths and if the strpad was null filled.
|
|
|
|
(ADB - 2017/05/18, HDFFV-9055, HDFFV-10128)
|
|
|
|
High-Level APIs:
|
|
------
|
|
- H5DOwrite_chunk() problems when overwriting an existing chunk with
|
|
no filters enabled.
|
|
|
|
When overwriting chunks and no filters were being used, the library would
|
|
fail (when asserts are enabled, e.g. debug builds) or incorrectly
|
|
insert additional chunks instead of overwriting (when asserts are not
|
|
enabled, e.g. production builds).
|
|
|
|
This has been fixed and a test was added to the hl/test_dset_opt test.
|
|
|
|
(DER - 2017/05/11, HDFFV-10187)
|
|
|
|
C++ APIs
|
|
--------
|
|
- Removal of memory leaks.
|
|
|
|
A private function was inadvertently called, causing memory leaks. This
|
|
is now fixed.
|
|
|
|
(BMR - 2018/03/12 - User's reported in email)
|
|
|
|
Testing
|
|
-------
|
|
- Memory for three variables in testphdf5's coll_write_test was malloced
|
|
but not freed, leaking memory when running the test.
|
|
|
|
The variables' memory is now freed.
|
|
|
|
(LRK - 2018/03/12, HDFFV-10397)
|
|
|
|
- Refactored the testpar/t_bigio.c test to include ALARM macros
|
|
|
|
Changed the test to include the ALARM_ON and ALARM_OFF macros which
|
|
are intended to prevent nightly test hangs that have been observed
|
|
with this particular parallel test example. The code was also modified to
|
|
simplify status reporting (only from MPI rank 0) and additional
|
|
status checking added.
|
|
|
|
(RAW - 2017/11/08, HDFFV-10301)
|
|
|
|
|
|
Supported Platforms
|
|
===================
|
|
|
|
Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
#1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
(ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
|
|
IBM XL C/C++ V13.1
|
|
IBM XL Fortran V15.1
|
|
|
|
Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
Version 4.9.3, Version 5.2.0,
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.0.098 Build 20160721
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc
|
|
(emu) Sun Fortran 95 8.6 SunOS_sparc
|
|
Sun C++ 5.12 SunOS_sparc
|
|
|
|
Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
|
|
Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
Visual Studio 2015 w/ Intel C, Fortran 2017 (cmake)
|
|
Visual Studio 2015 w/ MSMPI 8 (cmake)
|
|
Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3)
|
|
gcc and gfortran compilers (GCC 5.4.0)
|
|
(cmake and autotools)
|
|
|
|
Windows 10 Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3)
|
|
gcc and gfortran compilers (GCC 5.4.0)
|
|
(cmake and autotools)
|
|
|
|
Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
|
|
Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3
|
|
64-bit gfortran GNU Fortran (GCC) 5.2.0
|
|
(osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2
|
|
|
|
Mac OS Sierra 10.12.6 Apple LLVM version 8.1.0 (clang/clang++-802.0.42)
|
|
64-bit gfortran GNU Fortran (GCC) 7.1.0
|
|
(swallow/kite) Intel icc/icpc/ifort version 17.0.2
|
|
|
|
|
|
Tested Configuration Features Summary
|
|
=====================================
|
|
|
|
In the tables below
|
|
y = tested
|
|
n = not tested in this release
|
|
C = Cluster
|
|
W = Workstation
|
|
x = not working in this release
|
|
dna = does not apply
|
|
( ) = footnote appears below second table
|
|
<blank> = testing incomplete on this feature or platform
|
|
|
|
Platform C F90/ F90 C++ zlib SZIP
|
|
parallel F2003 parallel
|
|
Solaris2.11 32-bit n y/y n y y y
|
|
Solaris2.11 64-bit n y/n n y y y
|
|
Windows 7 y y/y n y y y
|
|
Windows 7 x64 y y/y y y y y
|
|
Windows 7 Cygwin n y/n n y y y
|
|
Windows 7 x64 Cygwin n y/n n y y y
|
|
Windows 10 y y/y n y y y
|
|
Windows 10 x64 y y/y n y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y
|
|
Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y
|
|
Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 PGI n y/y n y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 GNU y y/y y y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 Intel n y/y n y y y
|
|
Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y
|
|
|
|
|
|
Platform Shared Shared Shared Thread-
|
|
C libs F90 libs C++ libs safe
|
|
Solaris2.11 32-bit y y y y
|
|
Solaris2.11 64-bit y y y y
|
|
Windows 7 y y y y
|
|
Windows 7 x64 y y y y
|
|
Windows 7 Cygwin n n n y
|
|
Windows 7 x64 Cygwin n n n y
|
|
Windows 10 y y y y
|
|
Windows 10 x64 y y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit y n y y
|
|
Mac OS X Mavericks 10.9.5 64-bit y n y y
|
|
Mac OS X Yosemite 10.10.5 64-bit y n y y
|
|
Mac OS X El Capitan 10.11.6 64-bit y n y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 PGI y y y n
|
|
CentOS 7.2 Linux 2.6.32 x86_64 GNU y y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 Intel y y y n
|
|
Linux 2.6.32-573.18.1.el6.ppc64 y y y n
|
|
|
|
Compiler versions for each platform are listed in the preceding
|
|
"Supported Platforms" table.
|
|
|
|
|
|
More Tested Platforms
|
|
=====================
|
|
The following platforms are not supported but have been tested for this release.
|
|
|
|
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(mayll/platypus) Version 4.4.7 20120313
|
|
Version 4.9.3, 5.3.0, 6.2.0
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86-64;
|
|
Version 17.10-0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 17.0.4.196 Build 20170411
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers
|
|
#1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
(jelly) with NAG Fortran Compiler Release 6.1(Tozai)
|
|
GCC Version 7.1.0
|
|
OpenMPI 3.0.0-GCC-7.2.0-2.29
|
|
Intel(R) C (icc) and C++ (icpc) compilers
|
|
Version 17.0.0.098 Build 20160721
|
|
with NAG Fortran Compiler Release 6.1(Tozai)
|
|
|
|
Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0
|
|
#1 SMP x86_64 GNU/Linux
|
|
(moohan)
|
|
|
|
Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with
|
|
#1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1
|
|
(ostrich) and IBM XL Fortran for Linux, V15.1
|
|
|
|
Debian 8.4 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux
|
|
gcc, g++ (Debian 4.9.2-10) 4.9.2
|
|
GNU Fortran (Debian 4.9.2-10) 4.9.2
|
|
(cmake and autotools)
|
|
|
|
Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
|
|
gcc, g++ (GCC) 6.1.1 20160621
|
|
(Red Hat 6.1.1-3)
|
|
GNU Fortran (GCC) 6.1.1 20160621
|
|
(Red Hat 6.1.1-3)
|
|
(cmake and autotools)
|
|
|
|
Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux
|
|
gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2)
|
|
5.4.0 20160609
|
|
GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2)
|
|
5.4.0 20160609
|
|
(cmake and autotools)
|
|
|
|
|
|
Known Problems
|
|
==============
|
|
|
|
At present, metadata cache images may not be generated by parallel
|
|
applications. Parallel applications can read files with metadata cache
|
|
images, but since this is a collective operation, a deadlock is possible
|
|
if one or more processes do not participate.
|
|
|
|
Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29:
|
|
testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio)
|
|
t_shapesame (sscontig2)
|
|
t_pflush1/fails on exit
|
|
The first two tests fail attempting collective writes.
|
|
|
|
Known problems in previous releases can be found in the HISTORY*.txt files
|
|
in the HDF5 source. Please report any new problems found to
|
|
help@hdfgroup.org.
|
|
|
|
|
|
%%%%1.10.1%%%%
|
|
|
|
HDF5 version 1.10.1 released on 2017-04-27
|
|
================================================================================
|
|
|
|
INTRODUCTION
|
|
|
|
This document describes the differences between HDF5-1.10.0-patch1 and
|
|
HDF5 1.10.1, and contains information on the platforms tested and known
|
|
problems in HDF5-1.10.1. For more details check the HISTORY*.txt files
|
|
in the HDF5 source.
|
|
|
|
Links to HDF5 1.10.1 source code, documentation, and additional materials can
|
|
be found on The HDF5 web page at:
|
|
|
|
https://support.hdfgroup.org/HDF5/
|
|
|
|
The HDF5 1.10.1 release can be obtained from:
|
|
|
|
https://support.hdfgroup.org/HDF5/release/obtain5.html
|
|
|
|
User documentation for the snapshot can be accessed directly at this location:
|
|
|
|
https://support.hdfgroup.org/HDF5/doc/
|
|
|
|
New features in the HDF5-1.10.x release series, including brief general
|
|
descriptions of some new and modified APIs, are described in the "New Features
|
|
in HDF5 Release 1.10" document:
|
|
|
|
https://support.hdfgroup.org/HDF5/docNewFeatures/index.html
|
|
|
|
All new and modified APIs are listed in detail in the "HDF5 Software Changes
|
|
from Release to Release" document, in the section "Release 10.1 (current
|
|
release) versus Release 1.10.0
|
|
|
|
https://support.hdfgroup.org/HDF5/doc/ADGuide/Changes.html
|
|
|
|
If you have any questions or comments, please send them to the HDF Help Desk:
|
|
|
|
help@hdfgroup.org
|
|
|
|
|
|
CONTENTS
|
|
|
|
- Major New Features Introduced in HDF5 1.10.1
|
|
- Other New Features and Enhancements
|
|
- Support for New Platforms, Languages, and Compilers
|
|
- Bug Fixes since HDF5-1.10.0-patch1
|
|
- Supported Platforms
|
|
- Tested Configuration Features Summary
|
|
- More Tested Platforms
|
|
- Known Problems
|
|
|
|
|
|
Major New Features Introduced in HDF5 1.10.1
|
|
============================================
|
|
|
|
For links to the RFCs and documentation in this section please view
|
|
https://support.hdfgroup.org/HDF5/docNewFeatures in a web browser.
|
|
|
|
________________________________________
|
|
Metadata Cache Image
|
|
________________________________________
|
|
|
|
HDF5 metadata is typically small, and scattered throughout the HDF5 file.
|
|
This can affect performance, particularly on large HPC systems. The
|
|
Metadata Cache Image feature can improve performance by writing the
|
|
metadata cache in a single block on file close, and then populating the
|
|
cache with the contents of this block on file open, thus avoiding the many
|
|
small I/O operations that would otherwise be required on file open and
|
|
close. See the RFC for complete details regarding this feature. Also,
|
|
see the Fine Tuning the Metadata Cache documentation.
|
|
|
|
At present, metadata cache images may not be generated by parallel
|
|
applications. Parallel applications can read files with metadata cache
|
|
images, but since this is a collective operation, a deadlock is possible
|
|
if one or more processes do not participate.
|
|
|
|
________________________________________
|
|
Metadata Cache Evict on Close
|
|
________________________________________
|
|
|
|
The HDF5 library's metadata cache is fairly conservative about holding on
|
|
to HDF5 object metadata (object headers, chunk index structures, etc.),
|
|
which can cause the cache size to grow, resulting in memory pressure on
|
|
an application or system. The "evict on close" property will cause all
|
|
metadata for an object to be evicted from the cache as long as metadata
|
|
is not referenced from any other open object. See the Fine Tuning the
|
|
Metadata Cache documentation for information on the APIs.
|
|
|
|
At present, evict on close is disabled in parallel builds.
|
|
|
|
________________________________________
|
|
Paged Aggregation
|
|
________________________________________
|
|
|
|
The current HDF5 file space allocation accumulates small pieces of metadata
|
|
and raw data in aggregator blocks which are not page aligned and vary
|
|
widely in sizes. The paged aggregation feature was implemented to provide
|
|
efficient paged access of these small pieces of metadata and raw data.
|
|
See the RFC for details. Also, see the File Space Management documentation.
|
|
|
|
________________________________________
|
|
Page Buffering
|
|
________________________________________
|
|
|
|
Small and random I/O accesses on parallel file systems result in poor
|
|
performance for applications. Page buffering in conjunction with paged
|
|
aggregation can improve performance by giving an application control of
|
|
minimizing HDF5 I/O requests to a specific granularity and alignment.
|
|
See the RFC for details. Also, see the Page Buffering documentation.
|
|
|
|
At present, page buffering is disabled in parallel builds.
|
|
|
|
|
|
|
|
Other New Features and Enhancements
|
|
===================================
|
|
|
|
Library
|
|
-------
|
|
- Added a mechanism for disabling the SWMR file locking scheme.
|
|
|
|
The file locking calls used in HDF5 1.10.0 (including patch1)
|
|
will fail when the underlying file system does not support file
|
|
locking or where locks have been disabled. To disable all file
|
|
locking operations, an environment variable named
|
|
HDF5_USE_FILE_LOCKING can be set to the five-character string
|
|
'FALSE'. This does not fundamentally change HDF5 library
|
|
operation (aside from initial file open/create, SWMR is lock-free),
|
|
but users will have to be more careful about opening files
|
|
to avoid problematic access patterns (i.e.: multiple writers)
|
|
that the file locking was designed to prevent.
|
|
|
|
Additionally, the error message that is emitted when file lock
|
|
operations set errno to ENOSYS (typical when file locking has been
|
|
disabled) has been updated to describe the problem and potential
|
|
resolution better.
|
|
|
|
(DER, 2016/10/26, HDFFV-9918)
|
|
|
|
- The return type of H5Pget_driver_info() has been changed from void *
|
|
to const void *.
|
|
|
|
The pointer returned by this function points to internal library
|
|
memory and should not be freed by the user.
|
|
|
|
(DER, 2016/11/04, HDFFV-10017)
|
|
|
|
- The direct I/O VFD has been removed from the list of VFDs that
|
|
support SWMR.
|
|
|
|
This configuration was never officially tested and several SWMR
|
|
tests fail when this VFD is set.
|
|
|
|
(DER, 2016/11/03, HDFFV-10169)
|
|
|
|
Configuration:
|
|
--------------
|
|
- The minimum version of CMake required to build HDF5 is now 3.2.2.
|
|
|
|
(ADB, 2017/01/10)
|
|
|
|
- An --enable/disable-developer-warnings option has been added to
|
|
configure.
|
|
|
|
This disables warnings that do not indicate poor code quality such
|
|
as -Winline and gcc's -Wsuggest-attribute. Developer warnings are
|
|
disabled by default.
|
|
|
|
(DER, 2017/01/10)
|
|
|
|
- A bin/restore.sh script was added that reverts autogen.sh processing.
|
|
|
|
(DER, 2016/11/08)
|
|
|
|
- CMake: Added NAMESPACE hdf5:: to package configuration files to allow
|
|
projects using installed HDF5 binaries built with CMake to link with
|
|
them without specifying the HDF5 library location via IMPORTED_LOCATION.
|
|
|
|
(ABD, 2016/10/17, HDFFV-10003)
|
|
|
|
- CMake: Changed the CTEST_BUILD_CONFIGURATION option to
|
|
CTEST_CONFIGURATION_TYPE as recommended by the CMake documentation.
|
|
|
|
(ABD, 2016/10/17, HDFFV-9971)
|
|
|
|
|
|
Fortran Library:
|
|
----------------
|
|
|
|
- The HDF5 Fortran library can now be compiled with the NAG compiler.
|
|
|
|
(MSB, 2017/2/10, HDFFV-9973)
|
|
|
|
|
|
C++ Library:
|
|
------------
|
|
|
|
- The following C++ API wrappers have been added to the C++ Library:
|
|
|
|
// Sets/Gets the strategy and the threshold value that the library
|
|
// will employ in managing file space.
|
|
FileCreatPropList::setFileSpaceStrategy - H5Pset_file_space_strategy
|
|
FileCreatPropList::getFileSpaceStrategy - H5Pget_file_space_strategy
|
|
|
|
// Sets/Gets the file space page size for paged aggregation.
|
|
FileCreatPropList::setFileSpacePagesize - H5Pset_file_space_page_size
|
|
FileCreatPropList::getFileSpacePagesize - H5Pget_file_space_page_size
|
|
|
|
// Checks if the given ID is valid.
|
|
IdComponent::isValid - H5Iis_valid
|
|
|
|
// Sets/Gets the number of soft or user-defined links that can be
|
|
// traversed before a failure occurs.
|
|
LinkAccPropList::setNumLinks - H5Pset_nlinks
|
|
LinkAccPropList::getNumLinks - H5Pget_nlinks
|
|
|
|
// Returns a copy of the creation property list of a datatype.
|
|
DataType::getCreatePlist - H5Tget_create_plist
|
|
|
|
// Opens/Closes an object within a group or a file, regardless of object
|
|
// type
|
|
Group::getObjId - H5Oopen
|
|
Group::closeObjId - H5Oclose
|
|
|
|
// Maps elements of a virtual dataset to elements of the source dataset.
|
|
DSetCreatPropList::setVirtual - H5Pset_virtual
|
|
|
|
// Gets general information about this file.
|
|
H5File::getFileInfo - H5Fget_info2
|
|
|
|
// Returns the number of members in a type.
|
|
IdComponent::getNumMembers - H5Inmembers
|
|
|
|
// Determines if an element type exists.
|
|
IdComponent::typeExists - H5Itype_exists
|
|
|
|
// Determines if an object exists.
|
|
H5Location::exists - H5Lexists.
|
|
|
|
// Returns the header version of an HDF5 object.
|
|
H5Object::objVersion - H5Oget_info for version
|
|
|
|
(BMR, 2017/03/20, HDFFV-10004, HDFFV-10139, HDFFV-10145)
|
|
|
|
- New exception: ObjHeaderIException for H5O interface.
|
|
|
|
(BMR, 2017/03/15, HDFFV-10145)
|
|
|
|
- New class LinkAccPropList for link access property list, to be used by
|
|
wrappers of H5Lexists.
|
|
|
|
(BMR, 2017/01/04, HDFFV-10145)
|
|
|
|
- New constructors to open datatypes in ArrayType, CompType, DataType,
|
|
EnumType, FloatType, IntType, StrType, and VarLenType.
|
|
|
|
(BMR, 2016/12/26, HDFFV-10056)
|
|
|
|
- New member functions:
|
|
|
|
DSetCreatPropList::setNbit() to setup N-bit compression for a dataset.
|
|
|
|
ArrayType::getArrayNDims() const
|
|
ArrayType::getArrayDims() const
|
|
both to replace the non-const versions.
|
|
|
|
(BMR, 2016/04/25, HDFFV-8623, HDFFV-9725)
|
|
|
|
|
|
Tools:
|
|
------
|
|
- The following options have been added to h5clear:
|
|
-s: clear the status_flags field in the file's superblock
|
|
-m: Remove the metadata cache image from the file
|
|
|
|
(QAK, 2017/03/22, PR#361)
|
|
|
|
|
|
High-Level APIs:
|
|
---------------
|
|
- Added New Fortran 2003 API for h5tbmake_table_f.
|
|
|
|
(MSB, 2017/02/10, HDFFV-8486)
|
|
|
|
|
|
|
|
Support for New Platforms, Languages, and Compilers
|
|
===================================================
|
|
|
|
- Added NAG compiler
|
|
|
|
|
|
|
|
Bug Fixes since HDF5-1.10.0-patch1 release
|
|
==================================
|
|
|
|
Library
|
|
-------
|
|
- Outdated data structure was used in H5D_CHUNK_DEBUG blocks, causing
|
|
compilation errors when H5D_CHUNK_DEBUG was defined. This is fixed.
|
|
|
|
(BMR, 2017/04/04, HDFFV-8089)
|
|
|
|
- SWMR implementation in the HDF5 1.10.0 and 1.10.0-patch1 releases has a
|
|
broken metadata flush dependency that manifested itself with the following
|
|
error at the end of the HDF5 error stack:
|
|
|
|
H5Dint.c line 846 in H5D__swmr_setup(): dataspace chunk index must be 0
|
|
for SWMR access, chunkno = 1
|
|
major: Dataset
|
|
minor: Bad value
|
|
|
|
It was also reported at https://github.com/areaDetector/ADCore/issues/203
|
|
|
|
The flush dependency is fixed in this release.
|
|
|
|
- Changed the plugins dlopen option from RTLD_NOW to RTLD_LAZY
|
|
|
|
(ABD, 2016/12/12, PR#201)
|
|
|
|
- A number of issues were fixed when reading/writing from/to corrupted
|
|
files to ensure that the library fails gracefully in these cases:
|
|
|
|
* Writing to a corrupted file that has an object message which is
|
|
incorrectly marked as sharable on disk results in a buffer overflow /
|
|
invalid write instead of a clean error message.
|
|
|
|
* Decoding data from a corrupted file with a dataset encoded with the
|
|
H5Z_NBIT decoding can result in a code execution vulnerability under
|
|
the context of the application using the HDF5 library.
|
|
|
|
* When decoding an array datatype from a corrupted file, the HDF5 library
|
|
fails to return an error in production if the number of dimensions
|
|
decoded is greater than the maximum rank.
|
|
|
|
* When decoding an "old style" array datatype from a corrupted file, the
|
|
HDF5 library fails to return an error in production if the number of
|
|
dimensions decoded is greater than the maximum rank.
|
|
|
|
(NAF, 2016/10/06, HDFFV-9950, HDFFV-9951, HDFFV-9992, HDFFV-9993)
|
|
|
|
- Fixed an error that would occur when copying an object with an attribute
|
|
which is a compound datatype consisting of a variable length string.
|
|
|
|
(VC, 2016/08/24, HDFFV-7991)
|
|
|
|
- H5DOappend will no longer fail if a dataset has no append callback
|
|
registered.
|
|
|
|
(VC, 2016/08/14, HDFFV-9960)
|
|
|
|
- Fixed an issue where H5Pset_alignment could result in misaligned blocks
|
|
with some input combinations, causing an assertion failure in debug mode.
|
|
|
|
(NAF, 2016/08/11, HDFFV-9948)
|
|
|
|
- Fixed a problem where a plugin compiled into a DLL in the default plugin
|
|
directory could not be found by the HDF5 library at runtime on Windows
|
|
when the HDF5_PLUGIN_PATH environment variable was not set.
|
|
|
|
(ABD, 2016/08/01, HDFFV-9706)
|
|
|
|
- Fixed an error that would occur when calling H5Adelete on an attribute
|
|
which is attached to an externally linked object in the target file and
|
|
whose datatype is a committed datatype in the main file.
|
|
|
|
(VC, 2016/07/06, HDFFV-9940)
|
|
|
|
- (a) Throw an error instead of assertion when v1 btree level hits the 1
|
|
byte limit.
|
|
(b) Modifications to better handle error recovery when conversion by
|
|
h5format_convert fails.
|
|
|
|
(VC, 2016/05/29, HDFFV-9434)
|
|
|
|
- Fixed a memory leak where an array used by the library to track SWMR
|
|
read retries was unfreed.
|
|
|
|
The leaked memory was small (on the order of a few tens of ints) and
|
|
allocated per-file. The memory was allocated (and lost) only when a
|
|
file was opened for SWMR access.
|
|
|
|
(DER, 2016/04/27, HDFFV-9786)
|
|
|
|
- Fixed a memory leak that could occur when opening a file for the first
|
|
time (including creating) and the call fails.
|
|
|
|
This occurred when the file-driver-specific info was not cleaned up.
|
|
The amount of memory leaked varied with the file driver, but would
|
|
normally be less than 1 kB.
|
|
|
|
(DER, 2016/12/06, HDFFV-10168)
|
|
|
|
- Fixed a failure in collective metadata writes.
|
|
|
|
This failure only appeared when collective metadata writes
|
|
were enabled (via H5Pset_coll_metadata_write()).
|
|
|
|
(JRM, 2017/04/10, HDFFV-10055)
|
|
|
|
|
|
Parallel Library
|
|
----------------
|
|
- Fixed a bug that could occur when allocating a chunked dataset in parallel
|
|
with an alignment set and an alignment threshold greater than the chunk
|
|
size but less than or equal to the raw data aggregator size.
|
|
|
|
(NAF, 2016/08/11, HDFFV-9969)
|
|
|
|
|
|
Configuration
|
|
-------------
|
|
- Configuration will check for the strtoll and strtoull functions
|
|
before using alternatives
|
|
|
|
(ABD, 2017/03/17, PR#340)
|
|
|
|
- CMake uses a Windows pdb directory variable if available and
|
|
will generate both static and shared pdb files.
|
|
|
|
(ABD, 2017/02/06, HDFFV-9875)
|
|
|
|
- CMake now builds shared versions of tools.
|
|
|
|
(ABD, 2017/02/01, HDFFV-10123)
|
|
|
|
- Makefiles and test scripts have been updated to correctly remove files
|
|
created when running "make check" and to avoid removing any files under
|
|
source control. In-source builds followed by "make clean" and "make
|
|
distclean" should result in the original source files.
|
|
(LRK, 2017/01/17, HDFFV-10099)
|
|
|
|
- The tools directory has been divided into two separate source and test
|
|
directories. This resolves a build dependency and, as a result,
|
|
'make check' will no longer fail in the tools directory if 'make' was
|
|
not executed first.
|
|
|
|
(ABD, 2016/10/27, HDFFV-9719)
|
|
|
|
- CMake: Fixed a timeout error that would occasionally occur when running
|
|
the virtual file driver tests simultaneously due to test directory
|
|
and file name collisions.
|
|
|
|
(ABD, 2016/09/19, HDFFV-9431)
|
|
|
|
- CMake: Fixed a command length overflow error by converting custom
|
|
commands inside CMakeTest.cmake files into regular dependencies and
|
|
targets.
|
|
|
|
(ABD, 2016/07/12, HDFFV-9939)
|
|
|
|
- Fixed a problem preventing HDF5 to be built on 32-bit CYGWIN by
|
|
condensing cygwin configuration files into a single file and
|
|
removing outdated compiler settings.
|
|
|
|
(ABD, 2016/07/12, HDFFV-9946)
|
|
|
|
|
|
Fortran
|
|
--------
|
|
- Changed H5S_ALL_F from INTEGER to INTEGER(HID_T)
|
|
|
|
(MSB, 2016/10/14, HDFFV-9987)
|
|
|
|
|
|
Tools
|
|
-----
|
|
- h5diff now correctly ignores strpad in comparing strings.
|
|
|
|
(ABD, 2017/03/03, HDFFV-10128)
|
|
|
|
- h5repack now correctly parses the command line filter options.
|
|
|
|
(ABD, 2017/01/24, HDFFV-10046)
|
|
|
|
- h5diff now correctly returns an error when it cannot read data due
|
|
to an unavailable filter plugin.
|
|
|
|
(ADB 2017/01/18, HDFFV-9994 )
|
|
|
|
- Fixed an error in the compiler wrapper scripts (h5cc, h5fc, et al.)
|
|
in which they would erroneously drop the file argument specified via
|
|
the -o flag when the -o flag was specified before the -c flag on the
|
|
command line, resulting in a failure to compile.
|
|
|
|
(LRK, 2016/11/04, HDFFV-9938, HDFFV-9530)
|
|
|
|
- h5repack User Defined (UD) filter parameters were not parsed correctly.
|
|
|
|
The UD filter parameters were not being parsed correctly. Reworked coding
|
|
section to parse the correct values and verify number of parameters.
|
|
|
|
(ABD, 2016/10/19, HDFFV-9996, HDFFV-9974, HDFFV-9515, HDFFV-9039)
|
|
|
|
- h5repack allows the --enable-error-stack option on the command line.
|
|
|
|
(ADB, 2016/08/08, HDFFV-9775)
|
|
|
|
|
|
C++ APIs
|
|
--------
|
|
- The member function H5Location::getNumObjs() is moved to
|
|
class Group because the objects are in a group or a file only,
|
|
and H5Object::getNumAttrs to H5Location to get the number of
|
|
attributes at a given location.
|
|
|
|
(BMR, 2017/03/17, PR#466)
|
|
|
|
- Due to the change in the C API, the overloaded functions of
|
|
PropList::setProperty now need const for some arguments. They are
|
|
planned for deprecation and are replaced by new versions with proper
|
|
consts.
|
|
|
|
(BMR, 2017/03/17, PR#344)
|
|
|
|
- The high-level API Packet Table (PT) did not write data correctly when
|
|
the datatype is a compound type that has string type as one of the
|
|
members. This problem started in 1.8.15, after the fix of HDFFV-9042
|
|
was applied, which caused the Packet Table to use native type to access
|
|
the data. It should be up to the application to specify whether the
|
|
buffer to be read into memory is in the machine's native architecture.
|
|
Thus, the PT is fixed to not use native type but to make a copy of the
|
|
user's provided datatype during creation or the packet table's datatype
|
|
during opening. If an application wishes to use native type to read the
|
|
data, then the application will request that. However, the Packet Table
|
|
doesn't provide a way to specify memory datatype in this release. This
|
|
feature will be available in future releases.
|
|
|
|
(BMR, 2016/10/27, HDFFV-9758)
|
|
|
|
- The obsolete macros H5_NO_NAMESPACE and H5_NO_STD have been removed from
|
|
the HDF5 C++ API library.
|
|
|
|
(BMR, 2016/10/23, HDFFV-9532)
|
|
|
|
- The problem where a user-defined function cannot access both, attribute
|
|
and dataset, using only one argument is now fixed.
|
|
|
|
(BMR, 2016/10/11, HDFFV-9920)
|
|
|
|
- In-memory array information, ArrayType::rank and
|
|
ArrayType::dimensions, were removed. This is an implementation
|
|
detail and should not affect applications.
|
|
|
|
(BMR, 2016/04/25, HDFFV-9725)
|
|
|
|
|
|
Testing
|
|
-------
|
|
- Fixed a problem that caused tests using SWMR to occasionally fail when
|
|
running "make check" using parallel make.
|
|
|
|
(LRK, 2016/03/22, PR#338, PR#346, PR#358)
|
|
|
|
|
|
Supported Platforms
|
|
===================
|
|
|
|
Linux 2.6.32-573.18.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
|
|
#1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4)
|
|
(ostrich) GNU Fortran (GCC) 4.4.7 20120313
|
|
(Red Hat 4.4.7-4)
|
|
IBM XL C/C++ V13.1
|
|
IBM XL Fortran V15.1
|
|
|
|
Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
Version 4.9.3, Version 5.2.0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 15.0.3.187 Build 20150407
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc
|
|
(emu) Sun Fortran 95 8.6 SunOS_sparc
|
|
Sun C++ 5.12 SunOS_sparc
|
|
|
|
Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
|
|
Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
Visual Studio 2015 w/ MSMPI 8 (cmake)
|
|
Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3)
|
|
gcc and gfortran compilers (GCC 5.4.0)
|
|
(cmake and autotools)
|
|
|
|
Windows 10 Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3)
|
|
gcc and gfortran compilers (GCC 5.4.0)
|
|
(cmake and autotools)
|
|
|
|
Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
|
|
Mac OS X Mt. Lion 10.8.5 Apple clang/clang++ version 5.1 from Xcode 5.1
|
|
64-bit gfortran GNU Fortran (GCC) 4.8.2
|
|
(swallow/kite) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X Mavericks 10.9.5 Apple clang/clang++ version 6.0 from Xcode 6.2
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(wren/quail) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3 from Xcode 7.3
|
|
64-bit gfortran GNU Fortran (GCC) 5.2.0
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 16.0.2
|
|
|
|
|
|
Tested Configuration Features Summary
|
|
=====================================
|
|
|
|
In the tables below
|
|
y = tested
|
|
n = not tested in this release
|
|
C = Cluster
|
|
W = Workstation
|
|
x = not working in this release
|
|
dna = does not apply
|
|
( ) = footnote appears below second table
|
|
<blank> = testing incomplete on this feature or platform
|
|
|
|
Platform C F90/ F90 C++ zlib SZIP
|
|
parallel F2003 parallel
|
|
Solaris2.11 32-bit n y/y n y y y
|
|
Solaris2.11 64-bit n y/n n y y y
|
|
Windows 7 y y/y n y y y
|
|
Windows 7 x64 y y/y y y y y
|
|
Windows 7 Cygwin n y/n n y y y
|
|
Windows 7 x64 Cygwin n y/n n y y y
|
|
Windows 10 y y/y n y y y
|
|
Windows 10 x64 y y/y n y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y
|
|
Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y
|
|
Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 PGI n y/y n y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 GNU y y/y y y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 Intel n y/y n y y y
|
|
Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y
|
|
|
|
|
|
Platform Shared Shared Shared Thread-
|
|
C libs F90 libs C++ libs safe
|
|
Solaris2.11 32-bit y y y y
|
|
Solaris2.11 64-bit y y y y
|
|
Windows 7 y y y y
|
|
Windows 7 x64 y y y y
|
|
Windows 7 Cygwin n n n y
|
|
Windows 7 x64 Cygwin n n n y
|
|
Windows 10 y y y y
|
|
Windows 10 x64 y y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit y n y y
|
|
Mac OS X Mavericks 10.9.5 64-bit y n y y
|
|
Mac OS X Yosemite 10.10.5 64-bit y n y y
|
|
Mac OS X El Capitan 10.11.6 64-bit y n y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 PGI y y y n
|
|
CentOS 7.2 Linux 2.6.32 x86_64 GNU y y y y
|
|
CentOS 7.2 Linux 2.6.32 x86_64 Intel y y y n
|
|
Linux 2.6.32-573.18.1.el6.ppc64 y y y n
|
|
|
|
Compiler versions for each platform are listed in the preceding
|
|
"Supported Platforms" table.
|
|
|
|
|
|
More Tested Platforms
|
|
=====================
|
|
|
|
The following platforms are not supported but have been tested for this release.
|
|
|
|
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(mayll/platypus) Version 4.4.7 20120313
|
|
Version 4.8.4
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86-64;
|
|
Version 16.10-0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 15.0.3.187 (Build 20150407)
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers
|
|
#1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
(jelly) with NAG Fortran Compiler Release 6.1(Tozai)
|
|
Intel(R) C (icc) and C++ (icpc) compilers
|
|
Version 15.0.3.187 (Build 20150407)
|
|
with NAG Fortran Compiler Release 6.1(Tozai)
|
|
|
|
Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with
|
|
#1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1
|
|
(ostrich) and IBM XL Fortran for Linux, V15.1
|
|
|
|
Debian 8.4 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux
|
|
gcc, g++ (Debian 4.9.2-10) 4.9.2
|
|
GNU Fortran (Debian 4.9.2-10) 4.9.2
|
|
(cmake and autotools)
|
|
|
|
Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
|
|
gcc, g++ (GCC) 6.1.1 20160621
|
|
(Red Hat 6.1.1-3)
|
|
GNU Fortran (GCC) 6.1.1 20160621
|
|
(Red Hat 6.1.1-3)
|
|
(cmake and autotools)
|
|
|
|
Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux
|
|
gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2)
|
|
5.4.0 20160609
|
|
GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2)
|
|
5.4.0 20160609
|
|
(cmake and autotools)
|
|
|
|
|
|
Known Problems
|
|
==============
|
|
|
|
At present, metadata cache images may not be generated by parallel
|
|
applications. Parallel applications can read files with metadata cache
|
|
images, but since this is a collective operation, a deadlock is possible
|
|
if one or more processes do not participate.
|
|
|
|
Known problems in previous releases can be found in the HISTORY*.txt files
|
|
in the HDF5 source. Please report any new problems found to
|
|
help@hdfgroup.org.
|
|
|
|
|
|
%%%%1.10.0-patch1%%%%
|
|
|
|
|
|
HDF5 version 1.10.0-patch1 released on 2016-05-23
|
|
================================================================================
|
|
|
|
INTRODUCTION
|
|
|
|
This document describes the differences between HDF5-1.8 series and
|
|
HDF5 1.10.0 releases, and contains information on the platforms
|
|
tested.
|
|
|
|
Links to HDF5 1.10.0 source code can be found on The HDF Group's
|
|
development FTP server at the following location:
|
|
|
|
https://www.hdfgroup.org/HDF5/release/obtain5110.html
|
|
|
|
User documentation can be accessed directly at this location:
|
|
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/
|
|
|
|
For more information, see the HDF5 home page:
|
|
|
|
https://www.hdfgroup.org/HDF5/
|
|
|
|
If you have any questions or comments, please send them to the HDF
|
|
Help Desk:
|
|
|
|
help@hdfgroup.org
|
|
|
|
|
|
|
|
CONTENTS
|
|
|
|
- New Features
|
|
- Issues Addressed in this Release
|
|
- Supported Platforms
|
|
- Tested Configuration Features Summary
|
|
- More Tested Platforms
|
|
- Known Problems and Limitations
|
|
|
|
|
|
|
|
New Features
|
|
============
|
|
This release supports the following features:
|
|
|
|
Configuration
|
|
-------------
|
|
- API Compatibility with HDF5 1.8 Flag Was Added
|
|
|
|
The 1.10 version of the HDF5 Library can be configured to operate
|
|
identically to the 1.8 library with the --with-default-api-version=v18
|
|
configure flag. This allows existing code to be compiled with the 1.10
|
|
library without requiring immediate changes to the application source
|
|
code. For addtional configuration options and other details, see
|
|
"API Compatibility Macros in HDF5" at
|
|
https://www.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html.
|
|
|
|
- Autotools Configuration Has Been Extensively Reworked
|
|
|
|
The autotools configuration options have been updated to allow more
|
|
fine-grained control of the build options and to correct some bugs.
|
|
See configure --help for comprehensive information on each option.
|
|
|
|
Specific changes:
|
|
|
|
* --enable-debug and --enable-production are no longer accepted.
|
|
Use --enable-build-mode=(debug | production) instead. These set
|
|
appropriate defaults for symbols, optimizations, and other
|
|
configuration options. These defaults can be overridden by the
|
|
user.
|
|
|
|
* Extra debug output messages are no longer enabled with
|
|
--enable-debug=<package list>. Use --enable-internal-debug=<pkg list>
|
|
instead.
|
|
|
|
* A new --enable-symbols option allows symbols to be generated
|
|
independently of the build mode. --disable-symbols can be used
|
|
to strip symbols from the binary.
|
|
|
|
* A new --enable-asserts option sets/unsets NDEBUG. This is
|
|
independent of the build mode. This also enables some extra
|
|
low-overhead debug checks in the library.
|
|
|
|
* A new --enable-profiling option sets profiling flags. This is
|
|
independent of the build mode.
|
|
|
|
* A new --enable-optimization option sets the optimization level.
|
|
This is independent of the build mode.
|
|
|
|
* Many of these options can take a flags string that will be used
|
|
to build the library. This can be useful for specifying custom
|
|
optimization flags such as -Os and -Ofast.
|
|
|
|
* gnu C++ and Fortran use configure sub-files that update the
|
|
build flags and turn on warnings. The increase in warnings when
|
|
building these wrapper libraries is due to these flag changes
|
|
and not to a decrease in code quality.
|
|
|
|
* The option to clear file buffers has been removed. Any buffer that
|
|
will eventually be written to disk will now always be memset
|
|
to zero. This prevents the previous contents of the buffer from
|
|
being written to the disk if the buffer contents are not
|
|
completely overwritten, which has security implications.
|
|
|
|
- LFS Changes
|
|
|
|
The way the autotools handle large file support (LFS) has been
|
|
overhauled in this release.
|
|
|
|
* We assume ftello and fseeko exist
|
|
|
|
* We no longer explicitly use the *64 I/O functions. Instead, we
|
|
rely on a mapping provided by _FILE_OFFSET_BITS or its equivalent.
|
|
|
|
* _LARGEFILE(64)_SOURCE is no longer exported via AM_CPPFLAGS.
|
|
|
|
|
|
|
|
Parallel Library
|
|
-----------------
|
|
- Collective Metadata I/O
|
|
|
|
Calls for HDF5 metadata can result in many small reads and writes.
|
|
On metadata reads, collective metadata I/O can improve performance
|
|
by allowing the library to perform optimizations when reading the
|
|
metadata by having one rank read the data and broadcasting it to
|
|
all other ranks.
|
|
|
|
Collective metadata I/O improves metadata write performance through
|
|
the construction of an MPI derived datatype that is then written
|
|
collectively in a single call. For more information, see
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesCollectiveMetadataIoDocs.html.
|
|
|
|
|
|
|
|
Library
|
|
--------
|
|
- Concurrent Access to HDF5 Files - Single Writer/ Multple Reader (SWMR)
|
|
|
|
The Single Writer/ Multiple Reader or SWMR feature enables users to
|
|
read data concurrently while writing it. Communications between the
|
|
processes and file locking are not required. The processes can run
|
|
on the same or on different platforms as long as they share a common
|
|
file system that is POSIX compliant. For more information, see the
|
|
Single-Writer/Multiple-Reader (SWMR) documentation at
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html.
|
|
|
|
- Virtual Dataset (VDS)
|
|
|
|
The VDS feature enables data to be accessed across HDF5 files
|
|
using standard HDF5 objects such as groups and datasets without
|
|
rewriting or rearranging the data. An HDF5 virtual dataset (VDS)
|
|
is an HDF5 dataset that is composed of source HDF5 datasets in
|
|
a predefined mapping. VDS can be used with the SWMR feature. For
|
|
documentation, check
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html.
|
|
|
|
- Persistent Free File Space Tracking
|
|
|
|
Usage patterns when working with an HDF5 file sometimes result in
|
|
wasted space within the file. This can also impair access times
|
|
when working with the resulting files. The new file space management
|
|
feature provides strategies for managing space in a file to improve
|
|
performance in both of these areas. For more information, see
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesFileSpaceMgmtDocs.html.
|
|
|
|
- Version 3 Metadata Cache
|
|
|
|
The version 3 metadata cache moves management of metadata I/O from
|
|
the clients to the metadata cache proper. This change is essential for
|
|
SWMR and other features that have yet to be released.
|
|
|
|
|
|
|
|
C++ Library
|
|
------------
|
|
- New Member Function Added to H5::ArrayType
|
|
|
|
The assignment operator ArrayType::operator= was added because
|
|
ArrayType has pointer data members.
|
|
|
|
(BMR - 2016/03/07, HDFFV-9562)
|
|
|
|
|
|
|
|
Tools
|
|
------
|
|
- h5watch
|
|
|
|
The h5watch tool allows users to output new records appended to
|
|
a dataset under SWMR access as it grows. The functionality is
|
|
similar to the Unix user command "tail" with the follow option,
|
|
which outputs appended data as the file grows. For more
|
|
information, see
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html#Tools.
|
|
|
|
- h5format_convert
|
|
|
|
The h5format_convert tool allows users to convert the indexing
|
|
type of a chunked dataset made with a 1.10.x version of the HDF5
|
|
Library when the latest file format is used to the 1.8.x version 1 B-tree indexing
|
|
type. For example, datasets created using SWMR access, can be
|
|
converted to be accessed by the HDF5 1.18 library and tools. The
|
|
tool does not rewrite raw data, but it does rewrite HDF5 metadata.
|
|
|
|
|
|
|
|
High-Level APIs
|
|
----------------
|
|
- H5DOappend
|
|
|
|
The function appends data to a dataset along a specified dimension.
|
|
|
|
|
|
C Packet Table API
|
|
------------------
|
|
- Replacement of a Public Function with H5PTcreate
|
|
|
|
The existing function H5PTcreate_fl limits applications so they
|
|
can use the deflate compression only. The public function
|
|
H5PTcreate has been added to replace H5PTcreate_fl. H5PTcreate
|
|
takes a property list identifier to provide flexibility on
|
|
creation properties.
|
|
|
|
(BMR - 2016/03/04, HDFFV-8623)
|
|
|
|
- New Public Functions: H5PTget_dataset and H5PTget_type
|
|
|
|
Two accessor functions have been added. H5PTget_dataset returns
|
|
the identifier of the dataset associated with the packet table,
|
|
and H5PTget_type returns the identifier of the datatype used by
|
|
the packet table.
|
|
|
|
(BMR, 2016/03/04, HDFFV-8623)
|
|
|
|
- Regarding #ifdef VLPT_REMOVED
|
|
|
|
The #ifdef VLPT_REMOVED blocks have been removed from the packet
|
|
table (PT) library source except for the following functions:
|
|
+ H5PTis_varlen() has been made available again
|
|
+ H5PTfree_vlen_readbuff() is now H5PTfree_vlen_buff()
|
|
|
|
(BMR - 2016/03/04, HDFFV-442)
|
|
|
|
C++ Packet Table API
|
|
--------------------
|
|
- New Constructor Added to FL_PacketTable
|
|
|
|
An overloaded constructor has been added to FL_PacketTable and
|
|
takes a property list identifier to provide flexibility on
|
|
creation properties.
|
|
|
|
(BMR - 2016/03/08, HDFFV-8623)
|
|
|
|
- New Public Functions
|
|
|
|
Two accessor wrappers are added to class PacketTable.
|
|
PacketTable::GetDataset() returns the identifier of the dataset
|
|
associated with the packet table, and PacketTable::GetDatatype()
|
|
returns the identifier of the datatype that the packet table uses.
|
|
|
|
(BMR - 2016/03/04, HDFFV-8623)
|
|
|
|
- Member Functions with "char*" as an Argument
|
|
|
|
Overloaded functions were added to provide the "const char*"
|
|
argument; the existing version will be deprecated in future
|
|
releases.
|
|
|
|
(BMR - 2016/03/04, HDFFV-8623)
|
|
|
|
- Regarding #ifdef VLPT_REMOVED
|
|
|
|
The #ifdef VLPT_REMOVED blocks have been removed from the packet
|
|
table library source code except for the following functions:
|
|
+ VL_PacketTable::IsVariableLength() was moved to PacketTable
|
|
+ VL_PacketTable::FreeReadBuff() is now PacketTable::FreeBuff()
|
|
|
|
(BMR - 2016/03/04, HDFFV-442)
|
|
|
|
|
|
|
|
Java Wrapper Library
|
|
--------------------
|
|
|
|
The Java HDF5 JNI library has been integrated into the HDF5 repository.
|
|
The configure option is "--enable-java", and the CMake option is
|
|
HDF5_BUILD_JAVA:BOOL=ON. The package hierarchy has changed from the
|
|
HDF5 1.8 JNI, which was "ncsa.hdf.hdflib.hdf5", to HDF5 1.10,
|
|
"hdf.hdflib.hdf5".
|
|
|
|
A number of new APIs were added including some for VDS and SWMR.
|
|
|
|
|
|
|
|
Other Important Changes
|
|
-----------------------
|
|
|
|
The hid_t type was changed from 32-bit to a 64-bit value.
|
|
|
|
|
|
|
|
Issues Addressed in this Release Since 1.10.0
|
|
=============================================
|
|
|
|
- h5diff would return from a compare attributes abnormally if one of the datatypes
|
|
was a vlen. This resulted in a memory leak as well as an incorrect report of
|
|
attribute comparison.
|
|
|
|
Fixed.
|
|
(ADB - 2016/04/26, HDFFV-9784)
|
|
|
|
- The JUnit-interface test may fail on Solaris platforms. The result of
|
|
a test for verifying the content of the error stack to stdout is
|
|
in a different order on Solaris then other platforms.
|
|
|
|
This test is skipped on Solaris
|
|
(ADB - 2016/04/21, HDFFV-9734)
|
|
|
|
- When building HDF5 with Java using CMake and specifying Debug for CMAKE_BUILD_TYPE,
|
|
there was a missing command argument for the tests of the examples.
|
|
|
|
Fixed.
|
|
(ADB - 2016/04/21, HDFFV-9743)
|
|
|
|
- Changed h5diff to print a warning when a dataset is virtual, enabling
|
|
the data to be compared. In addition h5repack failed to copy the data
|
|
of a virtual dataset to the new file. Function H5D__get_space_status changed
|
|
to correctly determine the H5D_space_status_t allocation value.
|
|
|
|
CMake added the Fixed Array indexing tests that were only in the autotools
|
|
test scripts.
|
|
|
|
Fixed and tests added for vds issues.
|
|
(ADB,NAF - 2016/04/21, HDFFV-9756)
|
|
|
|
- CMake added the h5format_convert tool and tests that were only in the autotools
|
|
build and test scripts. The autotools test script was reworked to allow CMake
|
|
to execute the test suite in parallel.
|
|
|
|
Also, h5clear tool and tests were added to the misc folder.
|
|
|
|
Fixed.
|
|
(ADB - 2016/04/21, HDFFV-9766)
|
|
|
|
- CMake added the h5watch tool and argument tests that were only in the autotools
|
|
build and test scripts. The POSIX only tests were not added to CMake.
|
|
|
|
CMake HL tools files were refactored to move the CMake test scripts into each tool folder.
|
|
|
|
Fixed.
|
|
(ADB - 2016/04/21, HDFFV-9770)
|
|
|
|
- Configure fails to detect valid real KINDs on FreeBSD 9.3 (i386) with Fortran enabled.
|
|
|
|
Fixed. Added the exponential option to SELECTED_REAL_KIND to distinguish
|
|
KINDs of same precision
|
|
(MSB - 2016/05/14,HDFFV-9912)
|
|
|
|
|
|
- Corrected the f90 H5AWRITE_F integer interface's buf to be INTENT(IN).
|
|
(MSB - 2016/05/14)
|
|
|
|
- Configure fails in sed command on FreeBSD 9.3 (i386) with Fortran enabled.
|
|
|
|
Fixed.
|
|
(MSB - 2016/05/14,HDFFV-9912)
|
|
|
|
- Compile time error in H5f90global.F90 with IBM XL Fortran 14.1.0.13 on BG/Q with Fortran
|
|
enabled.
|
|
|
|
Fixed.
|
|
(MSB - 2016/05/16,HDFFV-9917)
|
|
|
|
- A cmake build with Fortran enabled does not install module h5fortkit
|
|
|
|
Fixed.
|
|
(MSB - 2016/05/23,HDFFV-9923)
|
|
|
|
|
|
Issues Addressed in this Release Since alpha1
|
|
=============================================
|
|
|
|
- H5Pget_virtual_printf_gap, H5Pget_virtual_view, H5Pget_efile_prefix
|
|
|
|
The correct access property list settings from the
|
|
H5Pget_virtual_printf_gap, H5Pget_virtual_view, and
|
|
H5Pget_efile_prefix function calls could not be retrieved
|
|
using H5Dget_access_plist().
|
|
|
|
Fixed.
|
|
|
|
(DER and NAF - 2016/03/14, HDFFV-9716)
|
|
|
|
- h5dump
|
|
|
|
When h5dump was provided with the name of a non-existing file or
|
|
when optional arguments were the last option on the command line,
|
|
h5dump would segfault.
|
|
|
|
Fixed.
|
|
|
|
(ADB 2016/02/28 HDFFV-9639, HDFFV-9684)
|
|
|
|
- No Error Message for Corrupt Metadata
|
|
|
|
The HDF5 Library did not propagate an error when it encountered
|
|
corrupt metadata in an HDF5 file. The issue was fixed for a
|
|
specific file provided by a user. If you still see the problem,
|
|
please contact help@hdfgroup.org
|
|
|
|
Fixed.
|
|
|
|
(MC - 2016/02/18, HDFFV-9670)
|
|
|
|
- Problem Reading Chunked Datasets with a String Datatype Larger
|
|
Than the Chunk Size in Bytes
|
|
|
|
When the latest file format was used and when a chunked dataset
|
|
was created with a datatype with the size bigger than a chunk
|
|
size, the data could not be read back. The issue was reported
|
|
for chunked datasets with a string datatype and was confirmed
|
|
for other datatypes with the sizes bigger than the chunk size in
|
|
bytes.
|
|
|
|
Fixed.
|
|
|
|
(JM - 2016/02/13, HDFFV-9672)
|
|
|
|
- Control over the Location of External Files
|
|
|
|
Users were unable to specify the locations of external files.
|
|
|
|
Two APIs - H5Pget_efile_prefix and H5Pset_efile_prefix - were
|
|
added so that users could specify the locations of external files.
|
|
|
|
(DER - 2016/02/04, HDFFV-8740)
|
|
|
|
|
|
|
|
Issues Addressed in this Release Since alpha0
|
|
=============================================
|
|
- h5format_convert
|
|
|
|
The h5format_convert tool did not downgrade the version of the
|
|
superblock.
|
|
|
|
Fixed. The tool now will downgrade the version of the superblock.
|
|
|
|
(EIP 2016/01/11)
|
|
|
|
- Crashes with multiple threads: invalid pointers
|
|
|
|
It was reported that alpha0 crashed when used with multiple
|
|
threads. The issue exists in the HDF5 Library versions 1.8 and
|
|
1.9. The problem is related to a shared file pointer used in some
|
|
miscellaneous data structures. The thread-safe library exposed
|
|
paths in the library where a file pointer became invalid.
|
|
|
|
The alpha1 release contains the fixes for the specific use case
|
|
as described in HDFFV-9643. We will keep working on identifying
|
|
and fixing other paths in the library with similar problems.
|
|
|
|
(EIP - 2016/01/15, HDFFV-9643)
|
|
|
|
|
|
|
|
Supported Platforms
|
|
===================
|
|
The following platforms are supported and have been tested for this release.
|
|
They are built with the configure process unless specified otherwise.
|
|
|
|
AIX 6.1 xlc/xlc_r 10.1.0.5
|
|
(NASA G-ADA) xlC/xlC_r 10.1.0.5
|
|
xlf90/xlf90_r 12.1.0.6
|
|
|
|
Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(mayll/platypus) Version 4.4.7 20120313 (Red Hat 4.4.7-16)
|
|
Version 4.9.3, Version 5.2.0
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86-64;
|
|
Version 15.7-0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 15.0.3.187 Build 20150407
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
Linux 2.6.32-573.18.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
|
|
#1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
|
|
(ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
|
|
IBM XL C/C++ V13.1
|
|
IBM XL Fortran V15.1
|
|
|
|
Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
Version 4.9.3, Version 5.2.0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 15.0.3.187 Build 20150407
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc
|
|
(emu) Sun Fortran 95 8.6 SunOS_sparc
|
|
Sun C++ 5.12 SunOS_sparc
|
|
|
|
Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
Cygwin(CYGWIN_NT-6.1 2.2.1(0.289/5/3) gcc(4.9.3) compiler and gfortran)
|
|
(cmake and autotools)
|
|
|
|
Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
|
|
Windows 8.1 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
|
|
Windows 8.1 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
|
|
Mac OS X Mt. Lion 10.8.5 Apple clang/clang++ version 5.1 from Xcode 5.1
|
|
64-bit gfortran GNU Fortran (GCC) 4.8.2
|
|
(swallow/kite) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X Mavericks 10.9.5 Apple clang/clang++ version 6.0 from Xcode 6.2
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(wren/quail) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.0 from Xcode 7.0
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X El Capitan 10.11.4 Apple clang/clang++ version 7.3.0 from Xcode 7.3
|
|
64-bit gfortran GNU Fortran (GCC) 5.2.0
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
|
|
|
|
Tested Configuration Features Summary
|
|
=====================================
|
|
|
|
In the tables below
|
|
y = tested
|
|
n = not tested in this release
|
|
C = Cluster
|
|
W = Workstation
|
|
x = not working in this release
|
|
dna = does not apply
|
|
( ) = footnote appears below second table
|
|
<blank> = testing incomplete on this feature or platform
|
|
|
|
Platform C F90/ F90 C++ zlib SZIP
|
|
parallel F2003 parallel
|
|
SunOS 5.11 32-bit n y/y n y y y
|
|
SunOS 5.11 64-bit n y/y n y y y
|
|
Windows 7 y y/y n y y y
|
|
Windows 7 x64 y y/y n y y y
|
|
Windows 7 Cygwin n y/y n y y n
|
|
Windows 8.1 n y/y n y y y
|
|
Windows 8.1 x64 n y/y n y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y
|
|
Mac OS X Yosemeti 10.10.5 64-bit n y/y n y y y
|
|
AIX 6.1 32- and 64-bit n y/n n y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 GNU y y/y y y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 Intel n y/y n y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 PGI n y/y n y y y
|
|
CentOS 7.1 Linux 3.10.0 x86_64 GNU y y/y y y y y
|
|
CentOS 7.1 Linux 3.10.0 x86_64 Intel n y/y n y y y
|
|
Linux 2.6.32-431.11.2.el6.ppc64 n y/n n y y y
|
|
|
|
Platform Shared Shared Shared Thread-
|
|
C libs F90 libs C++ libs safe
|
|
SunOS 5.11 32-bit y y y y
|
|
SunOS 5.11 64-bit y y y y
|
|
Windows 7 y y y y
|
|
Windows 7 x64 y y y y
|
|
Windows 7 Cygwin n n n y
|
|
Windows 8.1 y y y y
|
|
Windows 8.1 x64 y y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit y n y y
|
|
Mac OS X Mavericks 10.9.5 64-bit y n y y
|
|
Mac OS X Yosemeti 10.10.5 64-bit y n y y
|
|
AIX 6.1 32- and 64-bit y n n y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 GNU y y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 Intel y y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 PGI y y y y
|
|
CentOS 7.1 Linux 3.10.0 x86_64 GNU y y y y
|
|
CentOS 7.1 Linux 3.10.0 x86_64 Intel y y y y
|
|
Linux 2.6.32-431.11.2.el6.ppc64 y y y y
|
|
|
|
Compiler versions for each platform are listed in the preceding
|
|
"Supported Platforms" table.
|
|
|
|
|
|
|
|
More Tested Platforms
|
|
=====================
|
|
The following platforms are not supported but have been tested for this release.
|
|
|
|
Linux 2.6.18-431.11.2.el6 g95 (GCC 4.0.3 (g95 0.94!)
|
|
#1 SMP x86_64 GNU/Linux
|
|
(platypus)
|
|
|
|
Windows 7 Visual Studio 2008 (cmake)
|
|
|
|
Windows 7 x64 Visual Studio 2008 (cmake)
|
|
|
|
Windows 7 x64 Visual Studio 2010 (cmake) with SWMR using GPFS
|
|
|
|
Windows 10 Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
|
|
Windows 10 x64 Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
|
|
Debian7.5.0 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux
|
|
gcc (Debian 4.7.2-5) 4.7.2
|
|
GNU Fortran (Debian 4.7.2-5) 4.7.2
|
|
(cmake and autotools)
|
|
|
|
Fedora20 3.15.3-200.fc20.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
|
|
gcc (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1)
|
|
GNU Fortran (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1)
|
|
(cmake and autotools)
|
|
|
|
SUSE 13.1 3.11.10-17-desktop #1 SMP PREEMPT x86_64 x86_64 x86_64 GNU/Linux
|
|
gcc (SUSE Linux) 4.8.1
|
|
GNU Fortran (SUSE Linux) 4.8.1
|
|
(cmake and autotools)
|
|
|
|
Ubuntu 14.04 3.13.0-35-generic #62-Ubuntu SMP x86_64 GNU/Linux
|
|
gcc (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1
|
|
GNU Fortran (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1
|
|
(cmake and autotools)
|
|
|
|
hopper.nersc.gov PrgEnv-gnu/5.2.40
|
|
gcc (GCC) 4.9.2 20141030 (Cray Inc.)
|
|
GNU Fortran (GCC) 4.9.2 20141030 (Cray Inc.)
|
|
g++ (GCC) 4.9.2 20141030 (Cray Inc.)
|
|
|
|
|
|
|
|
Known Problems and Limitations
|
|
==============================
|
|
This section contains the list of known problems and limitations introduced
|
|
in this release of HDF5.
|
|
|
|
Note: this list is not exhaustive of all known issues discovered in HDF5
|
|
software to date. For a list of significant problems and known workarounds
|
|
identified in past releases, please refer to:
|
|
|
|
https://www.hdfgroup.org/HDF5/release/known_problems/
|
|
|
|
The HDF Group also maintains a JIRA issue-tracking database which is used to
|
|
capture all known issues which are too numerous to reasonably list in this
|
|
document. The HDF Group is taking steps to make our JIRA issue database
|
|
open to the public, and this section will refer to that database in a future
|
|
release. In the meantime, please contact help@hdfgroup.org if you come across
|
|
an issue not listed here or at the link above, and we will provide any
|
|
information about known workarounds that we have or add it to our list of
|
|
known issues if it is a new issue.
|
|
|
|
- The flush/refresh test occasionally fails on OS X platforms. This is
|
|
being investigated but no fix or workaround is available at this time.
|
|
(DER - 2016/03/22, HDFFV-9731)
|
|
|
|
- The VDS/SWMR test will fail with a segmentation fault if the library
|
|
is built with --enable-using-memchecker. The is due to a VDS shutdown
|
|
procedure freeing a shared resource too early when the memory
|
|
checker changes are built. This problem does not arise when the
|
|
memory checker changes are not used since the internal library free
|
|
lists behave differently. The memory checker configure option should
|
|
normally only be used under special circumstances so this should not
|
|
affect most users. Users should be aware that the --enable-using-memchecker
|
|
+ VDS combination may cause a segfault, however, so Valgrind et al. may
|
|
have to be used with an HDF5 library built without the feature if this
|
|
proves to be a problem.
|
|
(DER - 2016/03/21, HDFFV-9732)
|
|
|
|
- SWMR feature limitations
|
|
The SWMR feature will only work if an HDF5 file under SWMR access resides
|
|
on a file system that obeys POSIX write() ordering semantics. Because of
|
|
this, SWMR will not work on network file systems such as NFS or SMB/Windows
|
|
file shares since those systems do not guarantee write odering. SWMR
|
|
regression tests are likely to fail if run on a network file system. SWMR
|
|
is currently not tested on Windows though it can be tested manually
|
|
(some of the SWMR test programs are built by CMake), and there are no
|
|
obvious reasons for it to not work on NTFS or GPFS.
|
|
(EIP - 2016/03/20, HDFFV-9733)
|
|
|
|
- VDS feature limitation
|
|
Currently, the path to a VDS source file is interpreted as relative to the
|
|
directory where the executable program runs and not to the HDF5 file with
|
|
the VDS dataset unless a full path to the source file is specified during
|
|
the mapping.
|
|
(EIP - 2016/03/20, HDFFV-9724)
|
|
|
|
- The H5Lexists API changed behavior in HDF5-1.10 when used with a file handle
|
|
and root group name ("/"):
|
|
|
|
H5Lexists(fileid, "/")
|
|
|
|
In HDF5-1.8 it returns false (0) and in HDF5-1.10 it returns true (1).
|
|
The documentation will be updated with information regarding this change.
|
|
(LRK - 2016/03/30, HDFFV-8746)
|
|
|
|
|
|
%%%%1.10.0%%%%
|
|
|
|
HDF5 version 1.10.0 released on 2016-03-30
|
|
================================================================================
|
|
|
|
|
|
|
|
INTRODUCTION
|
|
|
|
This document describes the differences between HDF5-1.8 series and
|
|
HDF5 1.10.0 releases, and contains information on the platforms
|
|
tested.
|
|
|
|
Links to HDF5 1.10.0 source code can be found on The HDF Group's
|
|
development FTP server at the following location:
|
|
|
|
https://www.hdfgroup.org/HDF5/release/obtain5110.html
|
|
|
|
User documentation can be accessed directly at this location:
|
|
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/
|
|
|
|
For more information, see the HDF5 home page:
|
|
|
|
https://www.hdfgroup.org/HDF5/
|
|
|
|
If you have any questions or comments, please send them to the HDF
|
|
Help Desk:
|
|
|
|
help@hdfgroup.org
|
|
|
|
|
|
|
|
CONTENTS
|
|
|
|
- New Features
|
|
- Issues Addressed in this Release
|
|
- Supported Platforms
|
|
- Tested Configuration Features Summary
|
|
- More Tested Platforms
|
|
- Known Problems and Limitations
|
|
|
|
|
|
|
|
New Features
|
|
============
|
|
This release supports the following features:
|
|
|
|
Configuration
|
|
-------------
|
|
- API Compatibility with HDF5 1.8 Flag Was Added
|
|
|
|
The 1.10 version of the HDF5 Library can be configured to operate
|
|
identically to the 1.8 library with the --with-default-api-version=v18
|
|
configure flag. This allows existing code to be compiled with the 1.10
|
|
library without requiring immediate changes to the application source
|
|
code. For addtional configuration options and other details, see
|
|
"API Compatibility Macros in HDF5" at
|
|
https://www.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html.
|
|
|
|
- Autotools Configuration Has Been Extensively Reworked
|
|
|
|
The autotools configuration options have been updated to allow more
|
|
fine-grained control of the build options and to correct some bugs.
|
|
See configure --help for comprehensive information on each option.
|
|
|
|
Specific changes:
|
|
|
|
* --enable-debug and --enable-production are no longer accepted.
|
|
Use --enable-build-mode=(debug | production) instead. These set
|
|
appropriate defaults for symbols, optimizations, and other
|
|
configuration options. These defaults can be overridden by the
|
|
user.
|
|
|
|
* Extra debug output messages are no longer enabled with
|
|
--enable-debug=<package list>. Use --enable-internal-debug=<pkg list>
|
|
instead.
|
|
|
|
* A new --enable-symbols option allows symbols to be generated
|
|
independently of the build mode. --disable-symbols can be used
|
|
to strip symbols from the binary.
|
|
|
|
* A new --enable-asserts option sets/unsets NDEBUG. This is
|
|
independent of the build mode. This also enables some extra
|
|
low-overhead debug checks in the library.
|
|
|
|
* A new --enable-profiling option sets profiling flags. This is
|
|
independent of the build mode.
|
|
|
|
* A new --enable-optimization option sets the optimization level.
|
|
This is independent of the build mode.
|
|
|
|
* Many of these options can take a flags string that will be used
|
|
to build the library. This can be useful for specifying custom
|
|
optimization flags such as -Os and -Ofast.
|
|
|
|
* gnu C++ and Fortran use configure sub-files that update the
|
|
build flags and turn on warnings. The increase in warnings when
|
|
building these wrapper libraries is due to these flag changes
|
|
and not to a decrease in code quality.
|
|
|
|
* The option to clear file buffers has been removed. Any buffer that
|
|
will eventually be written to disk will now always be memset
|
|
to zero. This prevents the previous contents of the buffer from
|
|
being written to the disk if the buffer contents are not
|
|
completely overwritten, which has security implications.
|
|
|
|
- LFS Changes
|
|
|
|
The way the autotools handle large file support (LFS) has been
|
|
overhauled in this release.
|
|
|
|
* We assume ftello and fseeko exist
|
|
|
|
* We no longer explicitly use the *64 I/O functions. Instead, we
|
|
rely on a mapping provided by _FILE_OFFSET_BITS or its equivalent.
|
|
|
|
* _LARGEFILE(64)_SOURCE is no longer exported via AM_CPPFLAGS.
|
|
|
|
|
|
|
|
Parallel Library
|
|
-----------------
|
|
- Collective Metadata I/O
|
|
|
|
Calls for HDF5 metadata can result in many small reads and writes.
|
|
On metadata reads, collective metadata I/O can improve performance
|
|
by allowing the library to perform optimizations when reading the
|
|
metadata by having one rank read the data and broadcasting it to
|
|
all other ranks.
|
|
|
|
Collective metadata I/O improves metadata write performance through
|
|
the construction of an MPI derived datatype that is then written
|
|
collectively in a single call. For more information, see
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesCollectiveMetadataIoDocs.html.
|
|
|
|
|
|
|
|
Library
|
|
--------
|
|
- Concurrent Access to HDF5 Files - Single Writer/ Multple Reader (SWMR)
|
|
|
|
The Single Writer/ Multiple Reader or SWMR feature enables users to
|
|
read data concurrently while writing it. Communications between the
|
|
processes and file locking are not required. The processes can run
|
|
on the same or on different platforms as long as they share a common
|
|
file system that is POSIX compliant. For more information, see the
|
|
Single-Writer/Multiple-Reader (SWMR) documentation at
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html.
|
|
|
|
- Virtual Dataset (VDS)
|
|
|
|
The VDS feature enables data to be accessed across HDF5 files
|
|
using standard HDF5 objects such as groups and datasets without
|
|
rewriting or rearranging the data. An HDF5 virtual dataset (VDS)
|
|
is an HDF5 dataset that is composed of source HDF5 datasets in
|
|
a predefined mapping. VDS can be used with the SWMR feature. For
|
|
documentation, check
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html.
|
|
|
|
- Persistent Free File Space Tracking
|
|
|
|
Usage patterns when working with an HDF5 file sometimes result in
|
|
wasted space within the file. This can also impair access times
|
|
when working with the resulting files. The new file space management
|
|
feature provides strategies for managing space in a file to improve
|
|
performance in both of these areas. For more information, see
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesFileSpaceMgmtDocs.html.
|
|
|
|
- Version 3 Metadata Cache
|
|
|
|
The version 3 metadata cache moves management of metadata I/O from
|
|
the clients to the metadata cache proper. This change is essential for
|
|
SWMR and other features that have yet to be released.
|
|
|
|
|
|
|
|
C++ Library
|
|
------------
|
|
- New Member Function Added to H5::ArrayType
|
|
|
|
The assignment operator ArrayType::operator= was added because
|
|
ArrayType has pointer data members.
|
|
|
|
(BMR - 2016/03/07, HDFFV-9562)
|
|
|
|
|
|
|
|
Tools
|
|
------
|
|
- h5watch
|
|
|
|
The h5watch tool allows users to output new records appended to
|
|
a dataset under SWMR access as it grows. The functionality is
|
|
similar to the Unix user command "tail" with the follow option,
|
|
which outputs appended data as the file grows. For more
|
|
information, see
|
|
https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html#Tools.
|
|
|
|
- h5format_convert
|
|
|
|
The h5format_convert tool allows users to convert the indexing
|
|
type of a chunked dataset made with a 1.10.x version of the HDF5
|
|
Library when the latest file format is used to the 1.8.x version 1 B-tree indexing
|
|
type. For example, datasets created using SWMR access, can be
|
|
converted to be accessed by the HDF5 1.18 library and tools. The
|
|
tool does not rewrite raw data, but it does rewrite HDF5 metadata.
|
|
|
|
|
|
|
|
High-Level APIs
|
|
----------------
|
|
- H5DOappend
|
|
|
|
The function appends data to a dataset along a specified dimension.
|
|
|
|
|
|
C Packet Table API
|
|
------------------
|
|
- Replacement of a Public Function with H5PTcreate
|
|
|
|
The existing function H5PTcreate_fl limits applications so they
|
|
can use the deflate compression only. The public function
|
|
H5PTcreate has been added to replace H5PTcreate_fl. H5PTcreate
|
|
takes a property list identifier to provide flexibility on
|
|
creation properties.
|
|
|
|
(BMR - 2016/03/04, HDFFV-8623)
|
|
|
|
- New Public Functions: H5PTget_dataset and H5PTget_type
|
|
|
|
Two accessor functions have been added. H5PTget_dataset returns
|
|
the identifier of the dataset associated with the packet table,
|
|
and H5PTget_type returns the identifier of the datatype used by
|
|
the packet table.
|
|
|
|
(BMR, 2016/03/04, HDFFV-8623)
|
|
|
|
- Regarding #ifdef VLPT_REMOVED
|
|
|
|
The #ifdef VLPT_REMOVED blocks have been removed from the packet
|
|
table (PT) library source except for the following functions:
|
|
+ H5PTis_varlen() has been made available again
|
|
+ H5PTfree_vlen_readbuff() is now H5PTfree_vlen_buff()
|
|
|
|
(BMR - 2016/03/04, HDFFV-442)
|
|
|
|
C++ Packet Table API
|
|
--------------------
|
|
- New Constructor Added to FL_PacketTable
|
|
|
|
An overloaded constructor has been added to FL_PacketTable and
|
|
takes a property list identifier to provide flexibility on
|
|
creation properties.
|
|
|
|
(BMR - 2016/03/08, HDFFV-8623)
|
|
|
|
- New Public Functions
|
|
|
|
Two accessor wrappers are added to class PacketTable.
|
|
PacketTable::GetDataset() returns the identifier of the dataset
|
|
associated with the packet table, and PacketTable::GetDatatype()
|
|
returns the identifier of the datatype that the packet table uses.
|
|
|
|
(BMR - 2016/03/04, HDFFV-8623)
|
|
|
|
- Member Functions with "char*" as an Argument
|
|
|
|
Overloaded functions were added to provide the "const char*"
|
|
argument; the existing version will be deprecated in future
|
|
releases.
|
|
|
|
(BMR - 2016/03/04, HDFFV-8623)
|
|
|
|
- Regarding #ifdef VLPT_REMOVED
|
|
|
|
The #ifdef VLPT_REMOVED blocks have been removed from the packet
|
|
table library source code except for the following functions:
|
|
+ VL_PacketTable::IsVariableLength() was moved to PacketTable
|
|
+ VL_PacketTable::FreeReadBuff() is now PacketTable::FreeBuff()
|
|
|
|
(BMR - 2016/03/04, HDFFV-442)
|
|
|
|
|
|
|
|
Java Wrapper Library
|
|
--------------------
|
|
|
|
The Java HDF5 JNI library has been integrated into the HDF5 repository.
|
|
The configure option is "--enable-java", and the CMake option is
|
|
HDF5_BUILD_JAVA:BOOL=ON. The package hierarchy has changed from the
|
|
HDF5 1.8 JNI, which was "ncsa.hdf.hdflib.hdf5", to HDF5 1.10,
|
|
"hdf.hdflib.hdf5".
|
|
|
|
A number of new APIs were added including some for VDS and SWMR.
|
|
|
|
|
|
|
|
Other Important Changes
|
|
-----------------------
|
|
|
|
The hid_t type was changed from 32-bit to a 64-bit value.
|
|
|
|
|
|
|
|
Issues Addressed in this Release Since alpha1
|
|
=============================================
|
|
|
|
- H5Pget_virtual_printf_gap, H5Pget_virtual_view, H5Pget_efile_prefix
|
|
|
|
The correct access property list settings from the
|
|
H5Pget_virtual_printf_gap, H5Pget_virtual_view, and
|
|
H5Pget_efile_prefix function calls could not be retrieved
|
|
using H5Dget_access_plist().
|
|
|
|
Fixed.
|
|
|
|
(DER and NAF - 2016/03/14, HDFFV-9716)
|
|
|
|
- h5dump
|
|
|
|
When h5dump was provided with the name of a non-existing file or
|
|
when optional arguments were the last option on the command line,
|
|
h5dump would segfault.
|
|
|
|
Fixed.
|
|
|
|
(ADB 2016/02/28 HDFFV-9639, HDFFV-9684)
|
|
|
|
- No Error Message for Corrupt Metadata
|
|
|
|
The HDF5 Library did not propagate an error when it encountered
|
|
corrupt metadata in an HDF5 file. The issue was fixed for a
|
|
specific file provided by a user. If you still see the problem,
|
|
please contact help@hdfgroup.org
|
|
|
|
Fixed.
|
|
|
|
(MC - 2016/02/18, HDFFV-9670)
|
|
|
|
- Problem Reading Chunked Datasets with a String Datatype Larger
|
|
Than the Chunk Size in Bytes
|
|
|
|
When the latest file format was used and when a chunked dataset
|
|
was created with a datatype with the size bigger than a chunk
|
|
size, the data could not be read back. The issue was reported
|
|
for chunked datasets with a string datatype and was confirmed
|
|
for other datatypes with the sizes bigger than the chunk size in
|
|
bytes.
|
|
|
|
Fixed.
|
|
|
|
(JM - 2016/02/13, HDFFV-9672)
|
|
|
|
- Control over the Location of External Files
|
|
|
|
Users were unable to specify the locations of external files.
|
|
|
|
Two APIs - H5Pget_efile_prefix and H5Pset_efile_prefix - were
|
|
added so that users could specify the locations of external files.
|
|
|
|
(DER - 2016/02/04, HDFFV-8740)
|
|
|
|
|
|
|
|
Issues Addressed in this Release Since alpha0
|
|
=============================================
|
|
- h5format_convert
|
|
|
|
The h5format_convert tool did not downgrade the version of the
|
|
superblock.
|
|
|
|
Fixed. The tool now will downgrade the version of the superblock.
|
|
|
|
(EIP 2016/01/11)
|
|
|
|
- Crashes with multiple threads: invalid pointers
|
|
|
|
It was reported that alpha0 crashed when used with multiple
|
|
threads. The issue exists in the HDF5 Library versions 1.8 and
|
|
1.9. The problem is related to a shared file pointer used in some
|
|
miscellaneous data structures. The thread-safe library exposed
|
|
paths in the library where a file pointer became invalid.
|
|
|
|
The alpha1 release contains the fixes for the specific use case
|
|
as described in HDFFV-9643. We will keep working on identifying
|
|
and fixing other paths in the library with similar problems.
|
|
|
|
(EIP - 2016/01/15, HDFFV-9643)
|
|
|
|
|
|
|
|
Supported Platforms
|
|
===================
|
|
The following platforms are supported and have been tested for this release.
|
|
They are built with the configure process unless specified otherwise.
|
|
|
|
AIX 6.1 xlc/xlc_r 10.1.0.5
|
|
(NASA G-ADA) xlC/xlC_r 10.1.0.5
|
|
xlf90/xlf90_r 12.1.0.6
|
|
|
|
Linux 2.6.32-573.18.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(mayll/platypus) Version 4.4.7 20120313 (Red Hat 4.4.7-16)
|
|
Version 4.9.3, Version 5.2.0
|
|
PGI C, Fortran, C++ for 64-bit target on
|
|
x86-64;
|
|
Version 15.7-0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 15.0.3.187 Build 20150407
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
Linux 2.6.32-504.8.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
|
|
#1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
|
|
(ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11)
|
|
IBM XL C/C++ V13.1
|
|
IBM XL Fortran V15.1
|
|
|
|
Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++)
|
|
#1 SMP x86_64 GNU/Linux compilers:
|
|
(kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4)
|
|
Version 4.9.3, Version 5.2.0
|
|
Intel(R) C (icc), C++ (icpc), Fortran (icc)
|
|
compilers:
|
|
Version 15.0.3.187 Build 20150407
|
|
MPICH 3.1.4 compiled with GCC 4.9.3
|
|
|
|
SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc
|
|
(emu) Sun Fortran 95 8.6 SunOS_sparc
|
|
Sun C++ 5.12 SunOS_sparc
|
|
|
|
Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
Cygwin(CYGWIN_NT-6.1 2.2.1(0.289/5/3) gcc(4.9.3) compiler and gfortran)
|
|
(cmake and autotools)
|
|
|
|
Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2015 w/ Intel Fortran 16 (cmake)
|
|
|
|
Windows 8.1 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
|
|
Windows 8.1 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake)
|
|
Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
|
|
Mac OS X Mt. Lion 10.8.5 Apple clang/clang++ version 5.1 from Xcode 5.1
|
|
64-bit gfortran GNU Fortran (GCC) 4.8.2
|
|
(swallow/kite) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X Mavericks 10.9.5 Apple clang/clang++ version 6.0 from Xcode 6.2.0
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(wren/quail) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.0 from Xcode 7.0.0
|
|
64-bit gfortran GNU Fortran (GCC) 4.9.2
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
Mac OS X El Capitan 10.11.3 Apple clang/clang++ version 7.0.2 from Xcode 7.0.2
|
|
64-bit gfortran GNU Fortran (GCC) 5.2.0
|
|
(osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3
|
|
|
|
|
|
|
|
Tested Configuration Features Summary
|
|
=====================================
|
|
|
|
In the tables below
|
|
y = tested
|
|
n = not tested in this release
|
|
C = Cluster
|
|
W = Workstation
|
|
x = not working in this release
|
|
dna = does not apply
|
|
( ) = footnote appears below second table
|
|
<blank> = testing incomplete on this feature or platform
|
|
|
|
Platform C F90/ F90 C++ zlib SZIP
|
|
parallel F2003 parallel
|
|
SunOS 5.11 32-bit n y/y n y y y
|
|
SunOS 5.11 64-bit n y/y n y y y
|
|
Windows 7 y y/y n y y y
|
|
Windows 7 x64 y y/y n y y y
|
|
Windows 7 Cygwin n y/y n y y n
|
|
Windows 8.1 n y/y n y y y
|
|
Windows 8.1 x64 n y/y n y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y
|
|
Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y
|
|
Mac OS X Yosemeti 10.10.5 64-bit n y/y n y y y
|
|
AIX 6.1 32- and 64-bit n y/n n y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 GNU y y/y y y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 Intel n y/y n y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 PGI n y/y n y y y
|
|
CentOS 7.1 Linux 3.10.0 x86_64 GNU y y/y y y y y
|
|
CentOS 7.1 Linux 3.10.0 x86_64 Intel n y/y n y y y
|
|
Linux 2.6.32-431.11.2.el6.ppc64 n y/n n y y y
|
|
|
|
Platform Shared Shared Shared Thread-
|
|
C libs F90 libs C++ libs safe
|
|
SunOS 5.11 32-bit y y y y
|
|
SunOS 5.11 64-bit y y y y
|
|
Windows 7 y y y y
|
|
Windows 7 x64 y y y y
|
|
Windows 7 Cygwin n n n y
|
|
Windows 8.1 y y y y
|
|
Windows 8.1 x64 y y y y
|
|
Mac OS X Mountain Lion 10.8.5 64-bit y n y y
|
|
Mac OS X Mavericks 10.9.5 64-bit y n y y
|
|
Mac OS X Yosemeti 10.10.5 64-bit y n y y
|
|
AIX 6.1 32- and 64-bit y n n y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 GNU y y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 Intel y y y y
|
|
CentOS 6.7 Linux 2.6.32 x86_64 PGI y y y y
|
|
CentOS 7.1 Linux 3.10.0 x86_64 GNU y y y y
|
|
CentOS 7.1 Linux 3.10.0 x86_64 Intel y y y y
|
|
Linux 2.6.32-431.11.2.el6.ppc64 y y y y
|
|
|
|
Compiler versions for each platform are listed in the preceding
|
|
"Supported Platforms" table.
|
|
|
|
|
|
|
|
More Tested Platforms
|
|
=====================
|
|
The following platforms are not supported but have been tested for this release.
|
|
|
|
Linux 2.6.18-431.11.2.el6 g95 (GCC 4.0.3 (g95 0.94!)
|
|
#1 SMP x86_64 GNU/Linux
|
|
(platypus)
|
|
|
|
Windows 7 Visual Studio 2008 (cmake)
|
|
|
|
Windows 7 x64 Visual Studio 2008 (cmake)
|
|
|
|
Windows 7 x64 Visual Studio 2010 (cmake) with SWMR using GPFS
|
|
|
|
Windows 10 Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
|
|
Windows 10 x64 Visual Studio 2013 w/ Intel Fortran 15 (cmake)
|
|
|
|
Debian7.5.0 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux
|
|
gcc (Debian 4.7.2-5) 4.7.2
|
|
GNU Fortran (Debian 4.7.2-5) 4.7.2
|
|
(cmake and autotools)
|
|
|
|
Fedora20 3.15.3-200.fc20.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux
|
|
gcc (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1)
|
|
GNU Fortran (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1)
|
|
(cmake and autotools)
|
|
|
|
SUSE 13.1 3.11.10-17-desktop #1 SMP PREEMPT x86_64 x86_64 x86_64 GNU/Linux
|
|
gcc (SUSE Linux) 4.8.1
|
|
GNU Fortran (SUSE Linux) 4.8.1
|
|
(cmake and autotools)
|
|
|
|
Ubuntu 14.04 3.13.0-35-generic #62-Ubuntu SMP x86_64 GNU/Linux
|
|
gcc (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1
|
|
GNU Fortran (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1
|
|
(cmake and autotools)
|
|
|
|
hopper.nersc.gov PrgEnv-gnu/5.2.40
|
|
gcc (GCC) 4.9.2 20141030 (Cray Inc.)
|
|
GNU Fortran (GCC) 4.9.2 20141030 (Cray Inc.)
|
|
g++ (GCC) 4.9.2 20141030 (Cray Inc.)
|
|
|
|
|
|
|
|
Known Problems and Limitations
|
|
==============================
|
|
This section contains the list of known problems and limitations introduced
|
|
in this release of HDF5.
|
|
|
|
Note: this list is not exhaustive of all known issues discovered in HDF5
|
|
software to date. For a list of significant problems and known workarounds
|
|
identified in past releases, please refer to:
|
|
|
|
https://www.hdfgroup.org/HDF5/release/known_problems/
|
|
|
|
The HDF Group also maintains a JIRA issue-tracking database which is used to
|
|
capture all known issues which are too numerous to reasonably list in this
|
|
document. The HDF Group is taking steps to make our JIRA issue database
|
|
open to the public, and this section will refer to that database in a future
|
|
release. In the meantime, please contact help@hdfgroup.org if you come across
|
|
an issue not listed here or at the link above, and we will provide any
|
|
information about known workarounds that we have or add it to our list of
|
|
known issues if it is a new issue.
|
|
|
|
- The JUnit-interface test may fail on Solaris platforms. The result of
|
|
a test for verifying the content of the error stack to stdout is
|
|
in a different order on Solaris then other platforms. Use make -i option
|
|
to test beyond the java/test folder.
|
|
(ADB - 2016/03/22, HDFFV-9734)
|
|
|
|
- The flush/refresh test occasionally fails on OS X platforms. This is
|
|
being investigated but no fix or workaround is available at this time.
|
|
(DER - 2016/03/22, HDFFV-9731)
|
|
|
|
- The VDS/SWMR test will fail with a segmentation fault if the library
|
|
is built with --enable-using-memchecker. The is due to a VDS shutdown
|
|
procedure freeing a shared resource too early when the memory
|
|
checker changes are built. This problem does not arise when the
|
|
memory checker changes are not used since the internal library free
|
|
lists behave differently. The memory checker configure option should
|
|
normally only be used under special circumstances so this should not
|
|
affect most users. Users should be aware that the --enable-using-memchecker
|
|
+ VDS combination may cause a segfault, however, so Valgrind et al. may
|
|
have to be used with an HDF5 library built without the feature if this
|
|
proves to be a problem.
|
|
(DER - 2016/03/21, HDFFV-9732)
|
|
|
|
- SWMR feature limitations
|
|
The SWMR feature will only work if an HDF5 file under SWMR access resides
|
|
on a file system that obeys POSIX write() ordering semantics. Because of
|
|
this, SWMR will not work on network file systems such as NFS or SMB/Windows
|
|
file shares since those systems do not guarantee write odering. SWMR
|
|
regression tests are likely to fail if run on a network file system. SWMR
|
|
is currently not tested on Windows though it can be tested manually
|
|
(some of the SWMR test programs are built by CMake), and there are no
|
|
obvious reasons for it to not work on NTFS or GPFS.
|
|
(EIP - 2016/03/20, HDFFV-9733)
|
|
|
|
- VDS feature limitation
|
|
Currently, the path to a VDS source file is interpreted as relative to the
|
|
directory where the executable program runs and not to the HDF5 file with
|
|
the VDS dataset unless a full path to the source file is specified during
|
|
the mapping.
|
|
(EIP - 2016/03/20, HDFFV-9724)
|
|
|
|
- When building HDF5 with Java using CMake and specifying Debug for CMAKE_BUILD_TYPE,
|
|
there is a missing command argument for the tests of the examples.
|
|
|
|
This error can be avoided by not building Java with Debug, HDF5_BUILD_JAVA:BOOL=OFF,
|
|
or not building Examples, HDF5_BUILD_EXAMPLES:BOOL=OFF.
|
|
(LRK - 2016/03/30, HDFFV-9743)
|
|
|
|
- The H5Lexists API changed behavior in HDF5-1.10 when used with a file handle
|
|
and root group name ("/"):
|
|
|
|
H5Lexists(fileid, "/")
|
|
|
|
In HDF5-1.8 it returns false (0) and in HDF5-1.10 it returns true (1).
|
|
The documentation will be updated with information regarding this change.
|
|
(LRK - 2016/03/30, HDFFV-8746)
|
|
|