mirror/hdf5
2
0
Fork 0
mirror of https://github.com/HDFGroup/hdf5.git synced 2024-11-27 02:10:55 +08:00
Code Issues Packages Projects Releases Wiki Activity
6ea5252281
hdf5/release_docs/RELEASE.txt
Quincey Koziol 6ea5252281 [svn-r7018] Purpose: 
Bug fix

Description:
    The dataset's modification time was getting set whenever raw data was
written with H5Dwrite.  Unfortunately, this is a metadata change (which are
required to be performed collectively) and H5Dwrite may be called independently
from a parallel program, resulting in metadata cache corruption and/or program
hangs.

Solution:
    Don't update the modification time when raw data it written. :-(

Platforms tested:
    FreeBSD 4.8 (sleipnir) w/parallel
    h5committest

Misc. update:
    Noted in release notes and also sent to Frank for updating the docs.
2003-06-10 13:47:26 -05:00

518 lines
25 KiB
Plaintext
Raw Blame History

HDF5 version 1.5.57 currently under development
================================================================================
INTRODUCTION
This document describes the differences between HDF5-1.4.* and
HDF5-1.5.*, and contains information on the platforms tested and
known problems in HDF5-1.5.*. For more details check the HISTORY.txt
file in the HDF5 source.
The HDF5 documentation can be found on the NCSA ftp server
(ftp.ncsa.uiuc.edu) in the directory:
/HDF/HDF5/docs/
For more information look at the HDF5 home page at:
http://hdf.ncsa.uiuc.edu/HDF5/
If you have any questions or comments, please send them to:
hdfhelp@ncsa.uiuc.edu
CONTENTS
- New Features
- Support for new platforms and languages
- Bug Fixes since HDF5-1.4.0
- Platforms Tested
- Known Problems
New Features
============
Configuration:
--------------
The following flags have been added to the configuration script:
--enable-hdf5v1_4 Compile the HDF5 v1.4 compatibility interface
--enable-mpe Enable MPE instrumentation [default=no]
--enable-filters=all Turn on all internal I/O filters. One may also
specify a comma-separated list of filters or the
word no. The default is all internal I/O filters.
--with-szlib=DIR Use szlib library for external szlib I/O filter
[default=no]
Library:
--------
Summary: This release has the following new features that are not
available in 1.4.* releases
1. Generic properties to give application more control on I/O pipeline
2. Time allocation and fill value properties
3. New filters: external compression filter szip
internal shuffling and checksum filters
4. Compact storage layout for datasets
5. Redesigned I/O pipeline for better performance.
Complete list of changes:
* Changed dataset modification time to _not_ be updated when raw data is
written to a dataset. The modification time is only updated when the
dataset's metadata is modified. QAK - 2003/06/10
* Changed H5Sget_select_bounds to use 'hssize_t *' for start and end
parameters, instead of 'hsize_t *', to better match other parts of the
dataspace API. QAK - 2003/06/04
* Changed raw data I/O to only access each chunk once, improving performance
in many situations with hyperslabs and large chunks or chunks with
filters. These improvements are currently limited to serial I/O, with
similar parallel I/O improvements forthcoming sometime in the future.
QAK - 2003/05/07
* Added parameter to the MPI/POSIX driver. If GPFS is enabled (by
modifying the H5FDmpiposix.c file to uncomment the USE_GPFS_HINTS
macro), then this extra parameter will turn GPFS hints on and off
during runtime. BW - 2003/05/05
* Added option to print 1-byte integer datasets as ASCII to h5dump. BW -
2003/04/30
* Added a new utility "h5fc". It can be used to compile easily Fortran
programs which use HDF5. It automatically uses the compiler the HDF5
library was built with and links in any libraries HDF5 requires.
BW - 2003/04/10
* Added new dataset creation property list functions for working with
I/O filters: H5Pmodify_filter, H5Pget_filter_by_id and
H5Pall_filters_avail. Also changed H5Zregister to use new method
of registering filters with library. QAK - 2003/04/08
* The first version of szip compression support were implemented.
User should have static szlib library installed. Using function
H5Pset_szip to pass the szip parameters to the HDF5 library.
More detailed decription of the process will be followed.
KY-2003/04/01
* Added Fletcher32 checksum as a filter in pipeline. It only works in
chunked dataset. SLU - 2003/2/11
* MPICH/MPE instrumentation feature added. Use --enable-mpe to configure
it. AKC - 2003/1/3
* New functions H5Gget_num_objs, H5Gget_objname_by_idx and H5Gget_objtype_by_idx
are added to the library. SLU - 2002/11/25
* H5Dget_offset is added to return the offset of a dataset's data relative
to the beginning of the file. SLU - 2002/11/7
* Functions H5Tget_native_type and H5Tis_variable_str are added. The first
one reconstructs a datatype based on native memory datatype. The second
one checks if a datatype is variable string. SLU - 2002/11/6
* Added environment variable "HDF5_DISABLE_VERSION_CHECK", which disables
the version checking between the header files and the library linked into
an application if set to '1'. This should be used with caution, mis-
matched headers and library binaries can cause _serious_ problems.
QAK - 2002/10/15
* Added new API function to get the name of an object in a file, using
an open ID (hid_t). QAK - 2002/10/14
* Added API functions to return pointer to low-level file handle
(H5Fget_vfd_handle and H5FDget_vfd_handle) and related property list
setting functions(H5Pset_family_offset and H5Pset_multi_type).
SLU - 2002/09/30
* Changed "H5P[set|get]_space_time" functions to "H5P[set|get]_alloc_time"
Unify all symbolic names for these functions to use "alloc time" instead
of other names. QAK - 2002/09/13
* Added "H5D_SPACE_ALLOC_INCR" setting to H5D_SPACE_ALLOC_EARLY and
H5D_SPACE_ALLOC_LATE for H5Dset_space_time(). This allows chunked
datasets to be incrementally allocated as in the 1.4.x branch.
QAK - 2002/08/27
* Compact dataset is added to the library. The data will be stored in
the header message of dataset layout. Space allocation time has to be
EARLY. No hyperslab is supported for parallel collective write. There
is no API changes except activating H5Pset_layout and H5Pget_layout for
compact dataset. -SLU, 2002/8/20
* Added 'closing' parameter to VFL 'flush' callback function and H5FDflush.
This allows the library to indicate that the file will be closed
immediately following the call to 'flush' and can be used to avoid actions
that are duplicated in the VFL 'close' callback function. QAK - 2002/05/20
* Added feature to parallel chunk allocation routine to not write fill
values to chunks allocated if the user has set the "fill time" to never.
This can improve parallel I/O performance for chunked
datasets. QAK - 2002/05/17
* New functions H5Glink2 and H5Gmove2 were added to allow link and move to
be in different locations in the same file. The old functions H5Glink
and H5Gmove remain valid. SLU - 2002/04/26
* Fill-value's behaviors for contiguous dataset have been redefined.
Basicly, dataset won't allocate space until it's necessary. Full details
are available at http://hdf.ncsa.uiuc.edu/RFC/Fill_Value, at this moment.
SLU - 2002/04/11
* Added new routine "H5Dfill" to fill a selection with a particular value
in memory. QAK - 2002/04/09
* Improved performance of "regular" hyperslab I/O when using MPI-IO and the
datatype conversion is unneccessary. QAK - 2002/04/02
* Improved performance of single hyperslab I/O when datatype conversion is
unneccessary. QAK - 2002/04/02
* Added new "H5Sget_select_type" API function to determine which type of
selection is defined for a dataspace ("all", "none", "hyperslab" or
"point"). QAK - 2002/02/07
* Added support to read/write portions of chunks directly, if they are
uncompressed and too large to cache. This should speed up I/O on chunked
datasets for a few more cases. QAK - 2002/01/31
* Added H5Rget_obj_type() API function, which performs the same functionality
as H5Rget_object_type(), but requires the reference type as a parameter
in order to correctly handle dataset region references. Moved
H5Rget_object_type() to be only compiled into the library when v1.4
compatibility is enabled.
* Added a new file access property, file close degree, to control file
close behavior. It has four values, H5F_CLOSE_WEAK, H5F_CLOSE_SEMI,
H5F_CLOSE_STRONG, and H5F_CLOSE_DEFAULT. Two correspont functions
H5Pset_fclose_degree and H5Pget_fclose_degree are also provided. Two
new functions H5Fget_obj_count and H5Fget_obj_ids are offerted to assist
this new feature. For full details, please refer to the reference
manual under the description of H5Fcreate, H5Fopen, H5Fclose and the
functions mentioned above.
* Removed H5P(get|set)_hyper_cache API function, since the property is no
longer used.
* Improved performance of non-contiguous hyperslabs (built up with
several hyperslab selection calls).
* Improved performance of single, contiguous hyperslabs when reading or
writing.
* As part of the transition to using generic properties everywhere, the
parameter of H5Pcreate changed from H5P_class_t to hid_t, as well
the return type of H5Pget_class changed from H5P_class_t to hid_t.
Further changes are still necessary and will be documented here as they
are made.
* Improved regular hyperslab I/O by about a factor of 6 or so.
* Modified the Pablo build procedure to permit building of the instrumented
library to link either with the Trace libraries as before or with the
Pablo Performance Caputure Facility.
* Added new F90 APIs for generic properties, new filters, and
time/space allocation properties.
* C++ API:
- Added two new member functions: Exception::getFuncName() and
Exception::getCFuncName() to provide the name of the member
function, where an exception is thrown.
- IdComponent::operator= becomes a virtual function because
DataType, DataSpace, and PropList provide their own
implementation. The new operator= functions invoke H5Tcopy,
H5Scopy, and H5Pcopy to make a copy of a datatype, dataspace,
and property list, respectively.
Parallel Library:
-----------------
Tools:
------
* When the "-S" option for "simple" output is chosen, h5ls now displays
modification times of datasets in UTC instead of local time.
QAK - 2003/06/06
* h5diff to compare two HDF5 files was added
* h5import to import ascii and binary data to an HDF5 file was added.
Old h5import tool in the tools/misc directory was renamed to
h5createU8 to reflect its purpose. h5createU8 will be deleted in
1.6.1 release.
Support for new platforms and languages
=======================================
* Added C++ API support on HPUX11.00. BMR - 2003/03/19
Bug Fixes since HDF5-1.4.0 release
==================================
Library
-------
* Don't attempt to perform collective I/O on chunked datasets with
parallel I/O. QAK - 2003/06/05
* The library now correctly reuses space when objects are deleted in the
file. This should be handled correctly for every situation, except
datasets with variable-length datatypes are not returning the space they
use in the global heap currently. QAK - 2003/04/13
* Fixed error in B-tree deletion routine which could cause groups to be
corrupted when objects are removed from them.
QAK - 2003/04/11
* Fixed error in file space freeing code which could cause metadata to
fail to be written to the file.
QAK - 2003/04/11
* -O caused errors in AIX 5.x platforms. Removed it from
--enable-production mode. AKC - 2003/03/31
* Corrected memory/resource leaks in per-thread key information when
thread-safe operation was enabled. QAK - 2003/02/07
* Improved error assertion for nil VL strings, making it fails with error
stack instead of just assertion failure. SLU - 2002/12/16
* Added two new API functions: H5Zunregister & H5Zfilter_avail.
QAK - 2002/11/16
* Add data shuffle filter(source code H5Zshuffle.c), the combination of the
shuffling and compression can make data compression better without suffering
much encoding and decoding CPU time for many application datasets(especially
for floating point data). This adds a new API function: H5Pset_shuffle.
KY - 2002/11/13
* Allow scalar dataspaces to be used for parallel I/O. QAK - 2002/11/05
* New functions H5Gget_comment(modification), H5Aget_storage_size,
H5Arename. SLU - 2002/10/29
* Fixed an assertion of H5S_select_iterate that did not account for scalar
type that has no dimension sizes. AKC - 2002/10/15
* Partially fixed space allocation inefficiencies in the file by
improving our algorithms for re-using freed space. QAK - 2002/08/27
* Fixed data corruption problem which could occur when fill values were
written to a contiguously stored dataset in parallel. QAK - 2002/08/27
* Fixed VL memory leak when data is overwritten. The heap objects holding
old data are freed. If the fill value writting time is set to
H5D_FILL_TIME_NEVER, the library prohibits user to create VL type dataset.
The library free all the heap objects storing VL type if there is nested
VL type(a VL type contains another VL type). SLU - 2002/07/10
* Tweaked a few API functions to use 'size_t' instead of 'unsigned' or
'hsize_t', which may cause errors in some cases.
Configuration
-------------
* Included the both the examples of fortran and c++ "make check-install"
testing. This tests the correctness of the h5fc command. AKC - 2003/04/22
* When using gcc 3.x, we use -std=c99 instead of -ansi for compiling.
QAK - 2003/04/11
* IA64 platform has its own configure setting and use Intel Compilers as
the default compilers (were gcc and pgf90 before.) This also eliminated
the segmentation fault in the fortran test. The missing reference of
"exit" is fixed too. AKC - 2003/04/02
Performance
-------------
* Improved dataset creation time by about 30% (relative to the 1.4.x
branch).
Tools
-----
* Added a -force option to h5redeploy. AKC - 2003/03/04
* The VL string bug(data and datatype cannot be shown) in h5dump is fixed.
-SLU - 2002/11/18
* Fixed segfault if h5dump was invoked with some options but no file
(e.g., h5dump -H). -AKC, 2002/10/15
* Fixed so that the "-i" flag works correctly with the h5dumper.
* Fixed segfault when "-v" flag was used with the h5dumper.
Documentation
-------------
Platforms Tested
================
AIX 5.1 (32 and 64-bit) C for AIX Compiler, Version 6.0.0.2
xlf 8.1.0.3
poe 3.2.0.11
Cray T3E sn6606 2.0.6.08 Cray Standard C Version 6.6.0.2
Cray Fortran Version 3.6.0.0.2
Cray SV1 sn9617 10.0.1.0 Cray Standard C Version 6.6.0.2
Cray Fortran Version 3.6.0.0.2
Cray T90IEEE 10.0.1.01y Cray Standard C Version 6.4.0.2.3
Cray Fortran Version 3.4.0.3
FreeBSD 4.7 gcc 2.95.4
g++ 2.95.5
HP-UX B.11.00 HP C HP92453-01 A.11.01.20
HP F90 v2.4
HP ANSI C++ B3910B A.03.13
MPIch 1.2.4
IRIX 6.5 MIPSpro cc 7.30
IRIX64 6.5 (64 & n32) MIPSpro cc 7.3.1.3m
F90 MIPSpro 7.3.1.3m (64 only)
MPIch 1.2.4
Linux 2.4.18 gcc 3.2.2
g++ 3.22.
Intel(R) C++ Version 7.1
Intel(R) Fortran Compiler Version 7.1
PGI compilers (pgcc, pgf90, pgCC) version 4.0-2
MPIch 1.2.4
OSF1 V5.1 Compaq C V6.4-014
Compaq Fortran X5.4A-1684
Compaq C V6.3-027 , MPI_64bit_R5
gcc version 3.0 for C++
SunOS 5.7 WorkShop Compilers 5.0 98/12/15 C 5.0
(Solaris 2.7) WorkShop Compilers 5.0 98/12/15 C++ 5.0
WorkShop Compilers 5.0 98/10/25
FORTRAN 90 2.0 Patch 107356-04
SunOS 5.8/32 Sun WorkShop 6 update 2 C 5.3
(Solaris 2.8) Sun WorkShop 6 update 2 Fortran 90
Sun WorkShop 6 update 2 C++ 5.3
SunOS 5.8/64 Sun WorkShop 6 update 2 C 5.3
(Solaris 2.8) Sun WorkShop 6 update 2 Fortran 90
Sun WorkShop 6 update 2 C++ 5.3
TFLOPS r1.0.4 v4.3.3 i386 pgcc Rel 3.1-4i with mpich-1.2.4 with
local modifications
IA-32 Linux 2.4.9 gcc 2.96
Intel(R) C++ Version 7.0
Intel(R) Fortran Compiler Version 7.0
IA-64 Linux 2.4.16 ia64 gcc version 2.96 20000731
Intel(R) C++ Version 7.0
Intel(R) Fortran Compiler Version 7.0
Windows 2000 (NT5.0) MSVC++ 6.0
DEC Visual Fortran 6.0
Intel C and F90 compilers version 7.1
Windows XP .NET
Windows NT4.0 Code Warrior 6.0
MAC OS X Darwin 6.5
gcc and g++ Apple Computer, Inc. GCC
version 1161, based on gcc version 3.1
Supported Configuration Features Summary
========================================
In the tables below
y = tested and supported
n = not supported or not tested in this release
x = not working in this release
dna = does not apply
( ) = footnote appears below second table
Platform C C F90 F90 C++ Shared zlib
parallel parallel libraries (5)
Solaris2.6 y n y n y y y
Solaris2.7 64-bit y y (1) y y (1) y y y
Solaris2.7 32-bit y y (1) y y (1) y y y
Solaris2.8 64-bit y y y y (1) y y y
Solaris2.8 32-bit y y y y (1) y y y
IRIX6.5 y y (1) n n n y y
IRIX64_6.5 64-bit y y (2) y y y y y
IRIX64_6.5 32-bit y y (2) n n n y y
HPUX11.00 y y (1) y n y y y
OSF1 v5.1 y n y n y y y
T3E (6) y n y n n n y
SV1 y n y n n n y
T90 IEEE y n y n n n y
TFLOPS n y (1) n n n n y
AIX-5.1 32-bit y y y y y n y
AIX-5.1 64-bit y y y y y n y
WinXP (7) y n n n y y y
WinNT/2000 y n y n y y y
WinNT CW y n n n n n y
Mac OS X 10.2 y n n n y y y
FreeBSD y y (1) n n y y y
Linux 2.2 y y (1) y y (1) y y y
Linux 2.4 gcc (3) y y (1) y n y y y
Linux 2.4 Intel (3) y n y n n n y
Linux 2.4 PGI (3) y n y n y n y
Linux 2.4 IA32 y n y n n n y
Linux 2.4 IA64 y n y n n n y
Platform static- Thread- SRB GASS STREAM-
exec safe VFD
Solaris2.6 x y n n y
Solaris2.7 64-bit x y n n y
Solaris2.7 32-bit x y n n y
Solaris2.8 64-bit x y n n y
Solaris2.8 32-bit x y n n y
IRIX6.5 x n n n y
IRIX64_6.5 64-bit x y n y y
IRIX64_6.5 32-bit x y n y y
HPUX11.00 x n n n y
OSF1 v5.1 y n n n y
T3E (6) y n n n y
SV1 y n n n y
T90 IEEE y n n n y
TFLOPS y n n n n
AIX-5.1 32-bit y n n n y
AIX-5.1 64-bit y n n n y
WinXP (7) dna n n n n
WinNT/2000 dna n n n n
WinNT CW dna n n n n
Mac OS X 10.2 y n n n y
FreeBSD y y n n y
Linux 2.2 y y n n y
Linux 2.4 gcc (3) y y n n y
Linux 2.4 Intel (3) y n n n y
Linux 2.4 PGI (3) y n n n y
Linux 2.4 IA32 y n n n y
Linux 2.4 IA64 y n n n y
Notes: (1) Using mpich 1.2.4.
(2) Using mpt and mpich 1.2.4.
(3) Linux 2.4 with GNU, Intel, and PGI compilers.
(4) No HDF4-related tools.
(5) Shared libraries are provided only for the C library,
except on Windows where they are provided for all languages.
(6) Debug mode only.
(7) Binaries only; source code for this platform is not being
released at this time.
Known Problems
==============
* The h5dump tests may fail to match the expected output on some platforms
(e.g. parallel jobs, Windows) where the error messages directed to
"stderr" do not appear in the "right order" with output from stdout.
This is not an error.
* The stream-vfd test uses ip port 10007 for testing. If another
application is already using that port address, the test will hang
indefinitely and has to be terminated by the kill command. To try the
test again, change the port address in test/stream_test.c to one not
being used in the host.
* The --enable-static-exec configure flag fails to compile for Solaris
platforms. This is due to the fact that not all of the system
libraries on Solaris are available in a static format.
The --enable-static-exec configure flag also fails to correctly compile
on IBM SP2 platform for the serial mode. The parallel mode works fine
with this option.
It is suggested that you don't use this option on these platforms
during configuration.
* With the gcc 2.95.2 compiler, HDF 5 uses the `-ansi' flag during
compilation. The ANSI version of the compiler complains about not being
able to handle the `long long' datatype with the warning:
warning: ANSI C does not support `long long'
This warning is innocuous and can be safely ignored.
* The Stream VFD was not tested yet under Windows. It is not supported
in the TFLOPS machine.
* The ./dsets tests failed in the TFLOPS machine if the test program,
dsets.c, is compiled with the -O option. The hdf5 library still works
correctly with the -O option. The test program works fine if it is
compiled with -O1 or -O0. Only -O (same as -O2) causes the test
program to fail.
* Certain platforms give false negatives when testing h5ls:
- Cray J90 (and Cray T3E?) give errors during testing when displaying
some floating-point values. These are benign differences due to
the different precision in the values displayed and h5ls appears to
be dumping floating-point numbers correctly.
* Before building HDF5 F90 Library from source on Crays (T3E and J90)
replace H5Aff.f90, H5Dff.f90 and H5Pff.f90 files in the fortran/src
subdirectory in the top level directory with the Cray-specific files
from the site:
* On some platforms that use Intel compilers to build HDF5 fortran library,
compilation may fail for fortranlib_test.f90, fflush1.f90 and fflush2.f90
complaining about exit subroutine. Comment out the line
IF (total_error .ne. 0) CALL exit (total_error)
ftp://hdf.ncsa.uiuc.edu/pub/ougoing/hdf5/hdf5-1.6.0/F90_source_for_Crays
* On IA32 and IA64 systems, if you use a compiler other than GCC (such as
Intel's ecc or icc compilers), you will need to modify the generated
"libtool" program after configuration is finished. On or around line 104 of
the libtool file, there are lines which look like:
# How to pass a linker flag through the compiler.
wl=""
change these lines to this:
# How to pass a linker flag through the compiler.
wl="-Wl,"
UPDATE: This is now done automatically by the configure script. However, if
you still experience a problem, you may want to check this line in the
libtool file and make sure that it has the correct value.
Reference in New Issue View Git Blame Copy Permalink