mirror of
https://github.com/HDFGroup/hdf5.git
synced 2025-02-23 16:20:57 +08:00
40df66ebd0
Large code cleanup/re-write
Description:
This is phase 1 of the data I/O re-architecture, with the following changes:
- Changed the selection drivers to not actually do any I/O, they
only generate the sequences of offset/length pairs needed for
the I/O (or memory access, in the case of iterating or filling
a selection in a memory buffer)
- Wrote more abstract I/O routines which get the sequence of offset/
length pairs for each selection and access perform the I/O or
memory access.
Benefits of this change include:
- Removed ~3400 lines of quite redundant code, with corresponding
reduction in the size of library binary.
- Any selection can now directly access memory when performing I/O,
if no type conversions are required, instead of just "regular"
hyperslab and 'all' selections, which speeds up I/O.
- Sped up I/O for hyperslab selections which have contiguous lower
dimensions by "flattening" them out into lesser dimensional objects
for the I/O.
No file format or API changes were necessary for this change.
The next phase will be to create a "selection driver" for each type of
selection, allowing each type of selection to directly call certain
methods that only apply to that type of selection, instead of passing
through dozens of functions which have switch statements to call the
appropriate method for each selection type. This will also reduce
the amount of code in the library and speed things up a bit more.
Phase 3 will involve generating an MPI datatype for all types of selections,
instead of only "regular" hyperslab and 'all' selections. This will
allow collective parallel I/O for all I/O operations which don't
require type conversions. It will also open up the door for allowing
collective I/O on datasets which require type conversion.
Phase 4 will involve changing the access pattern to deal with chunked
datasets in a more optimal way (in serial).
Phase 5 will deal with accessing chunked datasets more optimally for
collective parallel I/O operations.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/ parallel & C++ and IRIX64 6.5 (modi4) w/parallel
|
||
|---|---|---|
| bin | ||
| c++ | ||
| config | ||
| doc | ||
| examples | ||
| fortran | ||
| hl/tools/gif2h5 | ||
| pablo | ||
| perform | ||
| release_docs | ||
| src | ||
| test | ||
| testpar | ||
| tools | ||
| windows | ||
| aclocal.m4 | ||
| configure | ||
| configure.in | ||
| COPYING | ||
| Makefile.dist | ||
| Makefile.in | ||
| MANIFEST | ||
| README.txt | ||
HDF5 version 1.5.32 currently under development
Please refer to the release_docs/INSTALL file for installation instructions.
------------------------------------------------------------------------------
This release is almost fully functional for the entire API defined in the
documentation, see the RELEASE.txt file in this directory for information
specific to this release of the library. The INSTALL file contains
instructions on compiling and installing the library. The INSTALL_parallel
file contains instructions on installing the parallel version of the
library. The INSTALL* files can be found in the release_docs/ directory.
Documentation for this release is in the html directory. Start with the
"index.html" in that directory.
Four mailing lists are currently set up for use with the HDF5
library.
hdf5 - For general discussion of the HDF5 library with
other users.
hdf5dev - For discussion of the HDF5 library development
with developers and other interested parties.
hdf5announce - For announcements of HDF5 related developments,
not a discussion list.
hdf5cvs - For checkin notices of code development on the library,
not a discussion list.
To subscribe to a list, send mail to "majordomo@ncsa.uiuc.edu",
with "subscribe <list>" in the _body_, not the Subject, of the message.
E.g., subscribe hdf5
Messages to be sent to the list should be sent to "<list>@ncsa.uiuc.edu".
Nearly daily code snapshots are now being provided at the following URL:
ftp://hdf.ncsa.uiuc.edu/pub/outgoing/hdf5/snapshots
Please read the readme file in that directory before working with a library
snapshot.
The HDF5 website is located at http://hdf.ncsa.uiuc.edu/HDF5/
Bugs should be reported to hdfhelp@ncsa.uiuc.edu.