Feature
Description:
Test program for the Flexible PHDF5 feature.
Platforms tested:
Tested on modi4 before but the feature is currenly disabled in
configure. Checking this in for future work.
Misc. update:
Bug fixes/API changes
Description:
Previously, the Communicator and Info object arguments supplied
to H5Pset_fapl_mpio() are stored in the property with its handle
values. This meant changes to the communicator or the Info object
after calling H5Pset_fapl_mpio would affect the how the property
list function. This was also the case when H5Fopen/create operated.
They just stored the handle value. This is not according to the
MPI-2 defined behavior of how Info objects should be handled. (MPI-2
defines Info objects must be parsed when called.)
The old design was trying to avoid numerous duplicates of the same
information (e.g., every property object holds one version, every
file opened holds another version, when all of them are referring to
the same original version.) Nevertheless it is safer to implement
it according to MPI-2 definition.
Futhermore, the library often needs to do message passing using the
supplied communicator. Using the same communicator as the application
version may result in some messages mix up.
Solution:
H5Pset_fapl_mpio now stores a duplicate of each of the communicator
and Info object.
H5Pget_fapl_mpio returns a duplicate of its stored communicator and
Info object. It is now the responsibility of the applications to free
those objects when done.
H5Fopen/create also stores a duplicate of the communicator and Info
object supplied by the File Access Property list.
H5Fclose frees those duplicates.
There are a few more internal VFL call back functions that they
follow this "make duplicates" requirement.
Platforms tested:
"h5committested".
What other platforms/configurations were tested?
Eirene (mpicc), sol(mpicc), copper(parallel)
Misc. update:
Need to update MANIFEST for the added t_ph5basic.c which tests the
correctness of duplicated communicator and INFO object.
Update
Description:
Updated (and in some cases added) the copyright statement.
Platforms tested:
Linux (Comment changes...only tested if they compile)
Misc. update:
Purpose:
More test.
Description:
Test independent read of groups and chunked dataset.
Solution:
This test is similar to multiple group test. So just add it in the
testphdf5.c,h.
Platforms tested:
modi4, eirene.
Bug fix (no id, just noticed myself)
Description:
Added a barrier before doing file cleanup to ensure all tests are
completed by then.
Moved the MPI_Finalize to be very end and insert an H5close before
it to ensure all potential MPI activities by HDF5 are done by then.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yes.
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
bug fix
Description:
Added a barrier to ensure all processes have finished using
the file before cleaning it away.
Added H5close() to ensure all HDF5 stuff are closed before
calling MPI_Finalize.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yes
New feature
Description:
Add support for scalar dataspaces in parallel I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Also, FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Update release_docs/RELEASE for bug fixes, new features, etc.
Testing fix.
Description:
Correctly detect when one of the parallel tests fails and stop the make
from proceeding.
Platforms tested:
modi4 (parallel), sleipnir (parallel)
API name change
Description:
Change all "space time" references to "alloc time", including API functions
and macro definitions, etc.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
Bug fix
Description:
script did not work for SP system.
Solution:
added "eval" command to launch the mpi execution. This works fine
for SP and is basically a no-op for non-batch systems like workstations.
Also got rid of the wrongle implemented TESTING message since the
result was not printed and the tests output mess up the format already.
Platforms tested:
SP (snow). Did not test more since it has been tested out fine
in the v1.4 checkin.
Additional test
Description:
Add in a fill-value to one of the tests, to make certain that they are
handled correctly.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel
Code cleanup
Description:
Cleaned up some compiler warnings.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel. Will be testing on IRIX64
6.5 (modi4) in serial & parallel shortly.
Purpose:
Design for compact dataset
Description:
Compact dataset is stored in the header message for dataset layout.
Platforms tested:
arabica, eirene.
Bug fix
Description:
t_mpi test program was not being run with the RUNPARALLEL command
Solution:
Put the t_mpi program into the TEST_PROGS_PARA macro instead of the
TEST_PROGS macro
Hopefully this'll fix the daily test problems on arabica, but I can't test
it because I can't get mpirun to work correctly for me.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
New feature
Description:
Changed parallel I/O tests to use test script instead of just running
testphdf5 executable. This allows the MPI-posix driver to be tested easily.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/parallel and IRIX64 6.5 (modi4) w/parallel
Code cleanup
Description:
Changed ifdef name from "VERBOSE" to "BARRIER_CHECKS", to better describe
what it affects.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
Tidy up
Description:
Old version showed tons of output even if MPI_Offset is too small
to support multiple GB sized files and destined to fail.
Output is pretty confusing.
Solution:
Prints the signness and size of MPI_Offset for information.
Skipped tests if MPI_Offset is not big enough to support the file
sizes.
Platforms tested:
modi4, eirene, burrwhite (all parallel).
Bug Fix
Description:
Currently, only process 0 is writing attribute data to a file. This is
incorrect, because the raw data for attributes is cached in memory until
the object header is written and other processes are not able to read the
correct attribute information.
Solution:
Have all processes participate in writing the attribute data.
Platforms tested:
IRIX64 6.5 (modi4)
Code cleanup
Description:
Platform dependent code related to the struct stat and fstat
calls polluted source codes. Hard to maintain.
Solution:
Platform dependent code are moved to H5private.h and then internal
code can #include H5private.h. Repeat those macro definition for
the stdio and multi drivers since they area examples for writing
a virtual file driver. They must not use any internal code.
Platforms tested:
eirene (parallel), modi4 (serial including gass driver.)
Users were alarmed by the OFFSET overflow and GB file size tests.
Those tests only checks the limits of the MPI implementation, not
really as an error.
Solution:
Changed the VRFY macro to indicate it is an "ERROR".
Modified the INFO macro to print messages as "REMARK (not an error)"
so that users would not be alarmed.
Added an explanation string in the GB file size write/read.
Platforms tested:
eirene and modi4 (parallel)
New test feature
Description:
Added create_faccess_plist() that create just MPIO or split+MPIO
file-access property list. This in turn can run parallel tests
with just MPIO or with Split-file VFD too.
Added -s option for split-file Plus MPIO tests.
For testphdf5.c: removed a bunch of old debug code that got left
in by mistake.
Platforms tested:
Modi4 and eirene parallel.
But it has uncovered errors in the library. The test program
is correct though. Checking the test program in so that it won't
get lost and can be used for debugging. Also, the -s is not used
by default during test. At least it won't abort "make check".
Code cleanup
Description:
This was "thrown" together in a quick way to test MPIO functionality.
Cleaned out some embrassingly useless declaration to reduce compiler
warnings.
Platforms tested:
modi4-pp and eirene-pp.
More code cleanups
Description:
Wrap up the code cleanups for changing the dataset transfer property lists
over to using the generic property list code.
Platforms tested:
IRIX64 6.5 (modi4)
Purpose:
Added attribute test
Description:
attribute test is added into t_mdset.c. At this moment, testing failed
on SunOS 5.7 and Linux. But I believe the problem is from dataset
collective writing in t_dset.c.
Platforms tested:
SGI MPI, MPICH(IRIX64 N32, IRIX64 64, IRIX, Linux, SunOS).
Multiple-group testing
Description:
Added multiple groups under root group and multiple subgroups of
certain levels. Also writes several datasets in each group in
parallel.
Solution:
Platforms tested:
MPICH(IRIX 6.5, IRIX64 6.5(64), IRIX64 6.5(N32), Linux, SunOS 5.6)
and SGI MPI(IRIX64 6.5).
Purpose:
Multiple-group testing
Description:
Added multiple groups under root group and multiple subgroups of
certain levels. Also writes several datasets in each group in
parallel.
Solution:
Platforms tested:
MPICH(IRIX 6.5, IRIX64 6.5(64), IRIX64 6.5(N32), Linux, SunOS 5.6)
and SGI MPI(IRIX64 6.5).
Bug fix
Description:
The t_mpi used to fail and exit if any error detected.
That aborted other process in a "make check" situation.
Solution:
Introduced a new error verification as INFO. INFO is for
information only. It does not increase nerrors count.
The program always exits with 0.
Platforms tested:
eirene with mpich.
Bug fix
Description:
Some mpi.h already includes MPI-IO definitions in it (e.g., SGI,
SP2). Made the #include of mpio.h be dependent on whether some
MPI-IO constant is already defined or not.
Platforms tested:
modi4 (IRIX64) and IBM SP2.
Changed MPI_File_seek then MPI_File_write or MPI_File_read to just
MPI_File_write_at and MPI_File_read_at. Some compiler, e.g., IBM
mpcc_r does not support MPI_File_seek and MPI_File_read or MPI_File_write.
This is a better measurement against HDF5 performance since HDF5
uses MPI_File_write_at and MPI_File_read_at all the times.
It is a more thread safe to use MPI_xxx_at than the other seek then
read/write approaches.
Platforms tested:
modi4 (irix64 parallel), IBM sp2.
Bug Fix
Description:
FILENAME_MAX is short on some systems. It's better to use PATH_MAX
which tends to be longer (we hope).
Solution:
Removed FILENAME_MAX and replaced it with PATH_MAX instead.
Platforms tested:
Skinner (SDSC HP N9000).
Bug fix
Description:
added a barrier to prevent racing condition before remove file and
open file.
Platforms tested:
modi4,pp
cVS: ----------------------------------------------------------------------
Feature
Description:
Show simple performance of the MPIO and the HDF5-IO.
The mpi-perf.c is contributed by Robert Ross of ANL.
The perf.c is derived from mpi-perf.c
Platforms tested:
Modi4 (O2K, parallel)
