feature
Description:
Change testphdf5 to use the common test program syntax.
Needed to change the protocols of all test programs to
fit the requirement of the common test syntax.
Platforms tested:
"h5committested".
Also tested in sol with PP mode.
Bug fix
Description:
make check without doing make first would fail to build testphdf5.
Solution:
Put the testphdf5 back in the target of TEST_PROG_PP and removed
the "testphdf" without any parameter from testph5.sh.
Platforms tested:
copper parallel. This is not relevant to non-parallel cases.
Misc. update:
Improvement
Description:
Changed parsing of verbose level by the common test library routine.
Change t_mpi.c to use the Verbose control better.
Platforms tested:
verena (pp).
Misc. update:
Improvement.
Description:
Complete change of the verbose control to use the routines provided by
the test/libh5test.a.
Also put in a temporary fix for the H5Eset_auto() and H5Eget_auto()
so that the Compat code are isolated in one place rather than all over
the source file.
Platforms tested:
Tested in Eirene (parallel).
Misc. update:
Added a test of fill value before any data is written to a dataset.
Rename short_dataset() as dataset_fillvalue() as it reflects better
the tests. Also removed the option of -S since the fill value test
will be tested always.
Platforms tested:
"h5committested"
Misc. update:
Code cleanup.
Description:
The H5Eclear() in the VRFY and INFO macros are not needed.
After removing them, there is no need to have a separate
v1.6 Compat version.
Platforms tested:
"h5committested"
Misc. update:
Code cleanup, bug fixes
Description:
Wrap up rest of changes necessary for fixing the "short" MPI-I/O read
problem that Robb reported.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Feature.
Description:
Added the short_dataset test (was in v1.6 first.)
Platforms tested:
Tested in eirene (pp) only since these have been tested in v1.6 already.
Misc. update:
Bug fix
Description:
Clean up a couple more 1.6 compat bugs that showed up when the library
was compiled with parallel support.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel & 1.6 compat
config not tested with h5committest
Update
Description:
This completes Phase I of the FPHDF5 testing. This test does the
following:
- Creates a file
- Creates multiple non-root groups
- For each process:
For each non-root group:
Create multiple datasets
Access all datasets created above
Write to all datasets created above
- Close the file
- Reopen the file and verify the data is correct
Platforms tested:
AIX (w/ FPHDF5)
Linux (w/ FPHDF5)
Solaris (w/ Fortran & C++)
Misc. update:
new feature.
Description:
Added tests of 1wMr with options to apply Atomicity or File_sync.
Platforms tested:
only tested in eirene and Teragrid as this is just an MPI test.
Misc. update:
Update
Description:
Extensive reworking of the code. More modular now. Tests for almost
all of "Phase 1"s requirements. Need to check that data is correct
after reopening the file.
Platforms tested:
Linux (w/ FPHDF5)
AIX (w/ and w/o FPHDF5)
FPHDF5-specific fixes...No need for H5committest
Misc. update:
new test.
Description:
Added test_mpio_1wMr test which verify if the file system can support
1 process writes, many processes read.
Platforms tested:
h5committested.
Misc. update:
Update
Description:
The FPHDF5 test now writes data simultaneously from all processes to
all created datasets. It's wicked cool!
Platforms tested:
Linux (w/ FPHDF5)
Copper (w/ FPHDF5 & w/o Parallel)
FPHDF5-specific, so no h5committest
Misc. update:
Update
Description:
Modified the FPHDF5 test so that a single process creates multiple
datasets in a non-root group then all of the processes (sans the SAP)
access it.
Modularized the code a bit more to make it easy to perform the above
tests.
Changed the code a bit so that >3 processes can (theoretically) be
used to test this.
Platforms tested:
Linux (FPHDF5 specific change)
Misc. update:
Update
Description:
Expanded the FPHDF5 test to include creating multiple datasets by one
process in a non-root group.
Cleaned up the code a bit more to make it more modular and easier to
work with.
Platforms tested:
Linux (FPHDF5 specific)
Misc. update:
Update
Description:
Added test for FPHDF5 to the testpar/ directory. If you specify
--enable-fphdf5 during configuration, it'll run this test during a
"make check".
Platforms tested:
Linux (FPHDF5 specific).
Misc. update:
Feature, sort of.
Description:
Reactivated the big dataset test. Changed it
to default off, can be turned on via -b.
Platforms tested:
Only in Copper which is the only local machine safe to run
mulitple GB size files.
Misc. update:
Bug/feature fix.
Description:
Relax restriction on parallel writing to compact datasets to allow partial
I/O.
Updates to reference manual mentioning the issues involved are delayed until
reference manual 'lock' is removed later this week.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Disable "big" parallel tests until I get a chance to look into the
problems (probably during/after the workshop).
Platforms tested:
Eyeballed
Too minor to require h5committest
Bug fix
Description:
The MPI_File_set_size() routine on ASCI Red is not able to extend files
so that they are larger than 2GB.
Solution:
Add an extra macro which controls whether MPI_File_set_size() can handle
>2GB offsets or if our "older" way of reading a byte, then writing a byte at
the appropriate offset should be used.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
Temporary files were not cleaned out because the definiton of
temporary files had not been updated.
Solution:
Updated it.
Platforms tested:
Tested by hand in Copper while i was testing the MPE change.
Misc. update:
Updated the copyright notice--mostly by rearranging
some text to make them consistent.
Solution:
Platforms tested:
"h5committested"--sol is down, so, no SUn test.
Misc. update:
Compatibility fix
Description:
The H5P[set|get]_fapl_mpiposix calls changed between v1.4.x and v1.5.x.
Solution:
Wrap them in the v1.4 backward compatibility #ifdefs and update tests, etc.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/paralle & v1.4 compatibility
h5committest pointless
Bug fix
Description:
An earlier checkin changed some of the assumptions about single block
hyperslabs, causing them to fail in odd ways.
Solution:
Fix errors with single block hyperslabs by keying off of count==1 instead
of stride==1.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/parallel
h5committested
Add printout to identify when the MPIPOSIX driver is being tested instead
of the MPI-IO driver..
Platforms tested:
Tested in copper in parallel mode only since it was just a simple
print statement in the parallel test code.
Misc. update:
Feature Add
Description:
Added knob so that the programmer can enable or disable GPFS
hints during runtime instead of having it only enabled at
configure/compile time. Some of the public APIs were changed
to add an extra parameter for this option...
Platforms tested:
Blue (LLNL). It only affects the MPI/POSIX driver, so no need
to test it on non-GPFS platforms.
Misc. update:
feature
Description:
added test_fapl_mpiposix_dup to verify the MPIPOSIX is making
duplicates of Communicator correctly.
Platforms tested:
Did not h5committested because these are all MPI code and well protected
by HAVE_PARALLEL.
Tested in eirene and modi4, both parallel mode.
Misc. update:
code cleanup.
Description:
Rename test_comm_info_delete to test_fapl_mpio_dup to better decribe
what it does. Updated the comments to describe more accurate.
Platforms tested:
Did not run h5commmittest since the changes were all in parallel area.
Ran parallel tests on modi4 and eirene.
Misc. update:
simple code cleanup.
Description:
While debug a problem in multiple_group_write(), noticed some returned
values were not checked. Added code to check on all returned code.
Platforms tested:
h5committested.
Misc. update:
bug fix
Description:
The fphdf5.o and similar files are not included in the make clean
target. So, if the same directory is used again for a different
build, the left behind file could cause a problem (e.g., if
the second build is for a different binary format.)
Solution:
Added fphdf5.c to the TEST_SRC list which indirect form the clean
target list.
Platforms tested:
Did not run h5committest tests because they don't verify this
failure. I tested the fix in modi4 pp to verify all files are cleaned.
Misc. update:
Feature
Description:
Test program for the Flexible PHDF5 feature.
Platforms tested:
Tested on modi4 before but the feature is currenly disabled in
configure. Checking this in for future work.
Misc. update:
Bug fixes/API changes
Description:
Previously, the Communicator and Info object arguments supplied
to H5Pset_fapl_mpio() are stored in the property with its handle
values. This meant changes to the communicator or the Info object
after calling H5Pset_fapl_mpio would affect the how the property
list function. This was also the case when H5Fopen/create operated.
They just stored the handle value. This is not according to the
MPI-2 defined behavior of how Info objects should be handled. (MPI-2
defines Info objects must be parsed when called.)
The old design was trying to avoid numerous duplicates of the same
information (e.g., every property object holds one version, every
file opened holds another version, when all of them are referring to
the same original version.) Nevertheless it is safer to implement
it according to MPI-2 definition.
Futhermore, the library often needs to do message passing using the
supplied communicator. Using the same communicator as the application
version may result in some messages mix up.
Solution:
H5Pset_fapl_mpio now stores a duplicate of each of the communicator
and Info object.
H5Pget_fapl_mpio returns a duplicate of its stored communicator and
Info object. It is now the responsibility of the applications to free
those objects when done.
H5Fopen/create also stores a duplicate of the communicator and Info
object supplied by the File Access Property list.
H5Fclose frees those duplicates.
There are a few more internal VFL call back functions that they
follow this "make duplicates" requirement.
Platforms tested:
"h5committested".
What other platforms/configurations were tested?
Eirene (mpicc), sol(mpicc), copper(parallel)
Misc. update:
Need to update MANIFEST for the added t_ph5basic.c which tests the
correctness of duplicated communicator and INFO object.
Update
Description:
Updated (and in some cases added) the copyright statement.
Platforms tested:
Linux (Comment changes...only tested if they compile)
Misc. update:
Purpose:
More test.
Description:
Test independent read of groups and chunked dataset.
Solution:
This test is similar to multiple group test. So just add it in the
testphdf5.c,h.
Platforms tested:
modi4, eirene.
Bug fix (no id, just noticed myself)
Description:
Added a barrier before doing file cleanup to ensure all tests are
completed by then.
Moved the MPI_Finalize to be very end and insert an H5close before
it to ensure all potential MPI activities by HDF5 are done by then.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yes.
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
bug fix
Description:
Added a barrier to ensure all processes have finished using
the file before cleaning it away.
Added H5close() to ensure all HDF5 stuff are closed before
calling MPI_Finalize.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yes
New feature
Description:
Add support for scalar dataspaces in parallel I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Also, FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Update release_docs/RELEASE for bug fixes, new features, etc.
Testing fix.
Description:
Correctly detect when one of the parallel tests fails and stop the make
from proceeding.
Platforms tested:
modi4 (parallel), sleipnir (parallel)
API name change
Description:
Change all "space time" references to "alloc time", including API functions
and macro definitions, etc.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
Bug fix
Description:
script did not work for SP system.
Solution:
added "eval" command to launch the mpi execution. This works fine
for SP and is basically a no-op for non-batch systems like workstations.
Also got rid of the wrongle implemented TESTING message since the
result was not printed and the tests output mess up the format already.
Platforms tested:
SP (snow). Did not test more since it has been tested out fine
in the v1.4 checkin.
Additional test
Description:
Add in a fill-value to one of the tests, to make certain that they are
handled correctly.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel
Code cleanup
Description:
Cleaned up some compiler warnings.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel. Will be testing on IRIX64
6.5 (modi4) in serial & parallel shortly.
Purpose:
Design for compact dataset
Description:
Compact dataset is stored in the header message for dataset layout.
Platforms tested:
arabica, eirene.
Bug fix
Description:
t_mpi test program was not being run with the RUNPARALLEL command
Solution:
Put the t_mpi program into the TEST_PROGS_PARA macro instead of the
TEST_PROGS macro
Hopefully this'll fix the daily test problems on arabica, but I can't test
it because I can't get mpirun to work correctly for me.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
New feature
Description:
Changed parallel I/O tests to use test script instead of just running
testphdf5 executable. This allows the MPI-posix driver to be tested easily.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/parallel and IRIX64 6.5 (modi4) w/parallel
Code cleanup
Description:
Changed ifdef name from "VERBOSE" to "BARRIER_CHECKS", to better describe
what it affects.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
Tidy up
Description:
Old version showed tons of output even if MPI_Offset is too small
to support multiple GB sized files and destined to fail.
Output is pretty confusing.
Solution:
Prints the signness and size of MPI_Offset for information.
Skipped tests if MPI_Offset is not big enough to support the file
sizes.
Platforms tested:
modi4, eirene, burrwhite (all parallel).
Bug Fix
Description:
Currently, only process 0 is writing attribute data to a file. This is
incorrect, because the raw data for attributes is cached in memory until
the object header is written and other processes are not able to read the
correct attribute information.
Solution:
Have all processes participate in writing the attribute data.
Platforms tested:
IRIX64 6.5 (modi4)
Code cleanup
Description:
Platform dependent code related to the struct stat and fstat
calls polluted source codes. Hard to maintain.
Solution:
Platform dependent code are moved to H5private.h and then internal
code can #include H5private.h. Repeat those macro definition for
the stdio and multi drivers since they area examples for writing
a virtual file driver. They must not use any internal code.
Platforms tested:
eirene (parallel), modi4 (serial including gass driver.)
Users were alarmed by the OFFSET overflow and GB file size tests.
Those tests only checks the limits of the MPI implementation, not
really as an error.
Solution:
Changed the VRFY macro to indicate it is an "ERROR".
Modified the INFO macro to print messages as "REMARK (not an error)"
so that users would not be alarmed.
Added an explanation string in the GB file size write/read.
Platforms tested:
eirene and modi4 (parallel)
New test feature
Description:
Added create_faccess_plist() that create just MPIO or split+MPIO
file-access property list. This in turn can run parallel tests
with just MPIO or with Split-file VFD too.
Added -s option for split-file Plus MPIO tests.
For testphdf5.c: removed a bunch of old debug code that got left
in by mistake.
Platforms tested:
Modi4 and eirene parallel.
But it has uncovered errors in the library. The test program
is correct though. Checking the test program in so that it won't
get lost and can be used for debugging. Also, the -s is not used
by default during test. At least it won't abort "make check".
Code cleanup
Description:
This was "thrown" together in a quick way to test MPIO functionality.
Cleaned out some embrassingly useless declaration to reduce compiler
warnings.
Platforms tested:
modi4-pp and eirene-pp.
More code cleanups
Description:
Wrap up the code cleanups for changing the dataset transfer property lists
over to using the generic property list code.
Platforms tested:
IRIX64 6.5 (modi4)
Purpose:
Added attribute test
Description:
attribute test is added into t_mdset.c. At this moment, testing failed
on SunOS 5.7 and Linux. But I believe the problem is from dataset
collective writing in t_dset.c.
Platforms tested:
SGI MPI, MPICH(IRIX64 N32, IRIX64 64, IRIX, Linux, SunOS).
Multiple-group testing
Description:
Added multiple groups under root group and multiple subgroups of
certain levels. Also writes several datasets in each group in
parallel.
Solution:
Platforms tested:
MPICH(IRIX 6.5, IRIX64 6.5(64), IRIX64 6.5(N32), Linux, SunOS 5.6)
and SGI MPI(IRIX64 6.5).
Purpose:
Multiple-group testing
Description:
Added multiple groups under root group and multiple subgroups of
certain levels. Also writes several datasets in each group in
parallel.
Solution:
Platforms tested:
MPICH(IRIX 6.5, IRIX64 6.5(64), IRIX64 6.5(N32), Linux, SunOS 5.6)
and SGI MPI(IRIX64 6.5).
Bug fix
Description:
The t_mpi used to fail and exit if any error detected.
That aborted other process in a "make check" situation.
Solution:
Introduced a new error verification as INFO. INFO is for
information only. It does not increase nerrors count.
The program always exits with 0.
Platforms tested:
eirene with mpich.
Bug fix
Description:
Some mpi.h already includes MPI-IO definitions in it (e.g., SGI,
SP2). Made the #include of mpio.h be dependent on whether some
MPI-IO constant is already defined or not.
Platforms tested:
modi4 (IRIX64) and IBM SP2.
Changed MPI_File_seek then MPI_File_write or MPI_File_read to just
MPI_File_write_at and MPI_File_read_at. Some compiler, e.g., IBM
mpcc_r does not support MPI_File_seek and MPI_File_read or MPI_File_write.
This is a better measurement against HDF5 performance since HDF5
uses MPI_File_write_at and MPI_File_read_at all the times.
It is a more thread safe to use MPI_xxx_at than the other seek then
read/write approaches.
Platforms tested:
modi4 (irix64 parallel), IBM sp2.
Bug Fix
Description:
FILENAME_MAX is short on some systems. It's better to use PATH_MAX
which tends to be longer (we hope).
Solution:
Removed FILENAME_MAX and replaced it with PATH_MAX instead.
Platforms tested:
Skinner (SDSC HP N9000).
Bug fix
Description:
added a barrier to prevent racing condition before remove file and
open file.
Platforms tested:
modi4,pp
cVS: ----------------------------------------------------------------------
Feature
Description:
Show simple performance of the MPIO and the HDF5-IO.
The mpi-perf.c is contributed by Robert Ross of ANL.
The perf.c is derived from mpi-perf.c
Platforms tested:
Modi4 (O2K, parallel)
Bug fix and clean up.
Description:
The part that should test 4GB was actually testing 2GB due to
typo.
Solution:
Corrected the typo to use 4GB constant. Rearranged the code
to group 2GB and 4GB tests in their own. Removed some duplicated
testing code.
Platforms tested:
modi4.
new test
Description:
Added two new tests.
test_mpio_offset:
Verify that MPI_Offset exceeding 2**31 can be computed correctly.
test_mpio_gb_file
Test if MPIO can write file from under 2GB to over 2GB and then
from under 4GB to over 4GB.
Platforms tested:
modi4(-64), tflops.
Improvement
Description:
Changed the default value of $NPROCS from 2 to 3. This is the
value libSAF uses since 3 processes have a much bigger chance
catching parallel errors than just 2 processes.
Changed the default dataset sizes from 800 to 1200 so that it is
a multiple for 3 (and 8 too which is ofter used in the Tflops test.)
Platforms tested:
Modi4 (IRIX64) both serial and parallel modes.
New features
Description:
Some testers found the filename lengths too short.
Changed it to use the FILENAME_MAX usually defined in stdio.h.
If not, set it to 512 which should be sufficient for users
but should not exceed any system limits.
Also added a new test parameters of ndatasets so that the tester
can specific a different number of datasets for the multiple
datasets tests.
Changed the datatype of datasets created to DOUBLE. This eliminates
the current racing conditions. But the racing bugs during conversion
still need to be tracked down and squashed.
Platforms tested:
Modi4 -64.
New parallel test features
Description:
Added test cases:
INDEPENDENT write with some processes do not participate at all.
The library should not hang for this case.
COLLECTIVE read/write with some processes asking for zero elements.
The library should work properly and not hanging.
Some house cleaning: get rid of a conditional directive that was
fixed already.
Platforms tested:
IRIX64,-64,parallel
Feature, kind of
Description:
Moved the multiple dataset test out of the write-test
part so that it can be skipped or tested independent of
the other tests.
Platforms tested:
IRIX64,-64,parallel.
Backward compatibility code
Description:
Add in code to allow the library to emulate the v1.2 API and behavior.
Platforms tested:
FreeBSD 4.2 (hawkwind)
New feature
Description:
Added tests to make sure collective read or write works correctly
even if some process has no data to transfer.
ZROW--process 0 asks for zero rows of data while other processes ask
for a slab of rows.
ZCOL--process 0 asks for zero columns of data while other processes ask
for a slab of columns.
Platforms tested:
IRIX64 parallel (-64,-n32).
cvS: ----------------------------------------------------------------------
Simple changes
Description:
testphdf5.h:
Call MPI_Abort when error is detected. MPI_Finalized was used
before but it might hang if the test has already encountered errors.
Also, it does not do the H5Eprint any more since auto report is on.
t_mdest.c:
Changed the variable name of rank and nprocs to mpi_rank and mpi_size
so that it is the same with the other tests and can use the VRFY macro
call.
Platforms tested:
modi4-64.
