Bug fix
Description:
Failed parallel tests now cause make to exit with an error.
Solution:
Edited config/conclude.am to throw an error if parallel test programs fail.
Platforms tested:
heping, modi4
Adding comments and code clean-up for code that tests collective irregular selection
Description:
For better maintenance in the future, Add comments to list the number for
(start,count,block,stride) for irregular selection for effective testing
collective chunk IO feature development in the future.
Solution:
Platforms tested:
Linux 2.4(heping), mostly comments, no need to test on other platforms.
Misc. update:
Bug fix
Description:
Changed configure.in to use an environment variable TR to set the path
to the tr utility.
Solution:
There are two kind of tr on Solaris with slightly different syntax.
HDF5's configure relies on the "standard" tr. Traditionally, HDF5ers
have needed to make sure that the "right" tr was found before the
wrong one in their path; now they can use an environment variable.
Platforms tested:
mir, shanti, sol
Misc. update:
Forgot to update release notes. Off to do that now.
Code clean-up for collective regular chunk IO tests.
Description:
Add descriptions for each tests for future maintenance.
Solution:
Platforms tested:
Mostly comments, No need to use h5committest.
heping(linux 2.4)
Misc. update:
Makefile bug fix
Description:
Previously, automake didn't output rules to build perform/mpi-perf or
the test/gen_* programs.
Now these can be built by typing 'make mpi-perf' (or 'make foo') or by
configuring with --enable-build-all.
Solution:
Automake doesn't like having rules for programs it doesn't build. Tricked
it by having these programs built "sometimes"--whenever the user enables
--build-all. This should be used mostly for testing and to ensure that
these helper programs compile.
***IMPORTANT***
These programs do *not* currently compile. When --enable-build-all is used
(not the default), gen_new_fill fails because it uses an old API. This is
an existing "bug" that has simply been exposed by this checkin.
Platforms tested:
sleipnir, modi4, sol
Misc. update:
bug fix.
Description:
The complex derived datatype test assumed that the fill value would be 0. This is not
the case on all systems.
Solution:
Modified the test to check against a known value in the outbuf array, instead of the fill value.
Platforms tested:
heping and MCR.
Misc. update:
bug fix.
Description:
Fixed typo in a comment. The word "file" was supposed to be "fill." The explanation of
how the complex derived datatype test works is much clearer now.
Solution:
Platforms tested:
minor change.
Misc. update:
bug fix.
Description:
When a parallel test script test fails, make would continue because the
way it was setup inside a for loop. Fixed it by issuing an exit 1 inside
the loop.
There was also a typo error in the newer command comparision that it
must be $${chkname} in order to be valid. Also, the test script itself
was not checked in the newer lists. All fixed.
Platforms tested:
h5committested and also hand tested in heping pp mode.
Repair synchronization bug in the metadata cache in PHDF5
Also repair numerous other bugs that surfaced in testing the
bug fix.
Description:
While operations modifying metadata must be collective, we allow
independant reads. This allows metadata caches on different processes
to adjust to different sizes, and to place the entries on their dirty
lists in different orders. Since only process 0 actually writes
metadata to disk (all other processes thought they did, but the writes
were discarded on the theory that they had to be collective), this made
it possible for another process to modify metadata, flush it, and then
read it back in in its original form (pre-modification) form. The
possibilities for file corruption should be obvious.
Solution:
Make the policy that only process 0 can write to file explicit, and
visible to the metadata caches. Thus only process 0 may flush dirty
entries -- all other caches must retain dirty entries until they are
informed by process 0 that the entries are clean.
Synchronization is handled by counting the bytes of dirty cache entries
created, and then synching up between the caches whenever the sum
exceeds an (eventually user specified) limit. Dirty metadata creation
is consistent across all processes because all operations modifying
metadata must be collective.
This change uncovered may bugs which are repaired in this checkin.
It also required modification of H5HL and H5O to allocate file space
on insertion rather than on flush from cache.
Platforms tested:
H5committest, heping(parallel & serial)
Misc. update:
A bug fix
Description:
MPI_Status_IGNORE is treated as a NULL pointer for mpich 1.2.4 or similar MPI packages.
It caused segmentation fault for MPI derived datatype test.
Solution:
Define MPI_STATUS status,
and pass &status into MPI_File_read and MPI_File_write.
Platforms tested:
too trivial to test.
Misc. update:
Improvement.
Description:
The test may hang if there are system failures that some processors
are not working.
Solution:
Added the ALARM calls to limit all tests be done with the default
alarm time. So, even if a process is hanging, the ALARM signal
would terminate the process.
Platforms tested:
tg-login2 of NCSA.
Misc. update:
bug fix
Description:
the MPI_File_get_size returns a different value in one of the tests in Windows
comment the code and not run the test in windows
a ULL suffix on the harcoded return VRY return number is needed on AIX
Solution:
Platforms tested:
Windows
AIX
Misc. update:
new features
Description:
add support for compiling the library and testphdf5 in Windows
Solution:
Platforms tested:
Linux
AIX
Solaris
Windows VC6
Misc. update:
New feature.
Description:
Added the time command to the make check target to report time usage
of the execute of each test and test scripts. This gives us some idea
how long each test takes and some vague idea it is compute bound or
not.
powerpc-ibm-aix5.x:
Change $RUNPARALLEL default setting to allow it being invoked by the
time command.
Platforms tested:
h5committested.
Feature
Description:
Added H5_CFLAGS, etc. to 1.7 branch.
Now compilation flags can be put in H5_*FLAGS and they'll be used when
building hdf5 but not in h5cc.
Platforms tested:
mir, sleipnir, modi4
Misc. update:
Bug fix
Description:
Disabled C++ shared libraries for Sun Workshop compiler.
Solution:
This bug only seems to happen when using the -xarch=v9 flag to compile in
64-bit mode, but disabling shared libraries entirely for this compiler is
an easier fix (I don't know how to detect 64 bit mode from the command line).
The framework for disabling shared libraries for other C++ compilers is
in place.
Platforms tested:
sol, mir, sleipnir, modi4
Bug fix/feature
Description:
Disabled shared libraries for a number of Fortran compilers that don't
support them.
This allows other compilers to support shared Fortran libraries.
Solution:
Added a conditional, SHARED_FORTRAN_CONDITIONAL, which is true if
Fortran supports shared libraries. It is set in configure.in.
Platforms tested:
mir, sleipnir, colonelk, heping
Bug fix
Description:
Building with --disable-static seems to have been broken on several platforms.
Fixed this.
Configure will now disable shared libraries automatically when using pgf90
or when building on Cygwin.
Solution:
To avoid errors when using shared libraries with pgf90 (which I had not
realized were compiler-specific), I had manually set convenience libraries
to use the -static linking flag. Apparently, this is not necessary, and
caused these libraries to be created as empty archives when --disable-static
was used.
Also, some libraries were including other libraries, which was not
necessary.
I also fixed code in configure.in to correctly detect whether shared
or static libraries are enabled, and moved code that disables shared libraries
to before libtool is created (rather than editing libtool after the fact).
Despite repeated warnings that only one shared library can be linked at a
time, I have yet to have shared libraries break the linking of tests on
any system. We'll see if the Daily Tests turn up anything.
Platforms tested:
mir (fortran, C++), sleipnir (C++), modi4 (fortran, C++, parallel),
sol (fortran, C++), cygwin
Bug fix.
Description:
Some tests showed the filesize was not as expected. But the error was
intermittent. This was a racing condition as some processes finish
extend_chunked_dataset() sooner than others and return to the main
body which proceeds to call the next test which also uses the same
test data file and alters it. That messes up the "slower" processes
which then see unexpected filesize.
Also, the routine create_chunked_dataset() which creates test data file
actually was executed by all processes. That is wrong.
Solution:
Added a barrier at the end of extend_chunked_dataset to make sure all
processes are done with the test data file before returning.
Changed create_chunked_dataset such that only one process would create
the test data file. The rest does nothing but just wait for it to finish.
Platforms tested:
Tested in TG-NCSA in which the errors were detected.
Misc. update:
Bug fix#281
Description:
Committed a wrong copy in the previous checkin.
Solution:
Checked in the right one and did some code cleanup, rearrangment.
Platforms tested:
heping pp.
Misc. update:
tidy up.
Description:
MPE option created *.clog file whenever it is executed in MPI.
The cleanup of *.clog files were done in individual Makefile.in.
Often, it is forgotten.
Solution:
Moved the cleaning of *.clog files to CHECK_CLEANFILES in commence.am
so that it is applied whenever check-clean is called.
Platforms tested:
heping pp using MPE.
Bug fixes
Description:
This checkin fixes an occasional error on kelgia on sol during distclean.
It also causes test scripts to depend properly on the programs they're
supposed to be testing.
Solution:
The kelgia bug was due to some files being cleaned by automake and manually.
Removed the manual cleaning in src/Makefile.am.
Test script dependencies now need to be specified manually, since the
makefile can't guess what they test from their name. Currently all test
scripts in a given directory have a single list of dependencies--this was
easy and seems to be sufficient.
These dependencies are listed in the SCRIPT_DEPEND variable in the Makefile.am.
Platforms tested:
heping, mir, modi4, sol
Misc. update:
Bug #281
Description:
A dataset created in serial mode with H5D_ALLOC_TIME_INCR allocation setting
was not extendible, either explicitly by H5Dextend or implicitly by writing
to unallocated chunks. This was because parallel mode expects the allocation
mode be H5D_ALLOC_TIME_INCR only.
Solution:
Modified library to allocate more space when needed or directed if the
file is opened by parallel mode, independent of what the dataset allocation
mode is.
Platforms tested:
Heping pp.
minor tidy up.
Description:
Changed both test programs and test scripts to use the same suffixes
(.chkexe and .chklog). Changed from .log to .chklog to avoid running
into potential conflicts by other "things" (e.g., config.log).
Tested:
Heping.
bug fix
Description:
The previous version always tests with mpi-IO driver even when mpi-posix driver is set.
Solution:
Add the option to test mpi-posix driver for the first round collective chunk IO test.
Platforms tested:
IRIX64, AIX 5.1, Linux 2.4
Misc. update:
"bug fix"
Description:
The test_mpio_derived_dtype() often hangs when fails. So it was
not run by default to avoid hanging the daily tests or confusing
users. But then new system or new codes in collective chunk fails
while one can't tell for sure if it is because of the complicated
derived type failures or something else.
Solution:
Changed the logic so it is skipped only if it is known that
the complicated MPI derived type does not work. (This is
indicated by macro H5_MPI_COMPLEX_DERIVED_DATATYPE_WORKS NOT
defined.
Platforms tested:
heping pp (where it is tested by default).
Modi4 pp (where it is SKIPPED by default.)
I also forced modi4 to test it and modi4 said it actually
is working and should change the setting of
H5_MPI_COMPLEX_DERIVED_DATATYPE_WORKS to working?!!
Misc. update:
Code cleaning.
Description:
The block of code that is conditioned by the H5_MPI_COMPLEX_DERIVED_DATATYPE_WORKS
and manipulates the return code of test_mpio_derived_dtype, does not really
belong in main. If return code of test_mpio_derived_dtype needs to be
adjusted, it should be done in test_mpio_derived_dtype.
Solution:
Moved that block of code test_mpio_derived_dtype.
Platforms tested:
heping PP.
Misc. update:
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Modified collective chunk Io test
Description:
One test(collective chunk IO test 4) is no longer needed. Comment out first.
The code should be cleaned up later.
Solution:
Platforms tested:
Misc. update:
Bug fix and improvement.
Description:
The test output were not displayed for parallel tests.
The serial tests output were always displayed whether the test was
actually ran this time or not.
Solution:
Moved the display of test output inside the target where the actual
tests are run.
Platforms tested:
Tested in heping using pp and fortran.
Bug fix and code minor cleanup.
Description:
The attempt to redirect stderr together to the log files were
done in the wrong order such that stderr output goes to where
stdout WAS. Fixed that.
Code minor cleanup--use shell variables to hold the name of the
log file and the test instead of generating them repeatedly. This
makes easier code reading, less crowded and avoid typos.
Only conclude.in is changed. The Makefile.in changes are all
derived from automake.
Platforms tested:
h5committested.
Makefile feature
Description:
'make check-clean' now cleans *.h5 files created by tests as well as
.log and .chkexe files.
Solution:
check-clean is now a separate target in its own right, which cleans less
than mostlyclean (check-clean < mostlyclean < clean < distclean).
Platforms tested:
mir, heping, modi4 (serial and parallel)
Feature: check-clean target
Description:
'make check-clean' cleans up output files from tests.
Solution:
Tests create foo.chkexe and foo.log files. Scripts create foo.chksh and
foo.logsh files. 'make check-clean' will clean these files up so that
the tests can be re-run.
Also suppressed some not-very-useful output of Makefiles when it would
echo commands.
Platforms tested:
mir, sleipnir, modi4
Misc. update:
Configuration feature
Description:
Serial test output is now stored in log files and printed when all tests
in a directory complete, or when a test fails. This should make test output
more readable and useful.
Also made changes to clean up ii_files directories that are created by some
C++ compilers/linkers.
Also fixed a few minor Makefile bugs.
Solution:
When serial tests run, their output is saved in *.log or *.logsh. While
running, tests only print when they begin and when they complete; their
more specific output (from the log file) is printed if the test fails or
when all tests have completed.
Comments welcome.
Platforms tested:
mir, modi4 (parallel and serial), copper, shanti
Purpose:
Make collective IO tests work with mpiposix
Description:
Although internally HDF5 is using independent IO for mpiposix driver,
it is still a good test to see whether HDF5 APIs can allow collective IO calls to pass
into mpiposix driver.
Previous tests ignore this test; now add it.
Solution:
Platforms tested:
AIX (copper), too minor to need to test with three platforms.
Misc. update:
Updating phase 2 work of collective IO
Description:
The current collective IO tests cannot test for number of processors bigger than
some values, this change will lift those restriction. However, the test may be slower.
Solution:
Platforms tested:
linux 2.4, AIX 5.1, Linux 2.4 IA64 and IRIX 6.5
(I haven't tested big number of processors with the restriction of the machine)
Misc. update:
Bug fix
Description:
The ".chksh" file for a test script was being created in the "source"
location rather than the build location. This can cause problems when
multiple builds are running because "slower" machines will see the ".cshsh"
file from faster machines and will not run the test script as they should.
Solution:
Use 'basename' command to strip off the path of the script and create
the ".chksh" file in the build location.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
bug fix.
Description:
-nt is not a universal option for the test command in all platforms.
The use of it in conclude.am cause some platforms to bark at the
Makefile generated.
Solution:
Created a command script bin/newer which tests if file1 is newer
than file2. Replace the "test file1 -nt file2" by
"newer file1 file2".
Platforms tested:
Tested in sol.
Bug fix
Description:
Fixed bugs that caused tests to be unable to find testhdf5.sh during
daily tests, and another that broke sol and shanti.
Solution:
When tests are run, their path is ./$testname .
When scripts are run, their path has no leading "./". Fixed.
Sol and shanti don't recognize the -e option for test, but they do know
-f. Fixed.
Also modified release notes.
Platforms tested:
mir, shanti, heping
Feature: tests now use parallel make and only run once
Description:
When make is invoked in parallel (using the -j option), several tests will
now be run simultaneously. This should speed up testing on a number of
systems.
When a test passes, it creates a foo.chkexe or foo.chksh file, which prevents
the test from running again unless the test or library changes.
Solution:
Most of the changes live in config/conclude.am. Added *.chkexe and *.chksh
files to the list of files to be cleaned at "make mostlyclean" time.
Parallel tests still run one at a time, but use the same make instructions
as serial tests.
Platforms tested:
mir, eirene, sleipnir, pommier, copper, modi4 (some errors occurred, probably
due to the test being run before updating hdf5. Will re-run tests after
checkin.)
bug fix
Description:
check-s and check-p were set as pre-requistics of test. They
would get executed in parallel if parallel make is used. This
could cause problems since serial tests are used to be executed
before parallel tests. It is not known if it is always okay to
run serial and parallel tests in simultantously.
Solution:
Change check-s and check-p as actions of the target test so
that they get executed sequnentially.
Platforms tested:
heping PP.
update.
Description:
TEST_SCRIPT_PARA is not corrected as Makefile.am has.
Ran reconfigure to update Makefile.in.
Platforms tested:
heping PP.
Misc. update:
Added check-s and check-p targets.
Description:
Added check-s and check-p targets to 1.7 branch to match changes to 1.6
branch. Now parallel and serial tests can be run separately.
Platforms tested:
mir, modi4, copper
