Remove feature
Description:
Retire threaded, balanced binary tree code from HDF5 use. Requiescat in
pace...
Also, regenerate dependencies files.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require full h5committesting (the code is already
disconnected from everything except its tests)
Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-)
Description:
Generally speaking, this is the "signed->unsigned" change to selections.
However, in the process of merging code back, things got stickier and stickier
until I ended up doing a big "sync the two branches up" operation. So... I
brought back all the "infrastructure" fixes from the development branch to the
release branch (which I think were actually making some improvement in
performance) as well as fixed several bugs which had been fixed in one branch,
but not the other.
I've also tagged the repository before making this checkin with the label
"before_signed_unsigned_changes".
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & fphdf5
FreeBSD 4.10 (sleipnir) w/threadsafe
FreeBSD 4.10 (sleipnir) w/backward compatibility
Solaris 2.7 (arabica) w/"purify options"
Solaris 2.8 (sol) w/FORTRAN & C++
AIX 5.x (copper) w/parallel & FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN
Linux 2.4 (heping) w/FORTRAN & C++
Misc. update:
Check in testing code for irregular hyperslab selection with multiple chunks.
Description:
For debugging only, will not be tested by h5committest.
Solution:
Platforms tested:
Linux 2.4 (mpich 1.2.6)
Misc. update:
Adding routines to test irrgular hyperslab selection inside one chunk.
Description:
For debugging purpose, tests are turned off now.
Solution:
Platforms tested:
AIX 5.1 and Linux 2.4 with parallel enabled.
Misc. update:
Modifying testing code for derived datatype.
However, this test won't be run in the HDF5 parallel test.
The code is checked in only for debugging purpose.
Description:
Solution:
Platforms tested:
Linux 2.4(mpich 1.2.6)
Misc. update:
To test MPI derived datatype
Description:
A very simple irregular hyperslab selection in parallel.
Will not be tested when testing HDF5 library.
Check in only for debugging purpose.
This code will be tested in the future.
Solution:
Platforms tested:
AIX 5.1, Linux 2.4+mpich 1.2.6
Misc. update:
Adding general MPI derived datatype testing code.
Description:
The testing code will not be tested. The purpose of checking in is for
better debugging later. HDF5 routine or daily test should not be aware of this.
Solution:
Platforms tested:
Copper(AIX 5.1),
Heping(Linux 2.4 + MPICH 1.2.6).
Misc. update:
Improvement.
Description:
Made all processes print hostname() by default so that it is easier
to spot problems.
Platforms tested:
Tested in copper only. It is a trivial small change.
Misc. update:
Bug fix
Description:
Relax restrictions on parallel I/O to allow compressed, chunked datasets
to be read in parallel (collective access will be degraded to independent
access, but will retrieve the information still).
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
IRIX64 6.5 (modi4)
h5committest
Code clean up
Description:
The tests used to have their own test file names due to
the original program layout. Now that it is using the
common test interface, individual test can be invoked
at will (via -o). There is no need to have different
test filenames. Only requirement is that certain read
tests must follow immediately the corresponding write
test since they use the same file.
I have combined all test file names into just one common
file name. This reduces the printout of test file name,
very beneficial when the test is run by large number of processes.
It makes future code maintenance easier too.
Platforms tested:
Tested in Eirene(PP) only. No h5committest because this is
limited to the parallel tests and the changes are trivial.
Misc. update:
Cleanup
Description:
Remove old hack for H5Eget_auto() which is not needed any more.
Reverse H5E_set_auto_stack(H5EDEFAULT,...) back to previous
code of H5E_set_auto(...). Same for H5E_get_auto_stack.
Platforms tested:
Only tested in Eirene PP as the change is pretty straight
forward.
Misc. update:
Add test to verify the fix of the parallel I/O mode confusion bug.
Description:
While the parallel I/O mode confusion bug is fixed, an automated
regression test for this bug would be useful.
Solution:
Added a modified version of the original bug demonstration program
to testphdf5.
Platforms tested:
copper
h5committested
eirene (parallel)
Misc. update:
Code cleanup
Description:
Tweak recent "forward compatibility" changes to the H5E* API (which allowed
for the old H5E API functions to remain unchanged) by allowing for the error
stack callback function (H5E_auto_t) to also remain unchanged from the 1.6
branch. This required changing the H5E{get|set}_auto routines to have the
old style H5E_auto_t type (which didn't have a stack ID parameter) and the new
H5E{get|set}_auto_stack routines to have a newer "H5E_auto_stack_t" type (which
has a stack ID parameter). This should make the H5E API changes as forwardly
compatible as possible.
One side-affect of this change was that it was impossible to determine if
the current auto error callback was the old style (H5E_auto_t) or the new style
(H5E_auto_stack_t) of callback, so a new API function (H5Eauto_is_stack) was
adde to query this.
Platforms tested:
FreeBSD 4.10 (sleipnir)
IRIX64 6.5 (modi4)
h5committest
Purpose:
Shrink HDF5 parallel test
Description:
Previously the collective chunk IO tests created and opened four HDF5 files;
it turned out that all collective chunk IO tests can use one file for testing.
Solution:
To use one file for all collective chunk IO tests with trunc option to be set in H5Fcreate.
Platforms tested:
copper(no need to use h5committest)
Misc. update:
Description: Restore 6 old error API functions back to the library to be backward
compatible with v1.6. They are H5Epush, H5Eprint, H5Ewalk, H5Eclear, H5Eset_auto,
H5Eget_auto. These functions do not have error stack as parameter.
Solution: Internally, these functions use default error stack.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
Text display change
Description:
Changed the test names to much shorter strings for easier
use.
Platforms tested:
No h5committed test. Only test parallel in eirene.
Misc. update:
Fix of bug/feature which caused testphdf5 to fail when run with more than
32 processes. This fix was originally applied to the 1.6 branch, and is
now ported to the 1.7 branch.
Description:
32 process limit was a hard coded constant.
Solution:
Modified most of the routines in t_mdset.c to adapt dynamically to
the current number of processes. In passing, I also tidied up a
few memory leaks.
Note that one new routine had been added to the 1.7 version of
t_mdset.c since the 1.6/1.7 split. I applied changes to this
routine as well.
Platforms tested:
h5committested
Tested on eirene with 4 and 8 processes. Couldn't go higher.
I would have like to repeat my 32 - 64 process tests on copper,
but was unable to do so as I don't have access to cu12t at
present. Perhaps I will be able to manage this in the next
few days. However, Albert wanted these changes checked in to
the 1.7 branch so he could work with them.
Misc. update:
Code cleanup.
Removed bunch of old options (r,w,v,i,b,e) that are no longer valid
or useful after adopting the general test interface. Moved the test
of sizeof MPI_Offset into the test routine itself.
Platforms tested:
Eirene and Sol using pp mode.
bug fix.
Description:
The test routines only print error messages but not all of them
return number of errors detected back to the main routine which
always exit with a 0 status. Thus make or shell commands could
not detect there were errors.
Solution:
Changed the test routines to return appropriate number of
errors to main routine which in turn exit with the appropriate
exit code if errors found.
Platforms tested:
Tested in Sol and eirene (pp).
Misc. update:
feature
Description:
Another revamp of the test interface.
TestInit: is used to register Test Program name, test program specific
Usage and option parsing routines.
TestUsage: will invoke extra usage routine if provided.
TestParseCmdLine: will invoke extra option parsing routine if provided.
GetTestSummary() and GetTestCleanup() replaces the previous Summary and
CleanUp arguments of TestParseCmdLine.
test/testhdf5, test/ttsafe.c, testpar/t_mpi.c, testpar/testphdf5.c:
All have been updated to use the new Test Routines.
testpar/t_mpi.c:
Also a fix of a compiler optimization bug when pgcc in Linux is
used to compile it. Changed buf[] and expected to unsigned char
type to avoid a bug that failed to do sign-extension.
Platforms tested:
"h5committested"
Also tested thread-safe option in eirene.
Purpose:
bug fix
Description:
1. The IF-block of skipping collective chunk IO tests when the number of
processes is greater than some number essentially skipped all parallel tests.
2. John tested at copper with the number of processor = 64 and collective
chunk IO tests passed, so we increase the number of precessor =24 to 64 for
skipping the test.
Solution:
1. change the IF block flow so that it only skips collective chunk IO tests when
the number of process is greater than 64.
Platforms tested:
copper(only change a bit of parallel test code, no needs to test on other platforms).
Misc. update:
Correction of the previous check.
Description:
forget moving "debug #if macro" out of this file.
Solution:
remove that "#if 0 #endif" macro block.
Platforms tested:
Compile at eirene
Too trivial to test
Misc. update:
Better collective chunk IO test arrangements
Description:
collective chunk IO tests have been verified with the number of process greater than 24 and
the test is very slow with big number of process.
That may cause confusions to users who run collective chunk IO tests.
Solution:
To avoid possible confusions, A if-block will be used to check whether the number of process
is greater than 24. If yes, the collective chunk tests will be skipped and a message will be printed out.
Platforms tested:
linux 2.4
(too trivial to use h5committest)
Misc. update:
Make collective chunk IO test more general
Description:
Previously collective chunk IO test is only fit for processor =4 with
the dimension size to be set small; sometimes people would like to test
with more than 4 processors(5,6 or more), the test therefore failed.
Solution:
To make the test case more general, dimensional size of the data is set to be large(right now 288 for each dimension), the disjoint hyperslab selection is re-calculated. Now the test cases should pass with 5,6 or 12 processors. Note, there is nothing wrong with the implementation of the library, it is the test case that causes the failure with the number of processor greater than 4.
Platforms tested:
Only at eirene, since only the test code is modified a little and it is very slow to test the parallel case.
Misc. update:
Cleanup and bug fix.
Description:
Bug fix:
the checking of dobig test was done in the wrong place
such that tests added after it would be run even for the -b option.
Moved the dobig test checking to AFTER all tests are added.
Cleanup:
Removed old code now that the new way is working fine.
Platforms tested:
Tested in Eirene parallel only.
Misc. update:
Feature
Description:
Added a feature such that if the test name starts with '-', do not run it
by default.
Platforms tested:
Eirene both serial and parallel.
Misc. update:
Code cleanup
Description:
Fix another batch of minor differences between the development and release
branches.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Code cleanup
Description:
Clean up collective chunking code a bit.
Also, add '--enable-instrument' configure flag to have a mechanism for
determining that optimized operations happened correctly in the library (instead
of just the "normal" way) by allowing 'flag' properties to be set outside the
library and set when the "right" thing happens. This is mainly for debugging
and regression checks, so we make certain we don't break optimized I/O by
accident. It's enabled by default when --enable-debug is on (which is on by
default in the development branch and off by default in the release branch),
but can also be independently controlled with its own configure flag.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
IBM p690 (copper) w/parallel
To test collective chunk IO properly.
Description:
See the previous message.
Solution:
See the previous message.
Platforms tested:
arabica(Sol 2.7), eirene(Linux), copper(AIX)
Misc. update:
To add collective chunk IO tests.
Description:
three tests are added.
1. Only one hyperslab for each process, and this hyperslab is fit in exactly one chunk.
2. non-contiguous hyperslabs in each process, these hyperslabs are fit in one chunk.
3. Single hyperslab for each process, smaller chunk is assigned. Number of chunks for
every process is equal.
Solution:
the dataset size is set to be very small, will enlarge later.
Platforms tested:
AIX 5.1(copper)
Misc. update:
Bug fix
Description:
Fix error in chunked dataset I/O where data written out wasn't read
correctly from a chunked, extendible dataset after the dataset was extended.
Also, fix parallel I/O tests to gather error results from all processes,
in order to detect errors that only occur on one process.
Solution:
Bypass chunk cache for reads as well as writes, if parallel I/O driver is
used and file is opened for writing.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Improvement
Description:
The MPI atomicity and file_sync tests may hang if a filesystem
is not able to support the operation. This will block the
whole tests. PHDF5 is not using either features. So, removed
them from the default tests.
Platforms tested:
Only in eirene using pp. Copper is still down.
Bug fix.
Description:
The data verification code didn't work. Also numerous minor bugs.
Solution:
Re-wrote data write and verification code. In particular, I
modified the verification code so it actually verifies data.
In the future, this test should be modified so it exercises
object deletes and chunk I/O.
Platforms tested:
h5committested
Code cleanup
Description:
Clean up lots of warnings based on those reported from the SGI compilers
as well as gcc.
Platforms tested:
SGI O3900, IRIX64 6.5 (Cheryl's SGI machine)
FreeBSD 4.9 (sleipnir) w/ & w/o parallel
h5committest
feature
Description:
Change testphdf5 to use the common test program syntax.
Needed to change the protocols of all test programs to
fit the requirement of the common test syntax.
Platforms tested:
"h5committested".
Also tested in sol with PP mode.
Bug fix
Description:
make check without doing make first would fail to build testphdf5.
Solution:
Put the testphdf5 back in the target of TEST_PROG_PP and removed
the "testphdf" without any parameter from testph5.sh.
Platforms tested:
copper parallel. This is not relevant to non-parallel cases.
Misc. update:
Improvement
Description:
Changed parsing of verbose level by the common test library routine.
Change t_mpi.c to use the Verbose control better.
Platforms tested:
verena (pp).
Misc. update:
Improvement.
Description:
Complete change of the verbose control to use the routines provided by
the test/libh5test.a.
Also put in a temporary fix for the H5Eset_auto() and H5Eget_auto()
so that the Compat code are isolated in one place rather than all over
the source file.
Platforms tested:
Tested in Eirene (parallel).
Misc. update:
Added a test of fill value before any data is written to a dataset.
Rename short_dataset() as dataset_fillvalue() as it reflects better
the tests. Also removed the option of -S since the fill value test
will be tested always.
Platforms tested:
"h5committested"
Misc. update:
Code cleanup.
Description:
The H5Eclear() in the VRFY and INFO macros are not needed.
After removing them, there is no need to have a separate
v1.6 Compat version.
Platforms tested:
"h5committested"
Misc. update:
Code cleanup, bug fixes
Description:
Wrap up rest of changes necessary for fixing the "short" MPI-I/O read
problem that Robb reported.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Feature.
Description:
Added the short_dataset test (was in v1.6 first.)
Platforms tested:
Tested in eirene (pp) only since these have been tested in v1.6 already.
Misc. update:
Bug fix
Description:
Clean up a couple more 1.6 compat bugs that showed up when the library
was compiled with parallel support.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel & 1.6 compat
config not tested with h5committest
Update
Description:
This completes Phase I of the FPHDF5 testing. This test does the
following:
- Creates a file
- Creates multiple non-root groups
- For each process:
For each non-root group:
Create multiple datasets
Access all datasets created above
Write to all datasets created above
- Close the file
- Reopen the file and verify the data is correct
Platforms tested:
AIX (w/ FPHDF5)
Linux (w/ FPHDF5)
Solaris (w/ Fortran & C++)
Misc. update:
new feature.
Description:
Added tests of 1wMr with options to apply Atomicity or File_sync.
Platforms tested:
only tested in eirene and Teragrid as this is just an MPI test.
Misc. update:
Update
Description:
Extensive reworking of the code. More modular now. Tests for almost
all of "Phase 1"s requirements. Need to check that data is correct
after reopening the file.
Platforms tested:
Linux (w/ FPHDF5)
AIX (w/ and w/o FPHDF5)
FPHDF5-specific fixes...No need for H5committest
Misc. update:
new test.
Description:
Added test_mpio_1wMr test which verify if the file system can support
1 process writes, many processes read.
Platforms tested:
h5committested.
Misc. update:
Update
Description:
The FPHDF5 test now writes data simultaneously from all processes to
all created datasets. It's wicked cool!
Platforms tested:
Linux (w/ FPHDF5)
Copper (w/ FPHDF5 & w/o Parallel)
FPHDF5-specific, so no h5committest
Misc. update:
Update
Description:
Modified the FPHDF5 test so that a single process creates multiple
datasets in a non-root group then all of the processes (sans the SAP)
access it.
Modularized the code a bit more to make it easy to perform the above
tests.
Changed the code a bit so that >3 processes can (theoretically) be
used to test this.
Platforms tested:
Linux (FPHDF5 specific change)
Misc. update:
Update
Description:
Expanded the FPHDF5 test to include creating multiple datasets by one
process in a non-root group.
Cleaned up the code a bit more to make it more modular and easier to
work with.
Platforms tested:
Linux (FPHDF5 specific)
Misc. update:
Update
Description:
Added test for FPHDF5 to the testpar/ directory. If you specify
--enable-fphdf5 during configuration, it'll run this test during a
"make check".
Platforms tested:
Linux (FPHDF5 specific).
Misc. update:
Feature, sort of.
Description:
Reactivated the big dataset test. Changed it
to default off, can be turned on via -b.
Platforms tested:
Only in Copper which is the only local machine safe to run
mulitple GB size files.
Misc. update:
Bug/feature fix.
Description:
Relax restriction on parallel writing to compact datasets to allow partial
I/O.
Updates to reference manual mentioning the issues involved are delayed until
reference manual 'lock' is removed later this week.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
Disable "big" parallel tests until I get a chance to look into the
problems (probably during/after the workshop).
Platforms tested:
Eyeballed
Too minor to require h5committest
Bug fix
Description:
The MPI_File_set_size() routine on ASCI Red is not able to extend files
so that they are larger than 2GB.
Solution:
Add an extra macro which controls whether MPI_File_set_size() can handle
>2GB offsets or if our "older" way of reading a byte, then writing a byte at
the appropriate offset should be used.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug fix
Description:
Temporary files were not cleaned out because the definiton of
temporary files had not been updated.
Solution:
Updated it.
Platforms tested:
Tested by hand in Copper while i was testing the MPE change.
Misc. update:
Updated the copyright notice--mostly by rearranging
some text to make them consistent.
Solution:
Platforms tested:
"h5committested"--sol is down, so, no SUn test.
Misc. update:
Compatibility fix
Description:
The H5P[set|get]_fapl_mpiposix calls changed between v1.4.x and v1.5.x.
Solution:
Wrap them in the v1.4 backward compatibility #ifdefs and update tests, etc.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/paralle & v1.4 compatibility
h5committest pointless
Bug fix
Description:
An earlier checkin changed some of the assumptions about single block
hyperslabs, causing them to fail in odd ways.
Solution:
Fix errors with single block hyperslabs by keying off of count==1 instead
of stride==1.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/parallel
h5committested
Add printout to identify when the MPIPOSIX driver is being tested instead
of the MPI-IO driver..
Platforms tested:
Tested in copper in parallel mode only since it was just a simple
print statement in the parallel test code.
Misc. update:
Feature Add
Description:
Added knob so that the programmer can enable or disable GPFS
hints during runtime instead of having it only enabled at
configure/compile time. Some of the public APIs were changed
to add an extra parameter for this option...
Platforms tested:
Blue (LLNL). It only affects the MPI/POSIX driver, so no need
to test it on non-GPFS platforms.
Misc. update:
feature
Description:
added test_fapl_mpiposix_dup to verify the MPIPOSIX is making
duplicates of Communicator correctly.
Platforms tested:
Did not h5committested because these are all MPI code and well protected
by HAVE_PARALLEL.
Tested in eirene and modi4, both parallel mode.
Misc. update:
code cleanup.
Description:
Rename test_comm_info_delete to test_fapl_mpio_dup to better decribe
what it does. Updated the comments to describe more accurate.
Platforms tested:
Did not run h5commmittest since the changes were all in parallel area.
Ran parallel tests on modi4 and eirene.
Misc. update:
simple code cleanup.
Description:
While debug a problem in multiple_group_write(), noticed some returned
values were not checked. Added code to check on all returned code.
Platforms tested:
h5committested.
Misc. update:
bug fix
Description:
The fphdf5.o and similar files are not included in the make clean
target. So, if the same directory is used again for a different
build, the left behind file could cause a problem (e.g., if
the second build is for a different binary format.)
Solution:
Added fphdf5.c to the TEST_SRC list which indirect form the clean
target list.
Platforms tested:
Did not run h5committest tests because they don't verify this
failure. I tested the fix in modi4 pp to verify all files are cleaned.
Misc. update:
Feature
Description:
Test program for the Flexible PHDF5 feature.
Platforms tested:
Tested on modi4 before but the feature is currenly disabled in
configure. Checking this in for future work.
Misc. update:
Bug fixes/API changes
Description:
Previously, the Communicator and Info object arguments supplied
to H5Pset_fapl_mpio() are stored in the property with its handle
values. This meant changes to the communicator or the Info object
after calling H5Pset_fapl_mpio would affect the how the property
list function. This was also the case when H5Fopen/create operated.
They just stored the handle value. This is not according to the
MPI-2 defined behavior of how Info objects should be handled. (MPI-2
defines Info objects must be parsed when called.)
The old design was trying to avoid numerous duplicates of the same
information (e.g., every property object holds one version, every
file opened holds another version, when all of them are referring to
the same original version.) Nevertheless it is safer to implement
it according to MPI-2 definition.
Futhermore, the library often needs to do message passing using the
supplied communicator. Using the same communicator as the application
version may result in some messages mix up.
Solution:
H5Pset_fapl_mpio now stores a duplicate of each of the communicator
and Info object.
H5Pget_fapl_mpio returns a duplicate of its stored communicator and
Info object. It is now the responsibility of the applications to free
those objects when done.
H5Fopen/create also stores a duplicate of the communicator and Info
object supplied by the File Access Property list.
H5Fclose frees those duplicates.
There are a few more internal VFL call back functions that they
follow this "make duplicates" requirement.
Platforms tested:
"h5committested".
What other platforms/configurations were tested?
Eirene (mpicc), sol(mpicc), copper(parallel)
Misc. update:
Need to update MANIFEST for the added t_ph5basic.c which tests the
correctness of duplicated communicator and INFO object.
Update
Description:
Updated (and in some cases added) the copyright statement.
Platforms tested:
Linux (Comment changes...only tested if they compile)
Misc. update:
Purpose:
More test.
Description:
Test independent read of groups and chunked dataset.
Solution:
This test is similar to multiple group test. So just add it in the
testphdf5.c,h.
Platforms tested:
modi4, eirene.
Bug fix (no id, just noticed myself)
Description:
Added a barrier before doing file cleanup to ensure all tests are
completed by then.
Moved the MPI_Finalize to be very end and insert an H5close before
it to ensure all potential MPI activities by HDF5 are done by then.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yes.
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
bug fix
Description:
Added a barrier to ensure all processes have finished using
the file before cleaning it away.
Added H5close() to ensure all HDF5 stuff are closed before
calling MPI_Finalize.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yes
New feature
Description:
Add support for scalar dataspaces in parallel I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Also, FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Update release_docs/RELEASE for bug fixes, new features, etc.
Testing fix.
Description:
Correctly detect when one of the parallel tests fails and stop the make
from proceeding.
Platforms tested:
modi4 (parallel), sleipnir (parallel)
API name change
Description:
Change all "space time" references to "alloc time", including API functions
and macro definitions, etc.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/C++
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
Bug fix
Description:
script did not work for SP system.
Solution:
added "eval" command to launch the mpi execution. This works fine
for SP and is basically a no-op for non-batch systems like workstations.
Also got rid of the wrongle implemented TESTING message since the
result was not printed and the tests output mess up the format already.
Platforms tested:
SP (snow). Did not test more since it has been tested out fine
in the v1.4 checkin.
Additional test
Description:
Add in a fill-value to one of the tests, to make certain that they are
handled correctly.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel
Code cleanup
Description:
Cleaned up some compiler warnings.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel. Will be testing on IRIX64
6.5 (modi4) in serial & parallel shortly.
Purpose:
Design for compact dataset
Description:
Compact dataset is stored in the header message for dataset layout.
Platforms tested:
arabica, eirene.
Bug fix
Description:
t_mpi test program was not being run with the RUNPARALLEL command
Solution:
Put the t_mpi program into the TEST_PROGS_PARA macro instead of the
TEST_PROGS macro
Hopefully this'll fix the daily test problems on arabica, but I can't test
it because I can't get mpirun to work correctly for me.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
New feature
Description:
Changed parallel I/O tests to use test script instead of just running
testphdf5 executable. This allows the MPI-posix driver to be tested easily.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/parallel and IRIX64 6.5 (modi4) w/parallel
Code cleanup
Description:
Changed ifdef name from "VERBOSE" to "BARRIER_CHECKS", to better describe
what it affects.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
Tidy up
Description:
Old version showed tons of output even if MPI_Offset is too small
to support multiple GB sized files and destined to fail.
Output is pretty confusing.
Solution:
Prints the signness and size of MPI_Offset for information.
Skipped tests if MPI_Offset is not big enough to support the file
sizes.
Platforms tested:
modi4, eirene, burrwhite (all parallel).
Bug Fix
Description:
Currently, only process 0 is writing attribute data to a file. This is
incorrect, because the raw data for attributes is cached in memory until
the object header is written and other processes are not able to read the
correct attribute information.
Solution:
Have all processes participate in writing the attribute data.
Platforms tested:
IRIX64 6.5 (modi4)
Code cleanup
Description:
Platform dependent code related to the struct stat and fstat
calls polluted source codes. Hard to maintain.
Solution:
Platform dependent code are moved to H5private.h and then internal
code can #include H5private.h. Repeat those macro definition for
the stdio and multi drivers since they area examples for writing
a virtual file driver. They must not use any internal code.
Platforms tested:
eirene (parallel), modi4 (serial including gass driver.)
Users were alarmed by the OFFSET overflow and GB file size tests.
Those tests only checks the limits of the MPI implementation, not
really as an error.
Solution:
Changed the VRFY macro to indicate it is an "ERROR".
Modified the INFO macro to print messages as "REMARK (not an error)"
so that users would not be alarmed.
Added an explanation string in the GB file size write/read.
Platforms tested:
eirene and modi4 (parallel)
New test feature
Description:
Added create_faccess_plist() that create just MPIO or split+MPIO
file-access property list. This in turn can run parallel tests
with just MPIO or with Split-file VFD too.
Added -s option for split-file Plus MPIO tests.
For testphdf5.c: removed a bunch of old debug code that got left
in by mistake.
Platforms tested:
Modi4 and eirene parallel.
But it has uncovered errors in the library. The test program
is correct though. Checking the test program in so that it won't
get lost and can be used for debugging. Also, the -s is not used
by default during test. At least it won't abort "make check".
Code cleanup
Description:
This was "thrown" together in a quick way to test MPIO functionality.
Cleaned out some embrassingly useless declaration to reduce compiler
warnings.
Platforms tested:
modi4-pp and eirene-pp.
More code cleanups
Description:
Wrap up the code cleanups for changing the dataset transfer property lists
over to using the generic property list code.
Platforms tested:
IRIX64 6.5 (modi4)
Purpose:
Added attribute test
Description:
attribute test is added into t_mdset.c. At this moment, testing failed
on SunOS 5.7 and Linux. But I believe the problem is from dataset
collective writing in t_dset.c.
Platforms tested:
SGI MPI, MPICH(IRIX64 N32, IRIX64 64, IRIX, Linux, SunOS).
Multiple-group testing
Description:
Added multiple groups under root group and multiple subgroups of
certain levels. Also writes several datasets in each group in
parallel.
Solution:
Platforms tested:
MPICH(IRIX 6.5, IRIX64 6.5(64), IRIX64 6.5(N32), Linux, SunOS 5.6)
and SGI MPI(IRIX64 6.5).
Purpose:
Multiple-group testing
Description:
Added multiple groups under root group and multiple subgroups of
certain levels. Also writes several datasets in each group in
parallel.
Solution:
Platforms tested:
MPICH(IRIX 6.5, IRIX64 6.5(64), IRIX64 6.5(N32), Linux, SunOS 5.6)
and SGI MPI(IRIX64 6.5).
Bug fix
Description:
The t_mpi used to fail and exit if any error detected.
That aborted other process in a "make check" situation.
Solution:
Introduced a new error verification as INFO. INFO is for
information only. It does not increase nerrors count.
The program always exits with 0.
Platforms tested:
eirene with mpich.
Bug fix
Description:
Some mpi.h already includes MPI-IO definitions in it (e.g., SGI,
SP2). Made the #include of mpio.h be dependent on whether some
MPI-IO constant is already defined or not.
Platforms tested:
modi4 (IRIX64) and IBM SP2.
Changed MPI_File_seek then MPI_File_write or MPI_File_read to just
MPI_File_write_at and MPI_File_read_at. Some compiler, e.g., IBM
mpcc_r does not support MPI_File_seek and MPI_File_read or MPI_File_write.
This is a better measurement against HDF5 performance since HDF5
uses MPI_File_write_at and MPI_File_read_at all the times.
It is a more thread safe to use MPI_xxx_at than the other seek then
read/write approaches.
Platforms tested:
modi4 (irix64 parallel), IBM sp2.
Bug Fix
Description:
FILENAME_MAX is short on some systems. It's better to use PATH_MAX
which tends to be longer (we hope).
Solution:
Removed FILENAME_MAX and replaced it with PATH_MAX instead.
Platforms tested:
Skinner (SDSC HP N9000).
Bug fix
Description:
added a barrier to prevent racing condition before remove file and
open file.
Platforms tested:
modi4,pp
cVS: ----------------------------------------------------------------------
Feature
Description:
Show simple performance of the MPIO and the HDF5-IO.
The mpi-perf.c is contributed by Robert Ross of ANL.
The perf.c is derived from mpi-perf.c
Platforms tested:
Modi4 (O2K, parallel)
Bug fix and clean up.
Description:
The part that should test 4GB was actually testing 2GB due to
typo.
Solution:
Corrected the typo to use 4GB constant. Rearranged the code
to group 2GB and 4GB tests in their own. Removed some duplicated
testing code.
Platforms tested:
modi4.
new test
Description:
Added two new tests.
test_mpio_offset:
Verify that MPI_Offset exceeding 2**31 can be computed correctly.
test_mpio_gb_file
Test if MPIO can write file from under 2GB to over 2GB and then
from under 4GB to over 4GB.
Platforms tested:
modi4(-64), tflops.
Improvement
Description:
Changed the default value of $NPROCS from 2 to 3. This is the
value libSAF uses since 3 processes have a much bigger chance
catching parallel errors than just 2 processes.
Changed the default dataset sizes from 800 to 1200 so that it is
a multiple for 3 (and 8 too which is ofter used in the Tflops test.)
Platforms tested:
Modi4 (IRIX64) both serial and parallel modes.
New features
Description:
Some testers found the filename lengths too short.
Changed it to use the FILENAME_MAX usually defined in stdio.h.
If not, set it to 512 which should be sufficient for users
but should not exceed any system limits.
Also added a new test parameters of ndatasets so that the tester
can specific a different number of datasets for the multiple
datasets tests.
Changed the datatype of datasets created to DOUBLE. This eliminates
the current racing conditions. But the racing bugs during conversion
still need to be tracked down and squashed.
Platforms tested:
Modi4 -64.
New parallel test features
Description:
Added test cases:
INDEPENDENT write with some processes do not participate at all.
The library should not hang for this case.
COLLECTIVE read/write with some processes asking for zero elements.
The library should work properly and not hanging.
Some house cleaning: get rid of a conditional directive that was
fixed already.
Platforms tested:
IRIX64,-64,parallel
Feature, kind of
Description:
Moved the multiple dataset test out of the write-test
part so that it can be skipped or tested independent of
the other tests.
Platforms tested:
IRIX64,-64,parallel.
Backward compatibility code
Description:
Add in code to allow the library to emulate the v1.2 API and behavior.
Platforms tested:
FreeBSD 4.2 (hawkwind)
New feature
Description:
Added tests to make sure collective read or write works correctly
even if some process has no data to transfer.
ZROW--process 0 asks for zero rows of data while other processes ask
for a slab of rows.
ZCOL--process 0 asks for zero columns of data while other processes ask
for a slab of columns.
Platforms tested:
IRIX64 parallel (-64,-n32).
cvS: ----------------------------------------------------------------------
Simple changes
Description:
testphdf5.h:
Call MPI_Abort when error is detected. MPI_Finalized was used
before but it might hang if the test has already encountered errors.
Also, it does not do the H5Eprint any more since auto report is on.
t_mdest.c:
Changed the variable name of rank and nprocs to mpi_rank and mpi_size
so that it is the same with the other tests and can use the VRFY macro
call.
Platforms tested:
modi4-64.
Bug fix, cleanup,...
Description:
The test was doing the hyperslab select incorrectly (thinking
count was the block length.
Solution:
Fixed it to do the correct hyperslab selection.
Changed it to calculate different data for different datasets.
Changed output by rows instead by cols. It tests the purpose
of creating multiple datasets the same but runs faster.
Platforms tested:
modi4-64.
Features, kind of.
Description:
Separated the MPI features test into its own independent
program so that it can be tested on its own without too
much HDF5 stuff involved.
Added automatic removal of temporary test files after
the tests completed.
Reduced the size of the dataset dimensions to avoid tripping
the SGI MPI problems of running out of internal mpi type entries.
Platforms tested:
O2K -64
Added features
Description:
There were no automatic tests for transfering zero elements.
Solution:
t_dset.c:
Added two new patterns of ZROW (zero rows for process 0)
and ZCOL(zero columns for process 0).
ZROW test was added but it failed because the current library
does not accept it. Not compiled in now. Need to fix the
library before turning it back on again and also to add the
ZCOL test.
t_mdset.c:
Added statement to show progress. Also the MPI_Barrier() call
get processes synchornoized. It eliminates the racing condition
but this is not a permenant solution. The library code needs to
be fixed.
testphdf5.c:
Added a bunch of MPI_Type_XXX debug code. Added the -md
option to skip the multiple datasets tests. Changed the cosmitic
appearance of the banner messages.
testphdf5.h:
When an error is detected, the old way was to call MPI_Finalize()
before exiting. This sometimes hangs because some processes
may be waiting for a message of a different tag. Changed to
call MPI_Abort() for now so that the whole MPI job would
abort rather than hanging due resource limits exceeded.
Added the definition of ZROW and ZCOL.
Platforms tested:
Modi4 -64.
Bug fix
Description:
The old code was using count as the block size. The result was
asking for a slab of count blocks, each of 1 element. The recent
change in the hyperslab algorithm exposed this problem. (The
old algorithm merge the count blocks back into 1 big block of
count elements.) (This error was due to that the block argument
was not in the very early version of hyperslab. Then it was
not updated since it had been "working".)
Solution:
Added in the block argument to the setup and calculation of
slab and its data. Also found a dumb error in the dataset_fill
algorithm in which stride was used in the calculation. Not so
for the cases of BYROW and BYCOL.
Platforms tested:
modi4 parallel, both -64 an -n32 modes.
Makefile.in:
Added test/ as one of the -I directories to search for header files.
Needed because <h5test.h> is used.
t_file.c t_mpi.c testphdf5.c testphdf5.h:
Added FILENAME to meet the assumption in h5test.h. (May use
CLEANUP in the future.) Moved the prefix setting to the
h5_fixname().
Increase the test size to 32. Put in a check to make sure number of
processes are not bigger than SIZE.
testphdf5.c:
Fixed a mistake in the prototype of pause_proc to reflect no arguments.
** src/H5FDpublic.h
Added a #define for H5_HAVE_VFL to make life easier for application
programmers.
** config/depend1
A minor tweak to the way GNU systems generate file dependencies for
the Makefiles.
** src/H5T.c
Fixed a bug with enumeration types not having the correct object
header pointer.
** bin/rpmsync
** config/depend1.in
Tweaks for versions of `make' that can't search directories.
** examples/Dependencies
** src/Dependencies
** test/Dependencies
** testpar/Dependencies
** tools/Dependencies
Regenerated all these with more complete path names.
----------------------
./MANIFEST
./configure.in
./configure [REGENERATED]
Added more checking for `make' features.
./Makefile.in
./doc/Makefile.in
./doc/html/Makefile.in
./doc/html/Tutor/Makefile.in
./examples/Makefile.in
./pablo/Makefile.in
./src/Makefile.in
./test/Makefile.in
./testpar/Makefile.in
./tools/Makefile.in
./config/commence.in
./config/conclude.in
./config/depend.in [REMOVED]
./config/depend1.in [NEW]
./config/depend2.in [NEW]
./config/depend3.in [NEW]
./config/depend4.in [NEW]
./config/dependN.in [NEW]
The directory search stuff was moved into commence.in, thereby
shortening the Makefile.in prologues.
./doc/html/Dependencies [NEW]
./doc/html/Tutor/Dependencies [NEW]
./examples/Dependencies [NEW]
./src/Dependencies [NEW]
./test/Dependencies [NEW]
./testpar/Dependencies [NEW]
./tools/Dependencies [NEW]
The `.distdep' files were all renamed to `Dependencies' to make them
more obvious. They are required (but may be empty) in every directory
that has a Makefile.in that ends with @CONCLUDE@ (you'll get an
obvious error from make if you forgot to create one).
./bin/trace
./src/H5.c
Added H5E_major_t and H5E_minor_t although tracing only prints the
integer value.
./src/H5E.c
./src/H5Epublic.h
Added tracing information.
./src/H5FDcore.c
./src/H5FDfamily.c
./src/H5FDgass.c
./src/H5FDmpio.c
./src/H5FDsec2.c
./src/H5FDstdio.c
Fixed places where FUNC_LEAVE() evaluated it's argument more than
once.
Added tracing information.
Wrapped long lines.
./config/gnu-flags
Fixed a syntax error when we don't have a gnu compiler.
----------------------
This extensive change is the virtual file layer implementation. I've
ported and tested the sec2, family, and core drivers and only ported
the mpio driver (Albert will test it). So if you need MPIO I would
recommend sticking with the previous version for a while.
You will get a few compile warnings about split and stdio drivers not
being implemented and possibly tracing information not inserted in
some of the drivers. You can safely ignore them but I plan to fix
them.
I'm still working on the split driver because I just realized that it
needs a part of the VFL that isn't written yet.
Documentation is being updated also because there were some minor
changes (mostly just name changes). It should be available on my web
site later this week.
./MANIFEST
./src/Makefile.in
./src/hdf5.h
./src/H5Flow.c [REMOVED]
./src/H5Fstdio.c [REMOVED]
./src/H5Fsec2.c [REMOVED]
./src/H5Fsplit.c [REMOVED]
./src/H5Fmpio.c [REMOVED]
./src/H5Ffamily.c [REMOVED]
./src/H5Fcore.c [REMOVED]
./src/H5MFpublic.h [REMOVED]
./src/H5FD.c [NEW]
./src/H5FDcore.c [NEW]
./src/H5FDcore.h [NEW]
./src/H5FDfamily.c [NEW]
./src/H5FDfamily.h [NEW]
./src/H5FDmpio.c [NEW]
./src/H5FDmpio.h [NEW]
./src/H5FDprivate.h [NEW]
./src/H5FDpublic.h [NEW]
./src/H5FDsec2.c [NEW]
./src/H5FDsec2.h [NEW]
Removed/added files for virtual file layer.
./bin/trace
./src/H5.c
Removed unused public datatypes and added new VFL public
datatypes.
Changed an error message.
./config/BlankForm
./config/dec-flags
./config/gnu-flags
./config/hpux10.20
./config/hpux9.03
./config/irix5.x
./config/irix6.x
./config/solaris2.x
./config/unicosmk
Removed the H5F_OPT_SEEK and H5F_LOW_DFLT constants from the
configuration since they're no longer applicable. The default
file driver is always the sec2 driver and it always optimizes
calls to lseek() or lseek64().
./config/depend.in
C preprocessor errors generated during automatic dependency
building are sent to /dev/null to prevent them from appearing
twice in the make output.
./src/H5AC.c
./src/H5B.c
./src/H5D.c
./src/H5F.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5Oattr.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Oshared.c
./src/H5T.c
./src/H5detect.c
./test/ohdr.c
Changed H5F_ADDR_UNDEF to HADDR_UNDEF to be more consistent
with the `haddr_t' datatype which is now a public type.
./src/H5D.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Tconv.c
./test/cmpd_dset.c
./test/dsets.c
./test/overhead.c
./test/tselect.c
./test/tvltypes.c
The H5P_DATASET_XFER constant was changed to H5P_DATA_XFER
because the properties apply to all types of I/O operations,
not just datasets.
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5R.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Smpio.c
./src/H5Spoint.c
./src/H5Sprivate.h
./test/big.c
./test/h5test.c
./test/istore.c
./testpar/t_dset.c
./testpar/t_file.c
./tools/h5debug.c
./tools/h5ls.c
Modified to work with the virtual file layer by calling H5FD_*
functions instead of H5F_low_* functions and by passing file
access and data transfer properties by object ID instead of
pointer.
Changed H5D_transfer_t to H5FD_mpio_xfer_t since the
COLLECTIVE vs. INDEPENDENT transfer mode is specific to the
MPIO file driver.
Moved MPIO-specific stuff into the MPIO driver.
./src/H5B.c
./src/H5D.c
./src/H5Fprivate.h
The H5F_mpio_* private functions were renamed and placed in
the H5FDmpio driver except those which appeared in H5Smpio.c.
./src/H5E.c
./src/H5Epublic.h
Added major error number H5E_VFL for virtual file layer
related errors.
./src/H5F.c
./src/H5Fprivate.h
Changed the logic that controls whether the boot block is
written. Instead of assuming that the first call to write the
boot block is only to allocate space, I've added a function
argument which makes this explicit.
Changed the way files are compared so that a driver-defined
comparison function can be called. Files which belong to
different drivers are always considered different.
Removed H5F_driver_t since file drivers are now identified by
object ID instead of a special non-user-extendible datatype.
Removed all the hard-coded low-level file properties which
have been replaced by the various file drivers.
./src/H5I.c
./src/H5Iprivate.h
Added the H5I_inc_ref() which was removed a few months ago
since we finally have a use for it.
./src/H5Ipublic.h
Added the H5I_VFL object ID type to identify file drivers in
the virtual file layer.
./src/H5MF.c
./src/H5MFprivate.h
Moved all the allocation/deallocation code into the virtual
file layer which allows file drivers to override much of it.
./src/H5P.c
./src/H5Ppublic.h
Moved file driver-specific code into the various file driver
files.
The H5Pcopy() and H5Pclose() functions make calls into the
virtual file driver to manage the memory for driver-specific
file access and data transfer properties.
./src/H5private.h
./src/H5public.h
The `haddr_t' type is now public.
./test/tfile.c
Added a few more comments.
----------------------
./Makefile.in [1.3]
./examples/Makefile.in [1.3]
./pablo/Makefile.in [1.3]
./src/Makefile.in [1.3]
./test/Makefile.in [1.3]
./testpar/Makefile.in [1.3]
./tools/Makefile.in [1.3]
./config/commence.in [1.3]
./config/conclude.in [1.3]
./config/depend.in [1.3]
Added a `.PATH' target for Irix pmake which is identical to
the VPATH used by most other `make' programs. Irix 6.5 (or
6.4?) and later `make' ignores VPATH.
Started all beginning-of-line comments with `##' instead of
just `#' because Irix `pmake' barfs on things like the
following because it sees the `# if' as a directive:
# make will barf on the following line
# if running Irix pmake
./config/commence.in [1.3]
Added empty definitions for optional `make' macros because
Irix pmake complains if it sees a macro which has never been
defined. The main body of the Makefiles can redefine these
macros to whatever they need.
Removed the unnecessary #include of mpi.h and mpio.h. (They are
included already in H5public.h when applicable.) Changed couple
debug printing to be controlled like the others.
src/H5public.h:
Officially, mpio.h should be in mpi.h but MPIO are not always
supported by the vendor yet. Make the #include of mpio.h conditional
when it is not in mpi.h.
testpar/testphdf5.h:
Removed the unnecessary #include of mpi.h and mpio.h. (They are
included already in H5public.h when applicable.)
Changed it to skip the test instead of aborting when there is not
enough processes to do the test. Also corrected an error in the
error reporting printf statement.
t_dset.c:
testphdf5.c:
testphdf5.h:
Added option for specifying chunk dimensions.
new feature
Makefile.in:
Added the dependence of *.c on the testphdf5.h
t_dset.c:
testphdf5.c:
testphdf5.h:
testphdf5 now takes optional arguements for dataset dimension sizes.
That allows testing with different dimension sizes without recompiling
the whole thing.
Platform tested:
O2K
----------------------
./config/commence.in
./examples/Makefile.in
./test/Makefile.in
./testpar/Makefile.in
./tools/Makefile.in
Fixed some dependency problems in Makefiles. If the library
version is updated then everything gets recompiled.
./src/H5G.c
./src/H5R.c
Failure return value for object type functions is now
H5G_UNKNOWN as documented.
./src/H5Shyper.c
./src/H5Spoint.c
Fixed an unused argument warning.
./tools/h5debug.c
Fixed a call to H5F_block_read() since the 4th argument is
different now.
./tools/h5ls.c
Added a space between the object name and class to make the
output readable when the object name is longer than 24
characters.
C
----------------------
./Makefile.in
./configure.in
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
./config/commence.in
./config/conclude.in
./config/dec-osf4.x
./config/depend.in
./config/freebsd
./config/linux-gnu
./config/linux-gnulibc1
./config/linux-gnulibc2
./config/solaris2.x
./examples/Makefile.in
./src/Makefile.in
./test/Makefile.in
./testpar/Makefile.in
./tools/Makefile.in
./tools/testh5dump.sh
Changes that allow hdf5 to be configured and compiled in a
directory other than the source directory. This is especially
useful if you want to concurrently compile with different
compilers and/or compile flags, or if the hdf5 source is on
read-only media like a CDROM. The changes were surprisingly
easy ;-) Here's what you can do now...
$ mkdir /tmp/build1 # or something
$ cd /tmp/build1
$ /cdrom/hdf5/configure # where ever the source is
$ make
Paul, you'll have to change the testh5toh4 script similar to
the way I changed testh5dump.sh. I started working on it but
then gave up because of a number of problems: (1) I had to
comment out all the tests that let h5toh4 choose the output
file name because it always tried to put the output file in
the same directory as the input file, (2) if path names are
used during the h4 dump then they interfere with the
diff. The test works fine when run in the source directory,
but try this instead:
$ gunzip <hdf5-1.1.67.tar.gz |tar xf -
$ chmod -R ugo-w hdf5-1.1.67
$ mkdir build
$ cd build
$ ../hdf5-1.1.67/configure
$ make check
Dan, I didn't modify the pablo/Makefile.in because I have no
way to test it. I think all you need to do is add a couple
lines before the @COMMENCE@ line and add a couple search
directories for header files. Anyway, it seems like almost an
exact duplicate of the src/Makefile.in, so it shouldn't be a
problem...
./Makefile.in
Removes a few more temporary files during make clean and
distclean.
./configure.in
I fixed the creation of the time-stamp files so that the
initial make doesn't have to regenerated all the makefiles
(only GNU make users will see any change).
./src/H5R.c
Changed a return type from `intn' to `int' for an API
function.
./src/H5detect.c
Added the volatile qualifier to a couple more variables.
./tools/h5tools.c
Fixed a compiler warning about an unused local.
./tools/h5toh4.c
Initialized `optind' to 1 because it's zero on my system. It
should always be one on a unix system.
./tools/testh5toh4
Redirected "broken pipe" messages to /dev/null so outut is
formatted correctly.
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
./src/H5private.h
Temporarily commented out the code that tries to link a simple
MPI-IO application because I'm not sure how to pass $LDFLAGS
and $LIBS to the compiler.
Removed the `--enable-parallel=ibm' switch because the library
we link with is either -lmpcc or -lmpcc_r but not both. The
only way to tell is to see what compiler was specified (mpcc
or mpcc_r) but if that compiler is specified then we don't
need any libraries (the compiler script supplies them). That
leaves just two choices: the user must use a compiler script:
CC=mpcc ./configure
or the user must state which library is desired:
LDFLAGS='-lmpcc' ./configure --enable-parallel
Checks for <setjmp.h>, longjmp(), and signal(). We don't check
for setjmp() because it could be a macro (in fact, Posix
requires it to be a macro) and if longjmp() is present then
setjmp() is probably present too ;-)
./src/H5detect.c
The alignment detection loop uses SIGBUS and setjmp/longjmp
instead of fork/wait in order to get around bugs with forking
in conjunction with mpich. This hasn't been tested on the
SP-2 yet but it does work on the DEC Alpha.
./test/Makefile.in
./testpar/Makefile.in
./tools/Makefile.in
Changed the order that libraries are linked so -lhdf5 is
always before $LDFLAGS.
----------------------
./INSTALL
./configure.in
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
Improvements for parallel library. If you have a properly
working mpicc you should be able to just say:
$ CC=mpicc ./configure
and you will see
checking for mpirun... /usr/local/mpi/bin/mpirun
checking for parallel support files... skipped
checking how to run on one processor...
/usr/local/mpi/bin/mpirun -np 1
checking how to run in parallel...
/usr/local/mpi/bin/mpirun -np $$NPROCS
To quote from the INSTALL file....
*** Parallel vs. serial library
The HDF5 library can be configured to use MPI and MPI-IO for
parallelizm on a distributed multi-processor system. The easy
way to do this is to have a properly installed parallel
compiler (e.g., MPICH's mpicc or IBM's mpcc) and supply that
executable as the value of the CC environment variable:
[NOTE: mpcc is not tested yet]
$ CC=mpcc ./configure
$ CC=/usr/local/mpi/bin/mpicc ./configure
If no such wrapper script is available then you must specify
your normal C compiler along with the distribution of
MPI/MPI-IO which is to be used (values other than `mpich' will
be added at a later date):
$ ./configure --enable-parallel=mpich
If the MPI/MPI-IO include files and/or libraries cannot be
found by the compiler then their directories must be given as
arguments to CPPFLAGS and/or LDFLAGS:
$ CPPFLAGS=-I/usr/local/mpi/include \
LDFLAGS=-L/usr/local/mpi/lib/LINUX/ch_p4 \
./configure --enable-parallel=mpich
If a parallel library is being built then configure attempts
to determine how to run a parallel application on one
processor and on many processors. If the compiler is mpicc
and the user hasn't specified values for RUNSERIAL and
RUNPARALLEL then configure chooses `mpirun' from the same
directory as `mpicc':
RUNSERIAL: /usr/local/mpi/bin/mpirun -np 1
RUNPARALLEL: /usr/local/mpi/bin/mpirun -np $${NPROCS:=2}
The `$${NPROCS:=2}' will be substituted with the value of the
NPROCS environment variable at the time `make check' is run
(or the value 2).
./testpar/Makefile.in
Saying `make check' (or `make test') will run the tests on two
processors by default. If you define NPROCS then that many
processors are used instead:
$ NPROCS=4 make check
./configure.in
Fixed (hopefully) bugs with detecting whether __attribute__
and __FUNCTION__ are special keywords for the compiler.
./Makefile.in
Saying `make install' from the top level directory shows
instructions for using shared libraries.
./config/commence.in
./src/Makefile.in
./test/Makefile.in
./testpar/Makefile.in
./tools/Makefile.in
Moved the @top_srcdir@ into the makefiles because it was
expanded too early and had the wrong value.
./INSTALL
Added a warning that if the wrong version of hdf4 tools are
installed then `make check' will fail in the tools directory.
----------------------
./INSTALL
Reorganized and added some additional examples.
./MANIFEST
./aclocal.m4 [NEW]
./configure.in
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
./bin/ltconfig [NEW]
./bin/ltmain.sh [NEW]
Added tests to determine how to compile shared libraries and
how to link programs with them before the libraries are
installed. Also how to install and uninstall shared
libraries. The configure step also prints the names of the
config files it's trying to load for easier debugging.
./bin/config.guess
./bin/config.sub
Replaced with a newer version from GNU. The changes we made
to that file to report `irix6.x' and `FreeBSD' without version
numbers have been incorporated into configure.in instead. In
the future, do not change these two files (see the top of
configure.in instead). By the way, this update was required to
get shared libraries working.
./config/linux [REMOVED]
./config/linux-gnulibc1 [NEW]
./config/linux-gnu [NEW]
./config/alpha-dec [REMOVED]
./config/alpha-dec-osf4.0 [REMOVED]
./config/dec-osf4.x [NEW]
./config/irix5.3 [REMOVED]
./config/irix5.x [NEW]
./config/irix64 [REMOVED]
./config/freebsd
Moved config files around to agree with output from the new
config.guess. The linux file was split into gnu (RedHat),
gnulibc1, and gnulibc2 versions. The alpha-dec file was
removed (I think it was unused) and the alpha-dec-osf4.0 was
changed to dec-osf4.x. The irix5.3 file renamed to irix5.x and
the irix64 file was renamed to irix6.x. The freebsd file was
changed to point to linux-gnulibc1. These changes were tested
on:
Linux 2.0
Linux 2.1
FreeBSD 3.2
Irix 5.3
Irix64 6.2
Irix64 6.4
HP/UX 10.20
OSF1 4.0
./config/alphaev56-dec-osf4.x
./config/irix64
Added warnings similar to linux/freebsd about using compilers
with known bugs.
./config/commence.in
./config/conclude.in
./src/Makefile.in
./test/Makefile.in
./testpar/Makefile.in
./tools/Makefile.in
Added definitions for shared libraries. This has been tested
on the following systems:
Linux 2.0
Linux 2.1
FreeBSD 3.2
Irix 5.3
Irix64 6.2
Irix64 6.4
HP/UX 10.20 static only
OSF1 4.0
If you want to disable use of shared libraries (you probably
do for development purposes since it takes a lot longer to
compile and because you have to run dynamically linked
programs in a special way if the library hasn't been
installed) then add `--disable-shared' to the configure
command line. This is all documented in the INSTALL file.
./bin/release
Temprarily commented out the MANIFEST checking when running
under svf since svf is about to be replaced by a newer
version. This change only affects error checking during the
release process.
./Makefile.dist
./Makefile.in
Added `make check' which does the same thing as `make _test'
since the former is endorsed by the GNU coding style and
people are used to it. The old `make _test' still works too
(and so does `make test' if you use GNU make).
Added a new test to verify the MPIO can support independent
overlapping writes.
testphdf5.c:
Bracketed the MPIO write test with a #ifdef so that it is
tested only when the explicitely turned on.
Platform tested:
O2K.
----------------------
./src/H5.c
./src/H5Smpio.c
./src/H5T.c
./src/H5Tconv.c
./src/H5Tpublic.h
./src/H5detect.c
./src/H5private.h
./src/H5public.h
./test/big.c
The `long long' type isn't quite as prevalent as we had hoped;
there is at least one system that defines `__int64' but not
`long long'. Therefore, I've temporarily changed all
occurrences of `long long' to `long_long' and then #define
that in H5private.h based on the existence of `long long' or
`__int64'. This gets rid of some #ifdef's in other parts of the code.
The semantics of the hdf5 types are:
*int8_t Exactly one byte
*int16_t At least two bytes. We favor 32-bit integers over
16-bit integers if the 16-bit integer is 4-bytes wide
(Cray)
*int32_t At least four bytes.
*int64_t At least eight bytes
long_long The widest integral integer type
The H5Smpio.c contains debugging code which is non-portable.
./tools/h5ls.c
Changed the order native types are detected so we favor the name `int'
over `short' or `long' if two of them are the same.
./config/conclude.in
Added a rule to make test programs depend on the hdf5 library. This
fixes a minor bug where changing H5detect.c and then running `make
test' caused H5Tinit.c to not be recompiled and therefore the test
files are not relinked.
----------------------
./INSTALL
Added instructions about specifying a path for GNU zlib and
HDF4 headers and library.
Added comments for each of the tool names. Added h5toh4 as a
tool name.
./configure.in
./conigure [REGENERATED]
./src/H5config.h.in [REGENERATED]
The config/* scripts get invoked with $CC_BASENAME set the
base name of the compiler in order to make it easier to handle
setting compiler flags for different compilers in a big case
statement. For instance, if $CC has the value
/usr/local/mpi/bin/mpicc -ansi -64
then $CC_BASENAME will be `mpicc'. The $CC_BASENAME is not
set if $CC is not set.
Fixed alignment in `configure --help'.
An include and/or library path can be specified for GNU zlib
if configure can't find it in normal places. The "normal"
means wherever your compiler normally searches, including
search paths you've added through environment variables like
CPPFLAGS and LDFLAGS. The INSTALL file has instructions.
The `-ljpeg' library is detected.
If `ssize_t' is not found then a #define is added to
H5config.h similar to what we already do for `size_t'.
We detect the hdf5 header file `mfhdf.h' and libraries
`-lmfhdf' and `-ldf' and if found define the H5TOH4 and
TESTH5TOH4 Makefile variables. The user can specify an
include and/or library path. The INSTALL file has
instructions.
The `RUNTEST' variable has been split into `RUNSERIAL' and
`RUNPARALLEL' because these are different commands. The
makefile still uses `RUNTEST', which defaults to the RUNSERIAL
value. The new testpar/Makefile.in sets RUNTEST to the
RUNPARALLEL value. The default RUNSERIAL value is empty and
the default RUNPARALLEL value is `mpirun -np 2'. These can
both be overridden in the config/* files. To make the value
the empty string set it like `RUNPARALLEL=none' in the
config/* file.
The new testpar/Makefile is generated from testpar/Makefile.in
./config/commence.in
Added the RUNSERIAL and RUNPARALLEL makefile definitions.
./src/Makefile.in
The `H5detect' program is run with RUNSERIAL, which is empty
on all platforms except intel-osf1 (ASCI/Red)
./config/conclude.in
Removed the `.c.a:' implicit rule -- we don't use it any more.
./config/BlankForm
./config/alpha-dec
./config/alpha-dec-osf4.0
./config/freebsd2.2.7
./config/hpux10.20
./config/hpux9.03
./config/intel-osf1
./config/irix5.3
./config/irix6.2
./config/irix64
./config/linux
./config/powerpc-ibm-aix4.2.1.0
./config/rs6000-ibm-aix4.1.4.0
./config/solaris2.5
All of these files have been updated to hande multiple
compilers. Most of them assume `cc' if CC is not set. The
documentation in `BlankForm' has been updated and
documentation in the other files refer to `BlankForm'.
The intel-osf1 uses LDFLAGS instead of LIBS to specify the extra
library search paths.
./examples/Makefile.in
./test/Makefile.in
./tools/Makefile.in
Simplified by grouping some dependency information together
more succinctly.
The tools Makefile.in has been modified to work with the
`h5toh4' and `testh5toh4' programs. Until Paul checks these
in you may have problems compiling. If so, just create files
h5toh4.c and testh5toh4.c that contain:
#include <stdio.h>
int main(int argc, char *argv[]) {
fprintf(stderr, "%s: not implemented\n", argv[0]);
return 1;
}
./src/H5public.h
Removed definition for `ssize_t' since this is now handled by
configure in H5config.h with a #define.
./MANIFEST
./testpar/Makefile.in [NEW]
Added a makefile for parallel tests. As soon as we get this
working properly we can remove the other four makefiles in
that directory and maintain just one.
New feature
Problem:
Source code must be modified prior to compiling to allow full
filenames to contain preferred user directories. Source must be
recompiled everytime the preferred user directory changes.
Solution:
Instead, allow file prefixes preferred by user to be passed at
runtime using the arguments "-f <prefix>". The "-f" to signal the
next argument is to be a file prefix, and the "<prefix>" to be the
actual prefix used.
Platform tested:
ASCI Red
The purpose of the change is to correct the H5 filenames declared in
testphdf5.c when used on the Intel TFLOP PFS system.
Problem:
The ROMIO library complains of the H5 filenames having no "piofs:",
"pfs:", "ufs:", or "nfs:" prefixes on the filenames. The only type
currently recognized are the "pfs:" prefixes, and the files must
reside in the Intel TFLOP PFS system: /pfs.
Solution: (REQUIRED)
The compile flag -DPOOMA_ARCH=TFLOP is added to the list of CFLAGS in
Makefile.ascired located in the parallel test directory. This compile
flag will allow a special set of filenames to be used when compiling
testphdf5.c on the Intel TFLOP. In addition, instructions to editing
the filenames in testphdf5.c to reside in the user's own /pfs directory
are included in INSTALL.ascired.
Platform tested:
Intel TFLOP
changed to use system supported MPI library. The path to it is
changed slightly too. Updated all the paths variables.
Also updated to reflect the new program source files structure.
Platform tested: Intel Red
big testphdf5.c file. Broke it into modules of related routines.
testphdf5.c -- main routine and global variables initialization plus
some ulitility routines.
t_file.c -- tests of file operations using parallel I/O.
t_dset.c -- tests of datasets operations.
Added routines to fill in test data and to verify data read back.
Would verify data and report errors when detected.
Added verbose flag to control output "volume"--be more verbose
only if it is run with -v option.
