Added a test of fill value before any data is written to a dataset.
Rename short_dataset() as dataset_fillvalue() as it reflects better
the tests. Also removed the option of -S since the fill value test
will be tested always.
Platforms tested:
"h5committested"
Misc. update:
Feature.
Description:
Added the short_dataset test (was in v1.6 first.)
Platforms tested:
Tested in eirene (pp) only since these have been tested in v1.6 already.
Misc. update:
Feature, sort of.
Description:
Reactivated the big dataset test. Changed it
to default off, can be turned on via -b.
Platforms tested:
Only in Copper which is the only local machine safe to run
mulitple GB size files.
Misc. update:
Disable "big" parallel tests until I get a chance to look into the
problems (probably during/after the workshop).
Platforms tested:
Eyeballed
Too minor to require h5committest
Bug fix
Description:
The MPI_File_set_size() routine on ASCI Red is not able to extend files
so that they are larger than 2GB.
Solution:
Add an extra macro which controls whether MPI_File_set_size() can handle
>2GB offsets or if our "older" way of reading a byte, then writing a byte at
the appropriate offset should be used.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Compatibility fix
Description:
The H5P[set|get]_fapl_mpiposix calls changed between v1.4.x and v1.5.x.
Solution:
Wrap them in the v1.4 backward compatibility #ifdefs and update tests, etc.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/paralle & v1.4 compatibility
h5committest pointless
Add printout to identify when the MPIPOSIX driver is being tested instead
of the MPI-IO driver..
Platforms tested:
Tested in copper in parallel mode only since it was just a simple
print statement in the parallel test code.
Misc. update:
Feature Add
Description:
Added knob so that the programmer can enable or disable GPFS
hints during runtime instead of having it only enabled at
configure/compile time. Some of the public APIs were changed
to add an extra parameter for this option...
Platforms tested:
Blue (LLNL). It only affects the MPI/POSIX driver, so no need
to test it on non-GPFS platforms.
Misc. update:
feature
Description:
added test_fapl_mpiposix_dup to verify the MPIPOSIX is making
duplicates of Communicator correctly.
Platforms tested:
Did not h5committested because these are all MPI code and well protected
by HAVE_PARALLEL.
Tested in eirene and modi4, both parallel mode.
Misc. update:
code cleanup.
Description:
Rename test_comm_info_delete to test_fapl_mpio_dup to better decribe
what it does. Updated the comments to describe more accurate.
Platforms tested:
Did not run h5commmittest since the changes were all in parallel area.
Ran parallel tests on modi4 and eirene.
Misc. update:
Bug fixes/API changes
Description:
Previously, the Communicator and Info object arguments supplied
to H5Pset_fapl_mpio() are stored in the property with its handle
values. This meant changes to the communicator or the Info object
after calling H5Pset_fapl_mpio would affect the how the property
list function. This was also the case when H5Fopen/create operated.
They just stored the handle value. This is not according to the
MPI-2 defined behavior of how Info objects should be handled. (MPI-2
defines Info objects must be parsed when called.)
The old design was trying to avoid numerous duplicates of the same
information (e.g., every property object holds one version, every
file opened holds another version, when all of them are referring to
the same original version.) Nevertheless it is safer to implement
it according to MPI-2 definition.
Futhermore, the library often needs to do message passing using the
supplied communicator. Using the same communicator as the application
version may result in some messages mix up.
Solution:
H5Pset_fapl_mpio now stores a duplicate of each of the communicator
and Info object.
H5Pget_fapl_mpio returns a duplicate of its stored communicator and
Info object. It is now the responsibility of the applications to free
those objects when done.
H5Fopen/create also stores a duplicate of the communicator and Info
object supplied by the File Access Property list.
H5Fclose frees those duplicates.
There are a few more internal VFL call back functions that they
follow this "make duplicates" requirement.
Platforms tested:
"h5committested".
What other platforms/configurations were tested?
Eirene (mpicc), sol(mpicc), copper(parallel)
Misc. update:
Need to update MANIFEST for the added t_ph5basic.c which tests the
correctness of duplicated communicator and INFO object.
Update
Description:
Updated (and in some cases added) the copyright statement.
Platforms tested:
Linux (Comment changes...only tested if they compile)
Misc. update:
Purpose:
More test.
Description:
Test independent read of groups and chunked dataset.
Solution:
This test is similar to multiple group test. So just add it in the
testphdf5.c,h.
Platforms tested:
modi4, eirene.
Bug fix (no id, just noticed myself)
Description:
Added a barrier before doing file cleanup to ensure all tests are
completed by then.
Moved the MPI_Finalize to be very end and insert an H5close before
it to ensure all potential MPI activities by HDF5 are done by then.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yes.
Code cleanup
Description:
Cleaned up some compiler warnings.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/serial & parallel. Will be testing on IRIX64
6.5 (modi4) in serial & parallel shortly.
Purpose:
Design for compact dataset
Description:
Compact dataset is stored in the header message for dataset layout.
Platforms tested:
arabica, eirene.
Code cleanup
Description:
Platform dependent code related to the struct stat and fstat
calls polluted source codes. Hard to maintain.
Solution:
Platform dependent code are moved to H5private.h and then internal
code can #include H5private.h. Repeat those macro definition for
the stdio and multi drivers since they area examples for writing
a virtual file driver. They must not use any internal code.
Platforms tested:
eirene (parallel), modi4 (serial including gass driver.)
New test feature
Description:
Added create_faccess_plist() that create just MPIO or split+MPIO
file-access property list. This in turn can run parallel tests
with just MPIO or with Split-file VFD too.
Added -s option for split-file Plus MPIO tests.
For testphdf5.c: removed a bunch of old debug code that got left
in by mistake.
Platforms tested:
Modi4 and eirene parallel.
But it has uncovered errors in the library. The test program
is correct though. Checking the test program in so that it won't
get lost and can be used for debugging. Also, the -s is not used
by default during test. At least it won't abort "make check".
Multiple-group testing
Description:
Added multiple groups under root group and multiple subgroups of
certain levels. Also writes several datasets in each group in
parallel.
Solution:
Platforms tested:
MPICH(IRIX 6.5, IRIX64 6.5(64), IRIX64 6.5(N32), Linux, SunOS 5.6)
and SGI MPI(IRIX64 6.5).
Bug Fix
Description:
FILENAME_MAX is short on some systems. It's better to use PATH_MAX
which tends to be longer (we hope).
Solution:
Removed FILENAME_MAX and replaced it with PATH_MAX instead.
Platforms tested:
Skinner (SDSC HP N9000).
New features
Description:
Some testers found the filename lengths too short.
Changed it to use the FILENAME_MAX usually defined in stdio.h.
If not, set it to 512 which should be sufficient for users
but should not exceed any system limits.
Also added a new test parameters of ndatasets so that the tester
can specific a different number of datasets for the multiple
datasets tests.
Changed the datatype of datasets created to DOUBLE. This eliminates
the current racing conditions. But the racing bugs during conversion
still need to be tracked down and squashed.
Platforms tested:
Modi4 -64.
Feature, kind of
Description:
Moved the multiple dataset test out of the write-test
part so that it can be skipped or tested independent of
the other tests.
Platforms tested:
IRIX64,-64,parallel.
Features, kind of.
Description:
Separated the MPI features test into its own independent
program so that it can be tested on its own without too
much HDF5 stuff involved.
Added automatic removal of temporary test files after
the tests completed.
Reduced the size of the dataset dimensions to avoid tripping
the SGI MPI problems of running out of internal mpi type entries.
Platforms tested:
O2K -64
Added features
Description:
There were no automatic tests for transfering zero elements.
Solution:
t_dset.c:
Added two new patterns of ZROW (zero rows for process 0)
and ZCOL(zero columns for process 0).
ZROW test was added but it failed because the current library
does not accept it. Not compiled in now. Need to fix the
library before turning it back on again and also to add the
ZCOL test.
t_mdset.c:
Added statement to show progress. Also the MPI_Barrier() call
get processes synchornoized. It eliminates the racing condition
but this is not a permenant solution. The library code needs to
be fixed.
testphdf5.c:
Added a bunch of MPI_Type_XXX debug code. Added the -md
option to skip the multiple datasets tests. Changed the cosmitic
appearance of the banner messages.
testphdf5.h:
When an error is detected, the old way was to call MPI_Finalize()
before exiting. This sometimes hangs because some processes
may be waiting for a message of a different tag. Changed to
call MPI_Abort() for now so that the whole MPI job would
abort rather than hanging due resource limits exceeded.
Added the definition of ZROW and ZCOL.
Platforms tested:
Modi4 -64.
Makefile.in:
Added test/ as one of the -I directories to search for header files.
Needed because <h5test.h> is used.
t_file.c t_mpi.c testphdf5.c testphdf5.h:
Added FILENAME to meet the assumption in h5test.h. (May use
CLEANUP in the future.) Moved the prefix setting to the
h5_fixname().
Increase the test size to 32. Put in a check to make sure number of
processes are not bigger than SIZE.
testphdf5.c:
Fixed a mistake in the prototype of pause_proc to reflect no arguments.
Changed it to skip the test instead of aborting when there is not
enough processes to do the test. Also corrected an error in the
error reporting printf statement.
t_dset.c:
testphdf5.c:
testphdf5.h:
Added option for specifying chunk dimensions.
new feature
Makefile.in:
Added the dependence of *.c on the testphdf5.h
t_dset.c:
testphdf5.c:
testphdf5.h:
testphdf5 now takes optional arguements for dataset dimension sizes.
That allows testing with different dimension sizes without recompiling
the whole thing.
Platform tested:
O2K
Added a new test to verify the MPIO can support independent
overlapping writes.
testphdf5.c:
Bracketed the MPIO write test with a #ifdef so that it is
tested only when the explicitely turned on.
Platform tested:
O2K.
New feature
Problem:
Source code must be modified prior to compiling to allow full
filenames to contain preferred user directories. Source must be
recompiled everytime the preferred user directory changes.
Solution:
Instead, allow file prefixes preferred by user to be passed at
runtime using the arguments "-f <prefix>". The "-f" to signal the
next argument is to be a file prefix, and the "<prefix>" to be the
actual prefix used.
Platform tested:
ASCI Red
The purpose of the change is to correct the H5 filenames declared in
testphdf5.c when used on the Intel TFLOP PFS system.
Problem:
The ROMIO library complains of the H5 filenames having no "piofs:",
"pfs:", "ufs:", or "nfs:" prefixes on the filenames. The only type
currently recognized are the "pfs:" prefixes, and the files must
reside in the Intel TFLOP PFS system: /pfs.
Solution: (REQUIRED)
The compile flag -DPOOMA_ARCH=TFLOP is added to the list of CFLAGS in
Makefile.ascired located in the parallel test directory. This compile
flag will allow a special set of filenames to be used when compiling
testphdf5.c on the Intel TFLOP. In addition, instructions to editing
the filenames in testphdf5.c to reside in the user's own /pfs directory
are included in INSTALL.ascired.
Platform tested:
Intel TFLOP
big testphdf5.c file. Broke it into modules of related routines.
testphdf5.c -- main routine and global variables initialization plus
some ulitility routines.
t_file.c -- tests of file operations using parallel I/O.
t_dset.c -- tests of datasets operations.
Added routines to fill in test data and to verify data read back.
Would verify data and report errors when detected.
Added verbose flag to control output "volume"--be more verbose
only if it is run with -v option.
----------------------
./configure.in
./src/H5Fprivate.h
./src/H5Fsec2.c
We now detect and use lseek64() on systems that have it (e.g.,
Irix64) and are able to generate >2GB files on Irix XFS file
systems (and anything else that supports large files). This
change also removed some warning messages from the Irix `-64'
compiler.
> $ ls -l istore.h5
> -rw-r--r-- 1 matzke meshtv 8605436856 Feb 17 14:03 istore.h5
./configure.in
./src/H5Fprivate.h
./src/H5Fstdio.h
We now detect and use fseek64() on systems that have it (e.g.,
Irix64) and are able to generate >2GB files on Irix XFS file
systems (and anything else that supports large files). This
change also removed some warning messages from the Iris `-64'
compiler.
./src/H5E.c
./src/H5Epublic.h
Added the H5E_OVERFLOW error to signal file address overflow.
./src/H5Fpublic.h
./examples/h5_chunk_read.c
./examples/h5_compound.c
./examples/h5_extend_write.c
./examples/h5_group.c
./examples/h5_read.c
./examples/h5_write.c
./html/Datasets.html
./html/Files.html
./html/H5.api.html
./html/H5.intro.html
./html/H5.sample_code.html
./html/ph5example.c
./html/review1.html
./test/cmpd_dset.c
./test/dsets.c
./test/extend.c
./test/tfile.c
./test/th5p.c
./test/theap.c
./test/tohdr.c
./test/tstab.c
./testpar/phdf5.c
Renamed file access constants to follow the naming scheme.
Also changed the base names a little to be more accurate as to
what they do. The old names H5ACC_WRITE and H5ACC_OVERWRITE
will temporarily work.
H5ACC_DEFAULT --> H5F_ACC_RDONLY for H5Fopen()
H5ACC_DEFAULT --> H5F_ACC_EXCL for H5Fcreate()
H5ACC_WRITE --> H5F_ACC_RDWR
H5ACC_OVERWRITE --> H5F_ACC_TRUNC
Albert or Kim: The H5ACC_INDEPENDENT and H5ACC_COLLECTIVE
macros in H5Fpublic.h should be an enum typedef and have names
more like H5F_MPIO_INDEPENDENT and H5F_MPIO_COLLECTIVE. Also
change the access_mode argument of H5Cset_mpio().
H5Fcreate() and H5Fopen() are more strict now about which
flags are acceptable for the operation.
./src/H5Fprivate.h
./src/H5F.c
./src/H5C.c
Changed the file access template to make it more general. A
union contains a struct for each type of low-level driver and
the default template is initialized at run-time.
./src/H5Fpublic.h
./src/H5F.c
Added H5Fget_access_template() and cleaned up
H5Fget_create_template().
./src/H5C.c
The H5Cset_mpi() no longer trashes the file access template
when an error is detected. We check for errors and *then*
update the file access template.
Added H5C_close() so Albert and Kim have a place to release
the MPI communicator and info from the file access property
list.
Kim or Albert: I notice in H5Cset_mpi() you copy the MPI
communicator. Do you need to do something similar in
H5C_copy()?
./src/H5F.c
Added more error checking for the file creation and access
property lists because it used to be possible to make the
library dump core by swapping the creation and access property
list ID numbers of H5Fcreate().
./test/istore.c
./test/tfile.c
./test/th5p.c
./test/theap.c
./test/tohdr.c
./test/tstab.c
./testpar/testphdf5.c
One must pass H5C_DEFAULT as the file creation or access
property list in order to get the default property list. It
is no longer possible to pass zero or any other arbitrary bad
object ID.
./src/H5Fcore.c
./src/H5Ffamly.c
./src/H5Flow.c
./src/H5Fmpio.c
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5MF.c
The file access property list is passed to all H5F_low_...()
functions and to the drivers.
./src/H5Fcore.c
The block size can be set at run time on a per-file basis
instead of at compile time across all files. The "5000 items in
a group test" now takes 1.6 seconds.
./src/H5private.h
Removed inclusion of mpi.h and mpio.h since they're included
from H5public.h.
./src/H5Cpublic.h
./src/H5C.c
Added H5Cset_stdio(), H5Cset_sec2(), H5Cset_core(),
H5Cset_split(), and H5Cset_family() in addition to the
H5Cset_mpio() that Kim and Albert already wrote. We still
need the H5Cget_driver() and an H5Cget...() counterpart for
each of those functions. The split and family drivers still
need a little work but I'm checking this in anyway.
