Code cleanup
Description:
Platform dependent code related to the struct stat and fstat
calls polluted source codes. Hard to maintain.
Solution:
Platform dependent code are moved to H5private.h and then internal
code can #include H5private.h. Repeat those macro definition for
the stdio and multi drivers since they area examples for writing
a virtual file driver. They must not use any internal code.
Platforms tested:
eirene (parallel), modi4 (serial including gass driver.)
New test feature
Description:
Added create_faccess_plist() that create just MPIO or split+MPIO
file-access property list. This in turn can run parallel tests
with just MPIO or with Split-file VFD too.
Added -s option for split-file Plus MPIO tests.
For testphdf5.c: removed a bunch of old debug code that got left
in by mistake.
Platforms tested:
Modi4 and eirene parallel.
But it has uncovered errors in the library. The test program
is correct though. Checking the test program in so that it won't
get lost and can be used for debugging. Also, the -s is not used
by default during test. At least it won't abort "make check".
Multiple-group testing
Description:
Added multiple groups under root group and multiple subgroups of
certain levels. Also writes several datasets in each group in
parallel.
Solution:
Platforms tested:
MPICH(IRIX 6.5, IRIX64 6.5(64), IRIX64 6.5(N32), Linux, SunOS 5.6)
and SGI MPI(IRIX64 6.5).
Bug Fix
Description:
FILENAME_MAX is short on some systems. It's better to use PATH_MAX
which tends to be longer (we hope).
Solution:
Removed FILENAME_MAX and replaced it with PATH_MAX instead.
Platforms tested:
Skinner (SDSC HP N9000).
New features
Description:
Some testers found the filename lengths too short.
Changed it to use the FILENAME_MAX usually defined in stdio.h.
If not, set it to 512 which should be sufficient for users
but should not exceed any system limits.
Also added a new test parameters of ndatasets so that the tester
can specific a different number of datasets for the multiple
datasets tests.
Changed the datatype of datasets created to DOUBLE. This eliminates
the current racing conditions. But the racing bugs during conversion
still need to be tracked down and squashed.
Platforms tested:
Modi4 -64.
Feature, kind of
Description:
Moved the multiple dataset test out of the write-test
part so that it can be skipped or tested independent of
the other tests.
Platforms tested:
IRIX64,-64,parallel.
Features, kind of.
Description:
Separated the MPI features test into its own independent
program so that it can be tested on its own without too
much HDF5 stuff involved.
Added automatic removal of temporary test files after
the tests completed.
Reduced the size of the dataset dimensions to avoid tripping
the SGI MPI problems of running out of internal mpi type entries.
Platforms tested:
O2K -64
Added features
Description:
There were no automatic tests for transfering zero elements.
Solution:
t_dset.c:
Added two new patterns of ZROW (zero rows for process 0)
and ZCOL(zero columns for process 0).
ZROW test was added but it failed because the current library
does not accept it. Not compiled in now. Need to fix the
library before turning it back on again and also to add the
ZCOL test.
t_mdset.c:
Added statement to show progress. Also the MPI_Barrier() call
get processes synchornoized. It eliminates the racing condition
but this is not a permenant solution. The library code needs to
be fixed.
testphdf5.c:
Added a bunch of MPI_Type_XXX debug code. Added the -md
option to skip the multiple datasets tests. Changed the cosmitic
appearance of the banner messages.
testphdf5.h:
When an error is detected, the old way was to call MPI_Finalize()
before exiting. This sometimes hangs because some processes
may be waiting for a message of a different tag. Changed to
call MPI_Abort() for now so that the whole MPI job would
abort rather than hanging due resource limits exceeded.
Added the definition of ZROW and ZCOL.
Platforms tested:
Modi4 -64.
Makefile.in:
Added test/ as one of the -I directories to search for header files.
Needed because <h5test.h> is used.
t_file.c t_mpi.c testphdf5.c testphdf5.h:
Added FILENAME to meet the assumption in h5test.h. (May use
CLEANUP in the future.) Moved the prefix setting to the
h5_fixname().
Increase the test size to 32. Put in a check to make sure number of
processes are not bigger than SIZE.
testphdf5.c:
Fixed a mistake in the prototype of pause_proc to reflect no arguments.
Changed it to skip the test instead of aborting when there is not
enough processes to do the test. Also corrected an error in the
error reporting printf statement.
t_dset.c:
testphdf5.c:
testphdf5.h:
Added option for specifying chunk dimensions.
new feature
Makefile.in:
Added the dependence of *.c on the testphdf5.h
t_dset.c:
testphdf5.c:
testphdf5.h:
testphdf5 now takes optional arguements for dataset dimension sizes.
That allows testing with different dimension sizes without recompiling
the whole thing.
Platform tested:
O2K
Added a new test to verify the MPIO can support independent
overlapping writes.
testphdf5.c:
Bracketed the MPIO write test with a #ifdef so that it is
tested only when the explicitely turned on.
Platform tested:
O2K.
New feature
Problem:
Source code must be modified prior to compiling to allow full
filenames to contain preferred user directories. Source must be
recompiled everytime the preferred user directory changes.
Solution:
Instead, allow file prefixes preferred by user to be passed at
runtime using the arguments "-f <prefix>". The "-f" to signal the
next argument is to be a file prefix, and the "<prefix>" to be the
actual prefix used.
Platform tested:
ASCI Red
The purpose of the change is to correct the H5 filenames declared in
testphdf5.c when used on the Intel TFLOP PFS system.
Problem:
The ROMIO library complains of the H5 filenames having no "piofs:",
"pfs:", "ufs:", or "nfs:" prefixes on the filenames. The only type
currently recognized are the "pfs:" prefixes, and the files must
reside in the Intel TFLOP PFS system: /pfs.
Solution: (REQUIRED)
The compile flag -DPOOMA_ARCH=TFLOP is added to the list of CFLAGS in
Makefile.ascired located in the parallel test directory. This compile
flag will allow a special set of filenames to be used when compiling
testphdf5.c on the Intel TFLOP. In addition, instructions to editing
the filenames in testphdf5.c to reside in the user's own /pfs directory
are included in INSTALL.ascired.
Platform tested:
Intel TFLOP
big testphdf5.c file. Broke it into modules of related routines.
testphdf5.c -- main routine and global variables initialization plus
some ulitility routines.
t_file.c -- tests of file operations using parallel I/O.
t_dset.c -- tests of datasets operations.
Added routines to fill in test data and to verify data read back.
Would verify data and report errors when detected.
Added verbose flag to control output "volume"--be more verbose
only if it is run with -v option.
----------------------
./configure.in
./src/H5Fprivate.h
./src/H5Fsec2.c
We now detect and use lseek64() on systems that have it (e.g.,
Irix64) and are able to generate >2GB files on Irix XFS file
systems (and anything else that supports large files). This
change also removed some warning messages from the Irix `-64'
compiler.
> $ ls -l istore.h5
> -rw-r--r-- 1 matzke meshtv 8605436856 Feb 17 14:03 istore.h5
./configure.in
./src/H5Fprivate.h
./src/H5Fstdio.h
We now detect and use fseek64() on systems that have it (e.g.,
Irix64) and are able to generate >2GB files on Irix XFS file
systems (and anything else that supports large files). This
change also removed some warning messages from the Iris `-64'
compiler.
./src/H5E.c
./src/H5Epublic.h
Added the H5E_OVERFLOW error to signal file address overflow.
./src/H5Fpublic.h
./examples/h5_chunk_read.c
./examples/h5_compound.c
./examples/h5_extend_write.c
./examples/h5_group.c
./examples/h5_read.c
./examples/h5_write.c
./html/Datasets.html
./html/Files.html
./html/H5.api.html
./html/H5.intro.html
./html/H5.sample_code.html
./html/ph5example.c
./html/review1.html
./test/cmpd_dset.c
./test/dsets.c
./test/extend.c
./test/tfile.c
./test/th5p.c
./test/theap.c
./test/tohdr.c
./test/tstab.c
./testpar/phdf5.c
Renamed file access constants to follow the naming scheme.
Also changed the base names a little to be more accurate as to
what they do. The old names H5ACC_WRITE and H5ACC_OVERWRITE
will temporarily work.
H5ACC_DEFAULT --> H5F_ACC_RDONLY for H5Fopen()
H5ACC_DEFAULT --> H5F_ACC_EXCL for H5Fcreate()
H5ACC_WRITE --> H5F_ACC_RDWR
H5ACC_OVERWRITE --> H5F_ACC_TRUNC
Albert or Kim: The H5ACC_INDEPENDENT and H5ACC_COLLECTIVE
macros in H5Fpublic.h should be an enum typedef and have names
more like H5F_MPIO_INDEPENDENT and H5F_MPIO_COLLECTIVE. Also
change the access_mode argument of H5Cset_mpio().
H5Fcreate() and H5Fopen() are more strict now about which
flags are acceptable for the operation.
./src/H5Fprivate.h
./src/H5F.c
./src/H5C.c
Changed the file access template to make it more general. A
union contains a struct for each type of low-level driver and
the default template is initialized at run-time.
./src/H5Fpublic.h
./src/H5F.c
Added H5Fget_access_template() and cleaned up
H5Fget_create_template().
./src/H5C.c
The H5Cset_mpi() no longer trashes the file access template
when an error is detected. We check for errors and *then*
update the file access template.
Added H5C_close() so Albert and Kim have a place to release
the MPI communicator and info from the file access property
list.
Kim or Albert: I notice in H5Cset_mpi() you copy the MPI
communicator. Do you need to do something similar in
H5C_copy()?
./src/H5F.c
Added more error checking for the file creation and access
property lists because it used to be possible to make the
library dump core by swapping the creation and access property
list ID numbers of H5Fcreate().
./test/istore.c
./test/tfile.c
./test/th5p.c
./test/theap.c
./test/tohdr.c
./test/tstab.c
./testpar/testphdf5.c
One must pass H5C_DEFAULT as the file creation or access
property list in order to get the default property list. It
is no longer possible to pass zero or any other arbitrary bad
object ID.
./src/H5Fcore.c
./src/H5Ffamly.c
./src/H5Flow.c
./src/H5Fmpio.c
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5MF.c
The file access property list is passed to all H5F_low_...()
functions and to the drivers.
./src/H5Fcore.c
The block size can be set at run time on a per-file basis
instead of at compile time across all files. The "5000 items in
a group test" now takes 1.6 seconds.
./src/H5private.h
Removed inclusion of mpi.h and mpio.h since they're included
from H5public.h.
./src/H5Cpublic.h
./src/H5C.c
Added H5Cset_stdio(), H5Cset_sec2(), H5Cset_core(),
H5Cset_split(), and H5Cset_family() in addition to the
H5Cset_mpio() that Kim and Albert already wrote. We still
need the H5Cget_driver() and an H5Cget...() counterpart for
each of those functions. The split and family drivers still
need a little work but I'm checking this in anyway.
