Purpose:
Feature Fix
Description:
Changed so the "pio_perf" module handles creating and destroying the
MPI Comm. Worked it so we get the minimum, maximum, and average times
over a set of iterations.
Solution:
Lots. Had to pull the MPI Comm code from the "pio_engine" module and
place it in the "pio_perf" module. Then worked on a way to have all
processes send their time output to process 0, who collects it and
gives back the min, max, and avg times for the iterations.
Platforms tested:
Linux. Doesn't work if you use more than 1 processor...*hrmph*
Purpose:
Feature Rollback
Description:
Shared C++ libraries aren't working on all platforms (read: Arabica).
We need to switch to the old method until we can sort this out.
Solution:
Re-add the "-static" flag to the library creation lines.
Purpose:
Bug Fix
Description:
There were weird "throw()"s which were being placed into the
configure script. Some C++ compilers were barfing on this syntax
(which was akin to "extern int exit(int) throw ();" wonder why?).
Solution:
Ran a different autoconf on the configure file which I'd downloaded
from ftp.gnu.org. It doesn't have those "throw()"s anymore
Platforms tested:
Linux
Purpose:
Bug Fix
Description:
The --raw, --hdf5, --mpiio options weren't being done correctly.
Solution:
I had to change some of the tests for the io_type to & instead of |
so that they'd work how I wanted them to work.
Platforms tested:
Linux
Bug Fix
Description:
Added regression test for proper library behavior when adding fields past
the end of a datatype.
Platforms Tested:
FreeBSD 4.4 (sleipnir)
Bug Fix
Description:
H5Tinsert was allowing compound datatype fields to be inserted past the end
of the datatype.
Solution:
Added range check in H5T_insert
Platforms Tested:
FreeBSD 4.4 (sleipnir)
Purpose:
Bug Fix
Description:
Shared libraries weren't being built for C++.
Solution:
Removed the -static flag from the libtool compile line. This allows
both shared and static libraries to be built instead of just static
ones. After talking with Quincey and Binh-Minh, there shouldn't be
any reason for C++ to be built static only.
Platforms tested:
Linux
Purpose:
Feature Changes
Description:
Okay, I needed to add in more parameters so that the user can modify
how things are supposed to work with the PIO programs. Also needed
to change the algorithm a bit to make these work. And needed to add
in timing for the READ option.
Solution:
Added the above things. The parameters took a major rewrite of the
command-line parsing stuff. Here's the usage statement:
usage: pio_perf [OPTIONS]
OPTIONS
-h, --help Print a usage message and exit
-d N, --num-dsets=N Number of datasets per file [default:1]
-f S, --file-size=S Size of a single file [default: 64M]
-F N, --num-files=N Number of files [default: 1]
-H, --hdf5 Run HDF5 performance test
-i, --num-iterations Number of iterations to perform [default: 1]
-m, --mpiio Run MPI/IO performance test
-o F, --output=F Output raw data into file F [default: none]
-P N, --max-num-processes=N Maximum number of processes to use [default: 1]
-p N, --min-num-processes=N Minimum number of processes to use [default: 1]
-r, --raw Run raw (UNIX) performance test
-X S, --max-xfer-size=S Maximum transfer buffer size [default: 1M]
-x S, --min-xfer-size=S Minimum transfer buffer size [default: 1K]
F - is a filename.
N - is an integer >=0.
S - is a size specifier, an integer >=0 followed by a size indicator:
K - Kilobyte
M - Megabyte
G - Gigabyte
Example: 37M = 37 Megabytes
Platforms tested:
Linux, but not fully finished...
Purpose:
Bug Fix
Description:
Committing the changes in the MPI/IO stuff so that parallel HDF5
will work on HP-UX. It seems that on HP-UX, the MPI_Status variable
needs to be initialized to 0 for it to work (i.e., in some other MPI
calls, if there's garbage in the MPI_Status variable, then it will
barf).
Solution:
Initialized to 0.
Platforms tested:
HP-UX parallel, Linux.
Added nfiles and ndsets parameters checking.
Removed iteration variable since iterations would be done
in Control routine.
Platforms tested:
Eirene(pp) and modi4(pp)
Purpose:
Feature Fix
Description:
The new feature (allowing the user to specify =DIR for a --with-xxx
option instead of requiring =INC,LIB for them) needed a better
explanation.
Solution:
Created a phoney --with-fnord option whose only purpose is to supply
a --help message.
Platforms tested:
Linux
Purpose:
Feature Add
Description:
Added the feature which will accept a directory for all of the
features which now accept an =INC,LIB directive. The old way of doing
things (=INC,LIB) is still valid, but we prefer the user uses the
=DIR method from now on. It saves typing.
Platforms tested:
Linux
Bug fix (or more like feature)
Description:
MPI_File_open does not truncate the filesize if file already exists.
This created confusion during debugging as what the real file size
is. It also interfere the real write bandwidth since the times
required to allocate new disk-space vanishes for subsequent writes
that are for offset shorter than previous file sizes.
Added a MPI_File_set_size to reset the file size to 0 for every new
file.
Another bug is that the 'remove()' call may not work for MPIO/PHDF5
files. (e.g., filename may have some MPI prefix like "pfs:filename").
Replaced "remove" with MPI_File_delete for those cases.
Platforms tested:
modi4(pp) and eirene (pp)
Purpose:
parameter file
Description:
handle parameters in various special cases.
Users donot even have to use this file if they don't have those special needs.
1) memory optimization (should always be set to 0 without handling huge SDS array conversion)
2) unlimited dimension with zero current size, users can use this file to define
their chunk size.
Solution:
Platforms tested:
RedHat Zoot 6.2
Purpose:
a bug fix
a feature added
Description:
1.conversion of unlimited dimension data with the current dimensional size 0
2. Use a parameter file to control some special cases:
1) To subdivide the huge array into hyperslabs, a memory buffer size has i
to be set.
2) when current dimensional size is 0, a default chunk size can be set.
Solution:
bug fixed 1. Old approach: the current dimensional size is set to H5S_UNLIMITED, which is
a huge number. The default chunk size is set as a *FIXED* default value.
2. New approach: the current dimensional size is set to the current value 0.
Users can provide the chunk size through a parameter file.
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
bug fix, new feature
Description:
Added HDF5 write in do_write.
Added a complete do_read.
Still need timing code for the read part.
Platforms tested:
eirene (pp), modi4(pp).
Did not test serial since all changes were done in parallel area.
Purpose:
Removing warnings
Description:
H5IdComponent::operator= shouldn't be virtual because the subclass'
operator= have different type for the rhs argument. Removed virtual.
Platforms tested:
SunOS 5.7 (arabica)
Linux 6.2 (eirene)
Purpose:
DOH!
Description:
Ugh! I forgot to add the -m flag to the list of "short" parameter
codes. It wasn't even looking for them. *sigh*
Solution:
Added it
Platforms tested:
Linux
Bug fixes, new features
Description:
There was a coding error in handling file open flags. Changed it
to use &.
Added do_cleanup to cleanup temporary test files but only if
$HDF5_NOCLEANUP is not set. This is consistent with other test programs.
Added logic so that each process is writing its own slabs of data only.
Moved the number of process, rank of process and the communicator used
for PIO run to be global variables. Makes the coding easier. (but this
is not thread-safe.)
Platforms tested:
modi4(pp) and eirene(pp).
Purpose:
Bug fixing
Description:
H5IdComponent::operator= shouldn't be virtual because the subclass'
operator= have different type for the rhs argument. Removed virtual
Platforms tested:
SunOS 5.7 (arabica)
IRIX64 (modi4) - only tested that the warnings went away, still working
on other configuration problems.
Code cleanup
Description:
Tweaked internal error handling macros to reduce the size of the library's
object code by about 10-20%.
Also cleaned up some compiler warnings...
Platforms tested:
FreeBSD 4.4 (sleipnir)
Purpose:
Feature Fix
Description:
Changed default size of file to 512MB. The "-m" flag is now in
megabytes as well. This makes running things a bit faster.
Platforms tested:
Linux
Purpose:
bug fixed
Description:
HDF4 Vdata is extensible, when converted, the new HDF4 data set should also be extensible.
Solution:
Make the HDF4 dataset converted from Vdata unlimited dimensions.
These three testing files should be updated accordingly.
Platforms tested:
Linux, RedHat 6.2
Purpose:
Bug Fix
Description:
Fixed so that it will display the correct timing data. It will also
write to the correct file (which it wasn't before).
Solution:
Put the code in for displaying the time. Had to change the way I was
passing an object to the pio_fopen() function from just being a
structure to being a pointer so that the changes could be propagated
back.
Platforms tested:
Linux
Purpose:
Bug Fix
Description:
I wasn't calculating the total time correctly.
Solution:
I had to subtract the previous time from the current time. This
wasn't being done...DOH
Platforms tested:
Linux
Purpose:
Modify H5Fclose behavior
Description:
The HDF5 actual file close behaves in several ways in terms of if there
are still objects(dataset, group, datatype) opened in file.
Solution:
Added a new file access property, file close degree. It has four values,
H5F_CLOSE_DEFAULT
H5F_CLOSE_WEAK
H5F_CLOSE_SEMI
H5F_CLOSE_STRONG
The way a file is closed is decided by these values.
Platforms tested:
IRIX64 6.5, SunOS 5.6, FreeBSD 4.4
Bug Fix
Description:
The file metadata macros generate unaligned access warnings on the IA64
architecture.
Solution:
Got rid of bogus big-endian vs. little-endian differentiation when encoding
and decoding file metadata and use proper set of macros to prevent unaligned
access problems.
This fixes bug #672.
Platforms tested:
FreeBSD 4.4 (sleipnir)
Bug Fix
Description:
The code in H5Sselect_hyperslab_valid contained an fencepost error and is
allowing selections which overlap the extent by exactly one element in any
dimension to pass as valid instead of flagging the selection as invalid.
This bug only affects hyperslabs which have been OR'ed together, not the
selection from a single H5Sselect_hyperslab.
This fixes bug #550.
Solution:
Changed an '>' to an '>=' and added new regression test to check for error.
Platforms tested:
FreeBSD 4.4 (sleipnir)
Purpose:
Small Fixes
Description:
After conversation with Albert, here are some small fixes for the
performance stuff. Not too significant. Though, we did add the
"buffer size" as a parameter I pass to the engine.
Purpose:
Feature Fix
Description:
Added code so that it measures the time it takes to do I/O and return
that to the calling function.
This code doesn't quite work yet. There is something wrong with the
MPI code in the "pio_engine.c" file...I don't know what's up with
it...
Platforms tested:
Linux
Purpose:
Bug Fix
Description & Solution:
In the test for "namespace", we needed to include the "iostream"
header file to remove errors for strict compilers who need iostream
for namespaces. I moved the test for namespace to after the test for
"old header filenames" to make sure that I'm #including the correct
iostream ("iostream" or "iostream.h").
Platforms tested:
Linux
Bug fix
Description:
Cleaned up obvious syntax errors to make it to compile.
Changed unnecessarily unsigned variables to signed to avoid
messy compiler warnings.
Platforms tested:
eirene (parallel)
Bug fix.
Description:
Fix fortran stub for H5P[gs]et_hyper_cache.
Solution:
Made the fortran subs not execute any function calls into the library,
unless v1.4 compatibility is turned on.
Elena will be following this up with a real fix.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Bug fix
Description:
A a build had occurred in the source tree ($CURRENT) and did not
do a distclean, the left over files would interfere with srcdir
build.
Solution:
Added a "make distclean" in the "checkout" option to clean up
the $CURRENT just in case. Not necessarily the most optimal
spot to do the cleaning but convenient.
Platforms tested:
Eirene.
Code cleanup
Description:
Property that H5P(g|s)et_hyper_cache uses is no longer in use inside the
library.
Solution:
Removed H5P(g|s)et_hyper_cache API functions, except when backward
compatibility is turned on. When backward compatibility is turned on,
the property is defined by the library, but unused internally.
Platforms tested:
FreeBSD 4.4 (sleipnir)
