Bug fix
Description:
The symbol "RAW" is already defined in some ioctl.h files.
Changed it to RAWIO.
Platforms tested:
Tflops.
(I am commiting this from TFLOPS. Will check it out immediate
to test on local machines.)
Code cleanup
Description:
Platform dependent code related to the struct stat and fstat
calls polluted source codes. Hard to maintain.
Solution:
Platform dependent code are moved to H5private.h and then internal
code can #include H5private.h. Repeat those macro definition for
the stdio and multi drivers since they area examples for writing
a virtual file driver. They must not use any internal code.
Platforms tested:
eirene (parallel), modi4 (serial including gass driver.)
Change default actions.
Description:
Change the default maximum number of processes (-P) to use all processes
instead of just 1 (old default). Someone most likely wants to test
the I/O performance with all processes involved.
Also starts performance measurement with maximum number of processes
and decrement it with each loop. If the performance measurement
needs to restart, it can run with fewer processes if those loops
have completed.
Platforms tested:
modi4 and eirene.
Feature add and algorithm reworking.
Description:
Added a "--debug" flag so we can print out various extra debugging
information.
Reworked the algorithm so that it's printing the correct throughput.
Here's how it's supposed to work:
process
T_0 T_1 T_2 T_3 ... T_n
iteration 1
2
.
.
.
m
Retrieve the maximum time from each iteration over the number of
processes. (So, if T_i had the maximum time in iteration j, then use
that time). Calculate the "Throughput" of iteration j:
S_j = (raw_size / T_i)
Collect that information over all of the iterations. Then output the
Max, Min, and Ave of all of the S_k's.
Platforms tested:
Linux (pp)
Purpose:
Feature Fix
Description:
Changed so the "pio_perf" module handles creating and destroying the
MPI Comm. Worked it so we get the minimum, maximum, and average times
over a set of iterations.
Solution:
Lots. Had to pull the MPI Comm code from the "pio_engine" module and
place it in the "pio_perf" module. Then worked on a way to have all
processes send their time output to process 0, who collects it and
gives back the min, max, and avg times for the iterations.
Platforms tested:
Linux. Doesn't work if you use more than 1 processor...*hrmph*
Purpose:
Feature Changes
Description:
Okay, I needed to add in more parameters so that the user can modify
how things are supposed to work with the PIO programs. Also needed
to change the algorithm a bit to make these work. And needed to add
in timing for the READ option.
Solution:
Added the above things. The parameters took a major rewrite of the
command-line parsing stuff. Here's the usage statement:
usage: pio_perf [OPTIONS]
OPTIONS
-h, --help Print a usage message and exit
-d N, --num-dsets=N Number of datasets per file [default:1]
-f S, --file-size=S Size of a single file [default: 64M]
-F N, --num-files=N Number of files [default: 1]
-H, --hdf5 Run HDF5 performance test
-i, --num-iterations Number of iterations to perform [default: 1]
-m, --mpiio Run MPI/IO performance test
-o F, --output=F Output raw data into file F [default: none]
-P N, --max-num-processes=N Maximum number of processes to use [default: 1]
-p N, --min-num-processes=N Minimum number of processes to use [default: 1]
-r, --raw Run raw (UNIX) performance test
-X S, --max-xfer-size=S Maximum transfer buffer size [default: 1M]
-x S, --min-xfer-size=S Minimum transfer buffer size [default: 1K]
F - is a filename.
N - is an integer >=0.
S - is a size specifier, an integer >=0 followed by a size indicator:
K - Kilobyte
M - Megabyte
G - Gigabyte
Example: 37M = 37 Megabytes
Platforms tested:
Linux, but not fully finished...
Purpose:
Small Fixes
Description:
After conversation with Albert, here are some small fixes for the
performance stuff. Not too significant. Though, we did add the
"buffer size" as a parameter I pass to the engine.
Purpose:
bug Fix
Description:
Changed the code so that if parallel stuff isn't enabled, then we
don't compile the parallel code.
Solution:
Cleaned up the code and put #ifdef's around it checking for parallel
flags.
Platforms tested:
Linux
New addition
Description:
Initial version of the Parallel I/O performance measurement program.
Not fully implemented yet but checking them in before I may destroy
them by accident.
Don't run this in small file system (like AFS or eirene) since it
generates gigabytes test files.
Platforms tested:
modi4 64bits. It compiled and ran but took a long time because
the current test parametes are too "wild".
