Bug Fix for bug #789
Description:
Creating a 1-D dataset region reference caused the library to hang (go into
an infinite loop).
Solution:
Corrected algorithm for serializing hyperslab regions.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Feature Update
Description:
It now writes an uncompressed version of the file out. The first
tests didn't look all that wonderful. Here is a typical output:
Buffer size == 1MB
Uncompressed Write Time: 0.55s
Uncompressed Write Throughput: 116.00MB/s
Compressed Write Time: 8.76s
Compressed Write Throughput: 7.31MB/s
Platforms tested:
Linux
Compress I/O Test Add
Description:
This is the initial check-in of the compress I/O test. It will write
out compressed buffers to a file.
I need to implement a write of uncompressed data first for
comparison...
Platforms tested:
Linux
Bug fix.
Description:
Added extra check to locations where metadata and "small data" blocks were
being returned to the file's list of free space to avoid freeing 0-sized
blocks. This should fix the problems in last night's daily tests.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Code Improvement
Description:
Split the metadata accumulator code into two parts: one for allowing writes
of the accumulator buffer during reads (when the buffer is dirty and needs
to be flushed to disk in order to hold the new metadata being read in) and
another for only allowing writes of the buffer during writes.
This allows the MPI-I/O VFL driver to use the metadata accumulator (but
only during writes) and benefit from the reduced number of metadata I/O
operations that it brings.
Platforms tested:
IRIX64 6.5 (modi4) w/parallel
Bug fix.
Description:
Return H5S_all_opt_possible() prototype to original position in H5Spkg.h,
it is not related to parallel I/O.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Code cleanup/bug fix
Description:
Unused space in the file was being "dropped on the floor" when the metadata
or "small data" block was moved to a new location in the file and there was
unused space in the old block.
Solution:
Put the space left in the allocation block into the free list of space for
the file. This allows it to be [potentially] reused and sometimes allows
the file to be smaller.
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
Code improvement.
Description:
Added boot block and driver info block checksumming feature to the shared
file information. This prevents these blocks from being written out
multiple times when they haven't changed.
This reduces the number of I/O operations which hit the disk for my test
program from 15 to 14 (i.e. from 393 to 14, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
New feature.
Description:
Added a "small data" block allocation mechanism to the library, similar to
the mechanism used for allocating metadata currently.
See the RFC for more details:
http://hdf.ncsa.uiuc.edu/RFC/SmallData/SmallData.html
This reduces the number of I/O operations which hit the disk for my test
program from 19 to 15 (i.e. from 393 to 15, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
Feature Add
Description:
Added support for testing only the "write" operations. Use the "-w"
or "--write-only" flags for this.
Platforms tested:
Linux (parallel) & Modi4 (parallel)
Code Cleanup
Description:
Removed some compiler warnings.
Solution:
In a few cases, NULL was being returned when a FAIL was supposed to
be returned instead. There were some header files which needed to be
included in a few of the sources. A couple of if-then statements had
assignments in the conditional part. The compiler warned that they
should have extra "()"s around them. Made the code check the values
instead.
Platforms tested:
Linux (parallel) Modi4 (parallel)
Code improvement
Description:
Now that the metadata accumulator is working correctly, the buffer size
tends to get rather large and stay there.
Solution:
Put a throttle on the accumulator buffer to reduce its size back down when
it isn't needed.
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
Purpose:
Bug fix (#699), fix provided by a user, approved by Quincey
Description:
When a scalar dataspace was written to the file and then
subsequently queried with the H5Sget_simple_extent_type function,
type was reported H5S_SIMPLE instead of H5S_SCALAR.
Solution:
Applied a fix (see bug report 699) and modified out test program
to exercise the case.
Platforms tested:
Solaris 2.7 and Linux 2.2.18
Purpose:
Bug fix (#699), fix provided by a user, approved by Quincey
Description:
When a scalar dataspace was written to the file and then
subsequently queried with the H5Sget_simple_extent_type function,
type was reported H5S_SIMPLE instead of H5S_SCALAR.
Solution:
Applied a fix (see bug report 699)
Platforms tested:
Solaris 2.7 and Linux 2.2.18
Code improvement
Description:
The metadata aggregation code in the library was not terribly smart about
extending contiguous regions of metadata in the file and would not extend
them as far as possible. This causes space in the file to be wasted, also.
Solution:
Be smarter about extending the space used in the file for metadata by
checking whether new metadata blocks allocated in the file are at the end
of the current metadata aggregation region and append them to the metadata
region if so. This has the nice side benefit of reducing the number of
bytes we waste in the file and reducing the size of the file by a small
amount in some cases.
This reduces the number of I/O operations which hit the disk for my test
program from 53 to 19 (i.e. from 393 to 19, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
Bug Fix
Description:
The "dirty" flag for symbol table entries and symbol table nodes was not
being cleared when they were flushed to the file, causing lots of extra
metadata I/O.
Solution:
Reset the symbol table entry & nodes' flags when thy are flushed to disk.
This reduces the number of I/O operations which hit the disk for my test
program from 83 to 53 (i.e. from 393 to 53, overall).
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
Tidy up
Description:
Old version showed tons of output even if MPI_Offset is too small
to support multiple GB sized files and destined to fail.
Output is pretty confusing.
Solution:
Prints the signness and size of MPI_Offset for information.
Skipped tests if MPI_Offset is not big enough to support the file
sizes.
Platforms tested:
modi4, eirene, burrwhite (all parallel).
Bug Fix
Description:
There was an error in the way it was determining if there were any
reads to report.
Solution:
Wrote a subroutine which would determine if the reads are defined in
the structure before trying to print them out.
Platforms tested:
Linux
Add Report Generator
Description:
This is a generator for ASCII and Excel(tm)(c) reports generated by
the pio_perf benchmark we run. There are instructions at the top of
the file on how to run the program and the input format it expects,
etc.
Platforms tested:
Linux
Code cleanup/bug fix
Description:
The "metadata accumulator" cache in the library (which is designed to catch
small metadata writes/reads and bundle them together into larger I/O
buffers) was incorrectly detecting the important case of metadata pieces
being written sequentially to the file, adjoining but not overlapping.
Additionally, the metadata accumulator was not being used to cache data
read in from disk, only caching writes.
Solution:
Fix accumulator to correctly cache adjoining metadata writes and also to
cache metadata read from disk.
Between these two fixes, the number of I/O requests which resulted in actual
reads/writes to the filesystem dropped from 393 requests to 82 for the
particular test I was using. :-)
Platforms tested:
Solaris 2.7 (arabica) w/FORTRAN & FreeBSD 4.5 (sleipnir) w/C++
Test Bug Fix
Description:
Under certain [obscure] circumstances, an object header would get paged out
of the metadata cache, and when it was accessed again and brought back into
the cache, and immediately had additional metadata added to it (an
attribute, usually, or perhaps adding an object to a group), and needed to
be extended with a continuation message, but there was no room in any
existing object header chunks for the continuation message and an existing
object header message needed to be moved to the new object header chunk (I
told you it was obscure :-), the object header message moved to the new
chunk (not the new metadata being added) would get corrupted. *whew* :-)
Solution:
Actually copy the "raw" object header message information of the object
header message being moved to the new chunk, instead of relying on the
"native" object header message information being re-encoded when the object
header is flushed. This is because when an object header is paged out of
the metadata cache and subsequently brought back in, the "native"
information pointer in memory is reset to NULL and only the "raw"
information exists.
[Actually, this additional testing doesn't trigger the bug, which needs
_lots_ of objects to be created and accessed, but it does execise the
object header continuation code more than other tests in the library.]
Platforms tested:
Solaris 2.7 (arabica) & FreeBSD 4.5 (sleipnir)
Code Bug Fix
Description:
Under certain [obscure] circumstances, an object header would get paged out
of the metadata cache, and when it was accessed again and brought back into
the cache, and immediately had additional metadata added to it (an
attribute, usually, or perhaps adding an object to a group), and needed to
be extended with a continuation message, but there was no room in any
existing object header chunks for the continuation message and an existing
object header message needed to be moved to the new object header chunk (I
told you it was obscure :-), the object header message moved to the new
chunk (not the new metadata being added) would get corrupted. *whew* :-)
Solution:
Actually copy the "raw" object header message information of the object
header message being moved to the new chunk, instead of relying on the
"native" object header message information being re-encoded when the object
header is flushed. This is because when an object header is paged out of
the metadata cache and subsequently brought back in, the "native"
information pointer in memory is reset to NULL and only the "raw"
information exists.
Platforms tested:
Solaris 2.7 (arabica) & FreeBSD 4.5 (sleipnir)
Document Bug Fix
Description:
Under certain [obscure] circumstances, an object header would get paged out
of the metadata cache, and when it was accessed again and brought back into
the cache, and immediately had additional metadata added to it (an
attribute, usually, or perhaps adding an object to a group), and needed to
be extended with a continuation message, but there was no room in any
existing object header chunks for the continuation message and an existing
object header message needed to be moved to the new object header chunk (I
told you it was obscure :-), the object header message moved to the new
chunk (not the new metadata being added) would get corrupted. *whew* :-)
Solution:
Actually copy the "raw" object header message information of the object
header message being moved to the new chunk, instead of relying on the
"native" object header message information being re-encoded when the object
header is flushed. This is because when an object header is paged out of
the metadata cache and subsequently brought back in, the "native"
information pointer in memory is reset to NULL and only the "raw"
information exists.
Platforms tested:
Solaris 2.7 (arabica) & FreeBSD 4.5 (sleipnir)
Bug Fix
Description:
C++ uses the "off_t" type for external files as well. I'm assuming
that there's going to be a problem with 4 byte off_ts and 8 byte
off_ts in the C++ code as well. I compiled everything on Linux and it
worked, but I don't think we have a check for this bug in the library
yet...
Platforms tested:
Linux
Purpose:
Code clean up.
Description:
Many F90 compilers were not happy about character*(*) declarations.
Solution:
Used F90 character(len=*) declarations.
Platforms tested:
Solaris 2.7 and Linux 2.4
Feature Enhancement
Description:
We can now set the MPI_Info object to more than one key/value
pairing. By setting the HDF5_MPI_INFO environment variable to a
semicolon separated list of "key=value" pairs, we can set the
MPI_Info variable to them.
Platforms tested:
Linux
Code cleanup
Description:
While working on the "External test" failure I restructured and
cleaned up some C stub code.
Platforms tested:
dangermouse (Linux 2.4)
Bug Fix
Description:
On Linux 2.4 kernel, we need to check for Linux LFS support. There
was a failure in the Fortran code when we tried to pass a 4 byte off_t
variable to a function which expected an 8 byte off_t. This is due to
there being no check for the LFS support in the Fortran code.
Solution:
Added the check for Linux LFS support.
Platforms tested:
Elena will test on Linux platform...
Bug fix
Description:
H5Ewalk_cb was being called from a C++ API, when it really is an
internal routine to the H5E interface.
Solution:
Removed C++ API wrapper for it.
Platforms tested:
FreeBSD 4.5 (sleipnir)
Code cleanup and new feature.
Description:
Cleaned out some old debug print statments.
Added option -B for interleaved I/O block size.
(Just parsing and sanity check is done. Real I/O implementation
is not done yet.)
Platforms tested:
eirene(pp)
Removal of bad code
Description:
Albert came up with a better way of determining which performance
tests to run. I removed the broken way I did them...
Platforms tested:
Linux-pp
Purpose:
Bug Fixing
Description:
In H5FD_family_write and H5FD_family_read, size_t is checked against
hsize_t for overflow, which fails on IA32 architecture machine supporting
large files.
Solution:
Use temporary variable which won't pass the limit of size_t.
Platforms tested:
Linux 2.4(platinum) and IRIX64 6.5(modi4)
bug fix
Description:
pio_info_g has been moved to the test/libh5test.a but the
pio_* source files still have the old definition and declaration.
Some compilers hated the duplcation.
Solution:
Replaced the duplication with #include h5test.h and added test/ to
the include search path too.
Platforms tested:
eirene, modi4.
Code cleanup
Description:
Broke the FUNC_ENTER macro into several macros, with more specialized
uses (which followup mail will describe). This was designed to move
most/all of the checks which could be done at compile time to that point,
instead of needlessly performing them (over & over :-) at run-time.
This reduces the library's size (and thus staticly linked binaries) and
has a minor speedup effect also.
Platforms tested:
IRIX64 6.5 (modi4) with parallel & FORTRAN enabled, and additional testing
on FreeBSD and Solaris immediately after the checkin.
Update
Description:
Only print out the parameters for a run if we're in debug mode one
and the 0'th process or if we're in debug mode > 0.
Platforms tested:
Linux
