Omnibus floating-point bug fix changes
Description:
There are a number of problems in the floating-point conversion code that
were exposed by Ray's recent int<->float checkin:
- The 'my_isnan' code in test/dtypes.c was broken and would always return
true. The meant that the actual values in the float<->float conversion
tests were _never_ checked, hiding the other bugs included in this
checkin.
- A recent change I made to the type conversion code used "FLT_MIN" instead
of "-FLT_MAX" for the most negative 'float' value for the double->float
conversion, which meant that any the negative number that was converted
from a double to a float would have been mapped to zero, essentially.
- A change that Robb appeared to have made ~2.5 years ago to the "generic"
float->float conversion routine appears to be incorrect and I've backed
it out.
- Floating-point conversions on SGI's which converted denormalized values
would be mapped to zero instead of being propertly preserved in the new
type. This was addressed by an SGI-specific system call to prevent the
behavior.
Solution:
Described above, generally.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Misc. update:
release_docs/RELEASE update forthcoming...
Bug fix & code cleanup
Description:
Allowing the library to call malloc with a size of 0 bytes causes problems
for some users, so we check for allocations of 0 bytes and disallow them now.
Cleaned up some code which could call malloc with 0 size.
Changed some code calling HDmalloc directly to call H5MM_malloc(), which
allows us to check for 0 sized allocations.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
More Checks
Description:
Added checks for correctly working "basename" and "xargs" programs.
Mike McKay was having troubles with the xargs. The basename check was
just a good idea.
Platforms tested:
Verbena, Arabica, Modi4
Misc. update:
updated help page
Description:
help message for enable-stream-vfd was still default=no.
changed to default=yes.
Platforms tested:
no h5committest. Only tested in eirene since change is
simple.
Misc. update:
Add check
Description:
Added a check to make sure that the "tr" program actually works.
Platforms tested:
Linux (small fix and only to configure)
Misc. update:
Update
Description:
Enable the stream-vfd driver by default. --disable-stream-vfd if you
don't want it.
Platforms tested:
Linux (configuration change, no need for full testing)
Misc. update:
Bug fix
Description:
The MPI_File_set_size() routine on ASCI Red is not able to extend files
so that they are larger than 2GB.
Solution:
Add an extra macro which controls whether MPI_File_set_size() can handle
>2GB offsets or if our "older" way of reading a byte, then writing a byte at
the appropriate offset should be used.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Code cleanup/minor bug fixes
Description:
Corrected a few typos and changed the order that the printf() format
strings are tested for 'long long's to move 'll' ahead of 'q', since 'll' is
the form for the ANSI C99 standard.
Platforms tested:
h5committested
Misc. update:
Bug Fix
Description:
On "user02", the Fortran compiler is "efc". We weren't testing for
this compiler (and I don't know how it was succeeding before...).
Solution:
Added efc to the list of Fortran compilers we check for.
Platforms tested:
Couldn't test as I don't have access to user02. But this doesn't
affect other platforms.
Misc. update:
Bug Fix
Description:
C++ on Irix 6.5 needs some special libraries (libCio or some such
thing). These were defined using the DEFAULT_LIBS macro. However,
they weren't being placed into the Makefiles since it wasn't
AC_SUBST'ed in configure.in and wasn't declared in commence.in.
Solution:
Added to configure.in and also added and commence.in
Platforms tested:
Modi4 (C++, small change and only Irix specific)
Misc. update:
Update
Description:
Revamped the configuration system. The configurations for the Fortran
and C++ libraries are no longer separate from the "main"
configuration system. This involved removing the "configure*" and
"aclocal.m4" files from the fortran/ and c++/ subdirectories. Also
merging settings in the config/ subdirectories into the main config/
subdirectory.
Fortran header files had to be modified a little for Linux. It was
checking if it was a Linux machine by some #defines, however with the
-std=c99 switch, these defines weren't there. I added a check for
some other ones which should be there whether the -std=c99 switch is
used or not.
Platforms tested:
Verbena (Fortran & C++)
Sol (Fortran & C++)
Copper (Fortran & C++)
Modi4 (Parallel, Fortran, & C++)
Misc. update:
Feature change.
Description:
--prefix default was /usr/local. This is not approprirate for
development version since it is not likely that one would want
to install a development version in the /usr/local area.
Solution:
Change --prefix default to `pwd`/hdf5, a directory in the build
directory. This eliminates the problem that the builder may not
have permission to install in the default install directory.
It also allows multiple builds in the same host or multiple srcdir
using the same source.
Platforms tested:
h5committested.
Also tested in eirene by trying various configure combination to
verify the new effect.
Misc. update:
Update configuration
Description:
Now that we've split the tree (Yea!), change the main trunk back to debug
on and production off.
Also, change the version # to 1.7.0
update for release
Description:
Set the version information to 1.6.0-pre1 to get ready for
v1.6 release.
Platforms tested:
h5committested.
Misc. update:
Code cleanup
Description:
Make the default for the function stack work like the default for the
API tracing: when debug is enabled, the function stack defaults to on and
when debug is disabled, the function stack defaults to off.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
Purpose: Maintenance
Description: Bumped the version number to 1.5.62 after creating
a tar ball for the fourth round of testing.
Solution: run bin/h5vers -s scipt on arabica
Platforms tested: N/A
Misc. update:
Update
Description:
Changed the --enable-mpe flag to --with-mpe so that we could
specify the directory information for where the MPE stuff is
kept. (with the =DIR option that we have for --with-* flags)
Solution:
7%
Platforms tested:
Copper (where MPE is...small change no requiring h5committest).
Misc. update:
Bug fix.
Description:
the MPI_Get_count test does not work correctly (the macro name
MPI_GET_COUNT_WORKS is not even used in source code.) For some
reason, this test sometimes hangs in AIX.
Solution:
Blocked out the test for now and removed the code later when
things are verified to be correct.
Platforms tested:
h5committested.
ALso tested in Copper (32/64bit parallel, 32 bit serial).
Misc. update:
Purpose: Maintenance for the third round of testing
Description: Increased the version number to 1.5.59 after creating
a tar ball for testing.
Solution:
Platforms tested:
Misc. update:
Purpose: Maintenance for the second round of testing
Description: Used bin/h5vers to change version number to 1.5.55
Solution:
Platforms tested: arabica
Misc. update:
Update
Description:
GPFS should only be checked for if the --enable-gpfs flag is used.
Otherwise, it shouldn't.
Platforms tested:
Linux (Configure only fix so only needed to test it to make sure the
configure does the right thing and that the build won't
break. I need to test it on a machine that has GPFS to make
sure it works there...)
Fix
Description:
The SZLIB stuff was being checked by default. This isn't right as not
everyone will have SZLIB stuff.
Solution:
Made the default "no" instead of "yes"
Platforms tested:
Modi4 (Fortran & Parallel)
Verbena (Fortran & C++)
Arabica (Fortran)
Misc. update:
Purpose: Maintenance
Description: I changed release version to 1.5.54.
Email about tetsing of 1.5.53 will be send after this checkins.
Solution: used bin/h5vers -s to change the version number
Platforms tested: Done on arabica
Misc. update:
Feature
Description:
Added new flag to the H5pubconf.h file --- NO_SHARED_WRITING
(H5_NO_SHARED_WRITING). It's only defined for CodeWarrior and should
never be defined for other platforms unless the "open()" bug that CW
has shows up on other platforms.
Platforms tested:
Verbena. Configure only, so no need for full tests.
Misc. update:
Update
Description:
Added printing of the compiler used in the end message.
Platforms tested:
Modi4 (Parallel & Fortran)
Burrwhite (Fortran & C++)
Baldric (Fortran), but make check didn't work because of "libucb.so"
error that I can't fix...)
Misc. update:
Update
Description:
The "--enable-parallel=TYPE" option was revamped but the help string
wasn't modified.
Solution:
Modified it to print out just "--enable-parallel" because we no
longer support the "=TYPE" option.
Platforms tested:
arabica (Fortran)
Modi4 (Parallel & Fortran)
Burrwhite (Fortran & C++)
Misc. update:
Fix
Description:
Previous changes in the configure were overwritten.
Solution:
Updated configure.in with the changes that were lost.
Platforms tested:
Linux & Modi4.
Misc. update:
Purpose: New tool
Description: Pankaj created h5import tool long time ago. I am finally checking
the source code in for the 1.6 release.
Known problems: h5import test will crash if srcdir option
is used; will be fixed before the release.
Solution:
Platforms tested: vebena, arabica, modi4 were tested. h5import tests passed
I also tested on arabica with srcdir option to make sure that
there is a problem when srcdir is used. After that I disabled
h5import tests. Daily tests will build h5import, but will not run
h5import tests.
Misc. update:
Code cleanup
Description:
Re-arranged configure output for external I/O filters to show that zlib
and szlib are used for external I/O filters and display them with the output
for the internal I/O filters.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir)
Linux 2.4 (eirene)
IRIX64 6.5 (modi4)
Misc. update:
This is the first check-in message related to szip support at HDF5 1.6 release
Description:
szip compression support is required by NASA ESDIS. The compression algorithm
is a good compression algorithm for scientific data. In HDF5, we add another filter
function to make szip as a default compression package as we did for gzip(or zlib).
Solution:
In the configure.in and configure, I handled szip exactly the same as what zlib was built into
the HDF5.
Currently if you don't specify in the configure command line with:
./configure --with-szlib=/.....
Unless you put your szip library in your path,you will build the same HDF5 library like before.
Also, I only use static szip library to test with HDF5.
Platforms tested:
Since there are changes of configure.in and configure,I didn't use h5committest.
I tested with four platforms.
1) Linux 2.4 (eirene)
2) Solaris 2.7(arabica)
3) windows 2000(VS 6.0)
4) SGI IRIX6.5-64(modi4)
For test 1)-3), only basic C tests were done
For modi4 test, I tested 64-bit C,parallel and fortran.
ALl tests passed, except a warning message from szip library when checksum is used in some order, which doesn't cause any real problems.
Misc. update:
Update
Description:
Added Flexible Parallel HDF5 support to the ending message that
configure spits out.
Reordered a few other of the messages so that they're in alphabetical
order.
Platforms tested:
Linux
Bug fix
Description:
The "system scope" for threads isn't supported on all platforms.
Solution:
Add detection of this feature to the configure script and check for
"H5_HAVE_SYSTEM_SCOPE_THREADS" in the appropriate places.
Platforms tested:
modi4 w/threadsafe
New feature for developers.
Description:
Added "function stack" tracing to library. This allows developers (there
is no public API) to call H5FS_print within the library and get a listing
of the functions traversed to reach that point in the library. Eventually,
I may add support for reporting the parameters to each function also...
Mainly for debugging parallel I/O programs, but I think it will come in
handy in other cases also.
The function stack tracking is controlled with a configure switch:
--enable-funcstack, which defaults to enabled currently. When we branch
for 1.6, we should change the default setting on the branch to be disabled.
Also, added a destructor to the thread-specific keys when thread-safety is
turned on in the library. Otherwise, they were leaking memory and causing
difficult to debug errors in threaded programs (like the test/ttsafe test).
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) w/thread-safety enabled.
Misc. update:
Updated MANIFEST with new files added (src/H5FS.c & src/H5FDprivate.h)
Update release_docs/RELEASE with thread-safety bug fix.
Update
Description:
Updated the configure scripts. Fortran and C++ didn't have proper
checks for Linux LFS support. Also needed to remove the "-g" flag
from the compile line if --enable-production is set. For that, I took
how it's being done in HDF5 Lite...
Platforms tested:
Linux
Bug Fix
Description:
Some compilers were having problems if we included the -I/usr/include
and -L/usr/lib flags which are completely redundant. This could occur
if the user configured with something like:
./configure --with-zlib=/usr/include,/usr/lib
Solution:
Check if the --with-* command is specified with /usr/include and
/usr/lib. If so, then don't place them in the compile macros.
Platforms tested:
Linux, Modi4
Another patch
Description:
libtool will fail if pgcc compiler is used.
Now apply the patch if gcc is NOT used.
Platforms tested:
Tested only on eirene because I am sure it is correct and am trying
to beat the daily test checkout.
a patch
Description:
When intel compilers (icc, ecc) are used in a linux system, the libtool
generated has a wrong setting of '$wl" for linking.
Put in a "ed ..." command script to edit the generated libtool with
the proper setting. The libtool people has fixed this problem.
This can ben taken out when that version of libtool is released.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? YES.
Other platforms/configurations tested? platinum and titan using
gcc and intel CC.
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
new feature
Description:
Added MPICH/MPE instrumentation support.
All source code are bracketed by the macro H5_HAVE_MPE.
Use "--enable-mpe" to configure it in.
Currently only worked in Eirene because the MPE library is
not installed in all machines yet.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yep.
Other platforms/configurations tested?
--enable-mpe feature tested in Eirene.
Cleanup
Description:
Removed the option (--enable-parallel=mpich). It did not work.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? All passed.
New feature.
Description:
Add in '--enable-filters=filter1,filter2' flag to configure script. This
can also be used as '--enable-filters=all' or just '--enable-filters' to
enable all the internal I/O filters in the library.
This flag can also be set to '--disable-filters' to disable all the internal
I/O filters. Individual filters can not be disabled due to the limitations
of autoconf.
Additionally, added "H5_HAVE_FILTER_<foo>" macros to H5pubconf.h header
file, so the presence of each filter can be detected.
This is in preparation for Kent's shuffling filter to be imported shortly.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Tested FreeBSD 4.7 (sleipnir) also
Update
Description:
Added new files to the MANIFEST and added a new configure switch for
enabling building of the Flexible parallel HDF5 module.
Platforms tested:
Modi4 Eirene Arabica
fix/feature
Description:
v1.4's configure had enable-trace default as on. That is not right for
production code as it increase the object code and uses up compute time.
Changed --enable-trace default to depend on --enable-debug. If debug is
on, trace is on. Otherwise it is off.
Also added the printing of CFLAGS, CPPFLAGS, and LDFLAGS in the summary.
Platforms tested:
eirene, arabica (pp).
Small Coding Change
Description:
Zeroed out the MPI_Status structure in one of our tests to determine
if MPI_Get_count() works on the machine or not. This is similar to
what we do in the library code to the status...
Fix
Description:
Better check for versions of Linux. We didn't support kernels with
major version numbers >2 and minor version numbers <4...
Solution:
Fixed to check just that.
Platforms tested:
Linux 2.2 && 2.4
Update, Bug Fix, and Feature Add
Description:
- Updated how AC_{ENABLE,WITH} help messages were being generated.
Autoconf now gives you an AC_HELP_STRING macro to use to create
them.
- Fixed the problem with Linux LFS on RedHat 7.3 machines. It wasn't
finding getdents64(), so we can't rely on that being present
anymore...
- Added GPFS detection and setting.
- Updated how compression is specified. It's no longer necessary to
test for HAVE_ZLIB_H, HAVE_LIBZ, and HAVE_COMPRESS2. The one macro
{H5_}HAVE_COMPRESSION takes care of all of these.
Solution:
Changed the check for Linux LFS from looking for getdents() to
looking at the version number of the kernel (using the uname -r
command). You can still override with the --enable-linux-lfs switch
if you really believe that your <2.4 kernel has LFS support.
Platforms tested:
Linux (2.2 and 2.4)
New feature
Description:
Changed parallel I/O tests to use test script instead of just running
testphdf5 executable. This allows the MPI-posix driver to be tested easily.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/parallel and IRIX64 6.5 (modi4) w/parallel
New feature
Description:
Add support in for "dmalloc" debugging malloc library to configure script
and library headers.
Platforms tested:
IRIX64 6.5 (modi4) and FreeBSD 4.6 (sleipnir)
