Update
Description:
Added printing of the compiler used in the end message.
Platforms tested:
Modi4 (Parallel & Fortran)
Burrwhite (Fortran & C++)
Baldric (Fortran), but make check didn't work because of "libucb.so"
error that I can't fix...)
Misc. update:
Update
Description:
The "--enable-parallel=TYPE" option was revamped but the help string
wasn't modified.
Solution:
Modified it to print out just "--enable-parallel" because we no
longer support the "=TYPE" option.
Platforms tested:
arabica (Fortran)
Modi4 (Parallel & Fortran)
Burrwhite (Fortran & C++)
Misc. update:
Fix
Description:
Previous changes in the configure were overwritten.
Solution:
Updated configure.in with the changes that were lost.
Platforms tested:
Linux & Modi4.
Misc. update:
Purpose: New tool
Description: Pankaj created h5import tool long time ago. I am finally checking
the source code in for the 1.6 release.
Known problems: h5import test will crash if srcdir option
is used; will be fixed before the release.
Solution:
Platforms tested: vebena, arabica, modi4 were tested. h5import tests passed
I also tested on arabica with srcdir option to make sure that
there is a problem when srcdir is used. After that I disabled
h5import tests. Daily tests will build h5import, but will not run
h5import tests.
Misc. update:
Code cleanup
Description:
Re-arranged configure output for external I/O filters to show that zlib
and szlib are used for external I/O filters and display them with the output
for the internal I/O filters.
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir)
Linux 2.4 (eirene)
IRIX64 6.5 (modi4)
Misc. update:
This is the first check-in message related to szip support at HDF5 1.6 release
Description:
szip compression support is required by NASA ESDIS. The compression algorithm
is a good compression algorithm for scientific data. In HDF5, we add another filter
function to make szip as a default compression package as we did for gzip(or zlib).
Solution:
In the configure.in and configure, I handled szip exactly the same as what zlib was built into
the HDF5.
Currently if you don't specify in the configure command line with:
./configure --with-szlib=/.....
Unless you put your szip library in your path,you will build the same HDF5 library like before.
Also, I only use static szip library to test with HDF5.
Platforms tested:
Since there are changes of configure.in and configure,I didn't use h5committest.
I tested with four platforms.
1) Linux 2.4 (eirene)
2) Solaris 2.7(arabica)
3) windows 2000(VS 6.0)
4) SGI IRIX6.5-64(modi4)
For test 1)-3), only basic C tests were done
For modi4 test, I tested 64-bit C,parallel and fortran.
ALl tests passed, except a warning message from szip library when checksum is used in some order, which doesn't cause any real problems.
Misc. update:
Update
Description:
Added Flexible Parallel HDF5 support to the ending message that
configure spits out.
Reordered a few other of the messages so that they're in alphabetical
order.
Platforms tested:
Linux
Bug fix
Description:
The "system scope" for threads isn't supported on all platforms.
Solution:
Add detection of this feature to the configure script and check for
"H5_HAVE_SYSTEM_SCOPE_THREADS" in the appropriate places.
Platforms tested:
modi4 w/threadsafe
New feature for developers.
Description:
Added "function stack" tracing to library. This allows developers (there
is no public API) to call H5FS_print within the library and get a listing
of the functions traversed to reach that point in the library. Eventually,
I may add support for reporting the parameters to each function also...
Mainly for debugging parallel I/O programs, but I think it will come in
handy in other cases also.
The function stack tracking is controlled with a configure switch:
--enable-funcstack, which defaults to enabled currently. When we branch
for 1.6, we should change the default setting on the branch to be disabled.
Also, added a destructor to the thread-specific keys when thread-safety is
turned on in the library. Otherwise, they were leaking memory and causing
difficult to debug errors in threaded programs (like the test/ttsafe test).
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) w/thread-safety enabled.
Misc. update:
Updated MANIFEST with new files added (src/H5FS.c & src/H5FDprivate.h)
Update release_docs/RELEASE with thread-safety bug fix.
Update
Description:
Updated the configure scripts. Fortran and C++ didn't have proper
checks for Linux LFS support. Also needed to remove the "-g" flag
from the compile line if --enable-production is set. For that, I took
how it's being done in HDF5 Lite...
Platforms tested:
Linux
Bug Fix
Description:
Some compilers were having problems if we included the -I/usr/include
and -L/usr/lib flags which are completely redundant. This could occur
if the user configured with something like:
./configure --with-zlib=/usr/include,/usr/lib
Solution:
Check if the --with-* command is specified with /usr/include and
/usr/lib. If so, then don't place them in the compile macros.
Platforms tested:
Linux, Modi4
Another patch
Description:
libtool will fail if pgcc compiler is used.
Now apply the patch if gcc is NOT used.
Platforms tested:
Tested only on eirene because I am sure it is correct and am trying
to beat the daily test checkout.
a patch
Description:
When intel compilers (icc, ecc) are used in a linux system, the libtool
generated has a wrong setting of '$wl" for linking.
Put in a "ed ..." command script to edit the generated libtool with
the proper setting. The libtool people has fixed this problem.
This can ben taken out when that version of libtool is released.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? YES.
Other platforms/configurations tested? platinum and titan using
gcc and intel CC.
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
new feature
Description:
Added MPICH/MPE instrumentation support.
All source code are bracketed by the macro H5_HAVE_MPE.
Use "--enable-mpe" to configure it in.
Currently only worked in Eirene because the MPE library is
not installed in all machines yet.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? Yep.
Other platforms/configurations tested?
--enable-mpe feature tested in Eirene.
Cleanup
Description:
Removed the option (--enable-parallel=mpich). It did not work.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? All passed.
New feature.
Description:
Add in '--enable-filters=filter1,filter2' flag to configure script. This
can also be used as '--enable-filters=all' or just '--enable-filters' to
enable all the internal I/O filters in the library.
This flag can also be set to '--disable-filters' to disable all the internal
I/O filters. Individual filters can not be disabled due to the limitations
of autoconf.
Additionally, added "H5_HAVE_FILTER_<foo>" macros to H5pubconf.h header
file, so the presence of each filter can be detected.
This is in preparation for Kent's shuffling filter to be imported shortly.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Tested FreeBSD 4.7 (sleipnir) also
Update
Description:
Added new files to the MANIFEST and added a new configure switch for
enabling building of the Flexible parallel HDF5 module.
Platforms tested:
Modi4 Eirene Arabica
fix/feature
Description:
v1.4's configure had enable-trace default as on. That is not right for
production code as it increase the object code and uses up compute time.
Changed --enable-trace default to depend on --enable-debug. If debug is
on, trace is on. Otherwise it is off.
Also added the printing of CFLAGS, CPPFLAGS, and LDFLAGS in the summary.
Platforms tested:
eirene, arabica (pp).
Small Coding Change
Description:
Zeroed out the MPI_Status structure in one of our tests to determine
if MPI_Get_count() works on the machine or not. This is similar to
what we do in the library code to the status...
Fix
Description:
Better check for versions of Linux. We didn't support kernels with
major version numbers >2 and minor version numbers <4...
Solution:
Fixed to check just that.
Platforms tested:
Linux 2.2 && 2.4
Update, Bug Fix, and Feature Add
Description:
- Updated how AC_{ENABLE,WITH} help messages were being generated.
Autoconf now gives you an AC_HELP_STRING macro to use to create
them.
- Fixed the problem with Linux LFS on RedHat 7.3 machines. It wasn't
finding getdents64(), so we can't rely on that being present
anymore...
- Added GPFS detection and setting.
- Updated how compression is specified. It's no longer necessary to
test for HAVE_ZLIB_H, HAVE_LIBZ, and HAVE_COMPRESS2. The one macro
{H5_}HAVE_COMPRESSION takes care of all of these.
Solution:
Changed the check for Linux LFS from looking for getdents() to
looking at the version number of the kernel (using the uname -r
command). You can still override with the --enable-linux-lfs switch
if you really believe that your <2.4 kernel has LFS support.
Platforms tested:
Linux (2.2 and 2.4)
New feature
Description:
Changed parallel I/O tests to use test script instead of just running
testphdf5 executable. This allows the MPI-posix driver to be tested easily.
Platforms tested:
FreeBSD 4.6 (sleipnir) w/parallel and IRIX64 6.5 (modi4) w/parallel
New feature
Description:
Add support in for "dmalloc" debugging malloc library to configure script
and library headers.
Platforms tested:
IRIX64 6.5 (modi4) and FreeBSD 4.6 (sleipnir)
Porting to AIX 5.x
Description:
These have been tested out in v1.4. Folding them in here.
aclocal.m4:
Snow's names were not included in the special hack.
Added its names in.
Removed the trailing * in pacific.llnl.gov too.
configure.in:
fortran/configure.in:
Added a case to group all AIX 5.* versions as aix5.x. This allows
them to share one common config/*aix5.x file.
configure:
fortran/configure:
Derived from configure.in (done in eirene).
config/powerpc-ibm-aix4.x:
Added a comment explaining why not to cache sizeof size_t and off_t
fortran/config/powerpc-ibm-aix4.x:
Turned of cache of sizeof size_t and off_t because they are depedent
on compiler options.
config/powerpc-ibm-aix5.x:
fortran/config/powerpc-ibm-aix5.x:
New config file for AIX 5.X
Platforms tests:
LLNL Blue, Frost, Snow, both serial and parallel.
