Purpose: Maintenance
Description: Fortran APIs MAC OS X port for IBM XL Fortran compiler
Solution: Brought back changes from 1.6 branch
Platforms tested: pommier, h5comittested; this time h5committest
complained about copperpp directory and didn't run;
tests on verbena and sol passed.
Misc. update:
Code cleanup
Description:
Refactor library testing framework (used for the testhdf5 & ttsafe tests)
to remove almost all of the duplicated code, moving the common code into a
new 'testframe.c' source file.
Platforms tested:
FreeBSD 4.9 (sleipnir) w & w/o thread-safety
h5committest
h5diff new feature
Description:
added compare for attributes
a new options flag (-a) was added to the options structure. it is 0 by default (no compare )
the output of the compare is the same that for datasets, and all the other flags also apply for attributes
(the memory compare is done in the same function diff_array)
all the other requirements for compare of datasets (type, space) are identical too
Platforms tested:
linux
solaris 2.7
IRIX
Misc. update:
Code cleanup
Description:
Removed "H5Git" routines, now that there are library routines which perform
the same functionality.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Linux 2.4 (verbena) w/FORTRAN
too minor for h5committest
Description: The standard output from Error API test has some non-standard
message like path name or line number.
Solution: use sed to remove any non-standard information in testerror.sh
Platforms tested: h5committest
to avoid printing error messages.
Description: If enable-hdf5v1_6 is configured in, make some functions
compatible with v1.6. Error test program print out some error messages as
it succeeds.
Solution: Use #ifdef H5_WANT_H5_V1_6_COMPAT statements. Use shell script
to compare error test output with the standard one.
Platforms tested: h5committest
Purpose:
Bug fix
Description:
pablo build failed.
Solution:
Fixed the makefile so that the needed header files are again created on the
fly.
Platforms tested:
modi4, copper, verbena, arabica
Misc. update:
Purpose: Bug fix.
Description: Parallel make fails because Pablo IDs are not assigned.
Solution: Extracted the Pablo IDs into header files and added them to the
pablo subdirectory instead of creating them on the fly.
Platforms tested: h5committested by Albert
Misc. update: Have updated MANIFEXT
This is an omibus update to the h5dump program.
1. Fixes bug 888
2. updates XML output to 1.6
Description:
Solution:
Platforms tested:
verbena, arabica, copper (64)
Misc. update:
MANIFEST updated
Need to do release notes.
Update
Description:
Revamped the configuration system. The configurations for the Fortran
and C++ libraries are no longer separate from the "main"
configuration system. This involved removing the "configure*" and
"aclocal.m4" files from the fortran/ and c++/ subdirectories. Also
merging settings in the config/ subdirectories into the main config/
subdirectory.
Fortran header files had to be modified a little for Linux. It was
checking if it was a Linux machine by some #defines, however with the
-std=c99 switch, these defines weren't there. I added a check for
some other ones which should be there whether the -std=c99 switch is
used or not.
Platforms tested:
Verbena (Fortran & C++)
Sol (Fortran & C++)
Copper (Fortran & C++)
Modi4 (Parallel, Fortran, & C++)
Misc. update:
New tool.
Description:
A new tool to build the HDF5 library. This was historically the
makeh5 command. Added here so that it can be used in machines outside
of the HDF Groups. Having it in the source allows version specific
customization.
Platforms tested:
Did run h5committest which failed due to other source code problems.
Tested it by hand by ../hdf5/bin/buildhdf5 and passed as far as its
own functionality.
Misc. update:
Purpose: Maintenance
Description: I added three new configuration files in c++/config
directory for Cray SV1, T3E and T90IEEE
Solution: Modified MANIFEST to add new files.
Platforms tested:
Misc. update:
Purpose: Maintenance for 1.6 release
Description: I added INSTALL_Windows_withf90.txt file that was lost
in 1.5 branch.
Solution: Modified MANIFEST to include new file.
Platforms tested:
Misc. update:
Code cleanup
Description:
Limit the scope on more function prototypes/macros/typedefs.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not necessary.
New test
Description:
Added test to check for dangling objects being handled correctly by the
library.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committested
New feature/enhancement
Description:
Chunked datasets are handled poorly in several circumstances involving
certain selections and chunks that are too large for the chunk cache and/or
chunks with filters, causing the chunk to be read from disk multiple times.
Solution:
Rearrange raw data I/O infrastructure to handle chunked datasets in a much
more friendly way by creating a selection in memory and on disk for each chunk
in a chunked dataset and performing all of the I/O on that chunk at one time.
There are still some scalability (the current code attempts to
create a selection for all the chunks in the dataset, instead of just the
chunks that are accessed, requiring portions of the istore.c and fillval.c
tests to be commented out) and performance issues, but checking this in will
allow the changes to be tested by a much wider audience while I address the
remaining issues.
Platforms tested:
h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
Purpose: Maintenance for T3E
Description: Cray updated T3E to run unicosmk2.0.6.X
Configuration failed.
Solution: Created new unicosmk2.0.6.X files in the config and fortran/config
directories and added them to MANIFEST
Platforms tested: Cray T3E
Misc. update:
Feature Add
Description:
It's now possible to print out characters as actual characters
instead of their decimal equivalent numbers. It's the same thing that
h5ls does. The flag to do this is "--string" or "-r".
Platforms tested:
Verbena (Fortran & C++)
Arabica (Fortran)
Modi4 (Fortran & Parallel)
Misc. update:
Purpose: Catching up with 1.4 branch
Description: 1.5 didn't have unicos10.0.X configuration files
for both C and fortran.
Solution: Added the files and updated MANIFEST
Platforms tested: bin.chmanifest on arabica
Misc. update:
Addition
Description:
Added "h5fc" utility. This is the same as the "h5cc" utility but for
Fortran. It only handles static libraries since that's the only
library we build for Fortran.
Note! I've tested the code on the normal platforms but I haven't
tested the utility itself.
Platforms tested:
Arabica (Fortran)
Modi4 (Fortran & Parallel)
Burrwhite (Fortran & C++
Misc. update:
Purpose: New tool
Description: Pankaj created h5import tool long time ago. I am finally checking
the source code in for the 1.6 release.
Known problems: h5import test will crash if srcdir option
is used; will be fixed before the release.
Solution:
Platforms tested: vebena, arabica, modi4 were tested. h5import tests passed
I also tested on arabica with srcdir option to make sure that
there is a problem when srcdir is used. After that I disabled
h5import tests. Daily tests will build h5import, but will not run
h5import tests.
Misc. update:
added test files for h5diff
Description:
2 binary hdf5 files, generated by h5difftst.c located in /tools/h5diff
1 of several text files to be added , that have a predifined h5diff output , to be compared with
the output of the program rum by testh5diff.sh
Solution:
Platforms tested:
Misc. update:
Purpose: Bug fix.
Description: When I added support for filters, I forgot to add
fortan man pages file to the MANIFEST file.
When I run chkmanifest, I discovered that there was also
a typo ( ./dac/Makefile.in instead of ./doc/Makefile.in )
Solution: added ./doc/html/fortran/h5z_FORTRAN.html and chnages "a" to "o"
Platforms tested: arabica with bin/chkmanifest
Misc. update:
Purpose: Catching up with C library
Description: Added new files for PGI compilers support on Linux 2.4
config/linux-gnuaout
fortran/config/linux-gnuaout
Added new source and test files for filters
fortran/src/H5Zf.c
fortran/src/H5Zff.f90
fortran/test/tH5Z.f90
Solution:
Platforms tested: arabica(C and F90), burrwhite (pgcc and pgf90), modi4 (F90 and parallel)
Misc. update:
Bug fix
Description:
"make distclean" is failing currently in the doc/html/ed_libs and
doc/html/ed_styles directories for lack of a 'Dependencies' file.
Solution:
Add it.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Misc. update:
Purpose:
Maintenance
Description:
Added support for variable length datatypes.
Solution:
I added support for Fortran VL types to 1.4 branch before the
hdf5-1.4.5 release. Now I am bringing changes back to the 1.5 branch
Platforms tested:
arabica (fortran), eirene (fortran), modi4 (parallel, fortran)
New internal feature
Description:
Add internal API for building and working with heaps (H5HP). This will be
used for the LRU algorithm in the new metadata cache code.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Code cleanup
Description:
Clean up miscellaneous warnings which have crept into the code.
Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD.
Adjust gcc compiler flags to be more concise for production mode.
Refactor the H5O code so that there is a stronger boundary between code
in the H5O package and code in the library which just calls H5O routines.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2
Misc. update:
Update MANIFEST if you add or remove any file.
Code cleanup.
Description:
Break up the ~9350 line H5T.c module into smaller pieces, which contain
code for a particular feature or support for a datatype class.
This should make the "main" H5T code (still in H5T.c) easier to support,
as well as removing some of the "minor" routines from the user applications
which don't use them (my rough estimates show about 4% reduction (~30K on
a FreeBSD machine) in optimized, staticly-linked binaries for very simple
programs)
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Misc. update:
Update MANIFEST
New feature for developers.
Description:
Added "function stack" tracing to library. This allows developers (there
is no public API) to call H5FS_print within the library and get a listing
of the functions traversed to reach that point in the library. Eventually,
I may add support for reporting the parameters to each function also...
Mainly for debugging parallel I/O programs, but I think it will come in
handy in other cases also.
The function stack tracking is controlled with a configure switch:
--enable-funcstack, which defaults to enabled currently. When we branch
for 1.6, we should change the default setting on the branch to be disabled.
Also, added a destructor to the thread-specific keys when thread-safety is
turned on in the library. Otherwise, they were leaking memory and causing
difficult to debug errors in threaded programs (like the test/ttsafe test).
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) w/thread-safety enabled.
Misc. update:
Updated MANIFEST with new files added (src/H5FS.c & src/H5FDprivate.h)
Update release_docs/RELEASE with thread-safety bug fix.
Update
Description:
MANIFEST:
- Added .autom4te.cfg and H5FDfphdf5.[ch] files.
.autom4te.cfg
- When the new versions of libtool/autoconf come out, this file
will help get rid of the annoying autom4te.cache directory.
Platforms tested:
Linux
Bug fix
Description:
Currently, when the library encounters an object header message that isn't
know, it fails to open that object in the file.
Solution:
Allow the library to skip over the unknown object header message and
continue to process the remaining messages, in the hope that the skipped
message isn't important later. If it is important, it will be caught at
a higher level of the library.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
fortran/config/powerpc-ibm-aix5.x
To cover AIX 5.x.
Removed Files:
fortran/config/powerpc-ibm-aix4.2.1.0
fortran/config/powerpc-ibm-aix4.3.2.0
These files are outdated and have been replaced by powerpc-ibm-aix4.x.
Modified Files:
MANIFEST
Update it.
Platforms tested:
Only on LLNL blue. LLNL SP are the only machines that use
these files and that I have access.
Update
Description:
Added new files to the MANIFEST and added a new configure switch for
enabling building of the Flexible parallel HDF5 module.
Platforms tested:
Modi4 Eirene Arabica
Purpose:
Bug fix, code improvement
Description:
Fortran tests didn't cleanup created files. Also HDF5_PREFIX and
HDF5_PARAPREFIX were not used to specify location of the files.
There was a redundant file in the testpar directory that contained
an error reporting function used by both serial and parallel tests.
Solution:
Created library h5test_fortran.a that contains functions used by the
serial and parallel tests. It includes Fortran and C functions that
may be called from Fortran programs to report errors, to modify
file names and to cleanup files after run. Modified test code to
use new functions.
Platforms tested:
Linux 2.2 (eirene) serial
IRIX64 (modi4) parallel with HDF5_PREFIX and HDF5_PARAPREFIX set to $SCR
Solris 2.7 with mpich 1.2.4 with HDF5_PARAPREFIX set to /tmp/epourmal
Purpose:
Design for compact dataset
Description:
Compact dataset is stored in the header message for dataset layout.
Platforms tested:
arabica, eirene.
Code cleanup
Description:
Move get/set routines for each type of property list (file creation,
dataset creation, file access and dataset transfer) into their own source
code module.
Platforms tested:
FreeBSD 4.6 (sleipnir)
Compress I/O Test Add
Description:
This is the initial check-in of the compress I/O test. It will write
out compressed buffers to a file.
I need to implement a write of uncompressed data first for
comparison...
Platforms tested:
Linux
Update
Description:
Updated the way we used the AC_INIT macros so that it's current to
the new standard way and not deprecated anymore.
Also, added a test for some functions during parallel Fortran
configure. This required the creation of an H5config_fortran.h file
so that we can test for these...
Platforms tested:
Elena is going to test the changes to the fortran after she checks in
her other code. I will test on SDSC machines after getting her code.
The other changes were tested on Linux.
Autotools Update
Description:
I've updated autoconf, automake, and libtool to the latest/greatest
versions; 2.53, 1.6, and 1.4.2 resp.
Many changes come with the new versions:
- ltconfig is no longer used
- acconfig.h is no longer used (#define values are declared with
the macro)
- regeneration of all of the aclocal.m4, configure, and
H5config.h.in files.
- new config.{guess,sub} files
- new ltmain.sh file
Platforms tested:
AIX (blue), and Linux
Purpose:
Feature Add
Description:
Added support for dumping Group Comments. This involved a
modification of the DDL as well.
Solution:
Steal code from h5ls and put it in the h5dump. The ddl.html file was
updated as normal. And a test was created...
Platforms tested:
Dangermouse, Modi4, Kelgia
Removing the DPSS (gridstorage) driver source code.
Description:
The DPSS (using Grid-Storage) driver is retired.
Removed the configure option with-gridstorage from configure.in.
Cvs remove the following files
./src/H5FDdpss.c
./src/H5FDdpss.h
./test/dpss_read.c
./test/dpss_write.c
Regenerated Dependencies files (some had to be hand-edited since
'make depend' did not cover them.)
Removed reference to DPSS Virtual file driver from H5F.c.
Platforms tested:
modi4 (Parallel; -with-gass=...), eirene, arabica (fortran, cxx).
Purpose:
Feature Add
Description:
Added "install-example" and "install-all" to the Makefile system.
The behaviour of the "make install*" options:
make install - Installs binaries, libraries, include
files, and example programs.
make install-examples - Installs only the example programs.
The directories are:
${prefix}/doc/hdf5/examples/{c,c++,fortran}
make install-all - Install the binaries, libraries, include
files, example programs, and
documentation. The whole kit-n'-caboodle.
make uninstall-examples - Get rid of those example files (but not
the ${prefix}/doc/hdf5/examples/...
directories)
There's a new bin/ program which helps create directories which are
deeply nested called "mkdirs". It's a simple shell script.
Platforms tested:
Linux
