Code cleanup
Description:
Fix another batch of minor differences between the development and release
branches.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
Code cleanup
Description:
Clean up collective chunking code a bit.
Also, add '--enable-instrument' configure flag to have a mechanism for
determining that optimized operations happened correctly in the library (instead
of just the "normal" way) by allowing 'flag' properties to be set outside the
library and set when the "right" thing happens. This is mainly for debugging
and regression checks, so we make certain we don't break optimized I/O by
accident. It's enabled by default when --enable-debug is on (which is on by
default in the development branch and off by default in the release branch),
but can also be independently controlled with its own configure flag.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
IBM p690 (copper) w/parallel
h5diff and h5repack changes
Description:
h5diff
introduced the following four modes of output:
Normal mode: print the number of differences found and where they occured
Report mode: print the above plus the differences
Verbose mode: print the above plus a list of objects and warnings
Quiet mode: do not print output (h5diff always returns an exit code of 1 when differences are found)
h5repack
added an extra parameter for SZIP filter (coding method)
the new syntax is
-f SZIP=<pixels per block,coding>
(pixels per block is a even number in 2-32 and coding method is 'EC' or 'NN')
Example of use:
./h5repack -i file1 -o file2 -f SZIP=8,NN -v
updated usage messages, test scripts and files accordingly
Solution:
Platforms tested:
linux
AIX
solaris
Misc. update:
Purpose:
Bug Fix
Description:
If an HDF5 file grows larger than its address space, it dies and is unable to
write any data. This is more likely to happen since users are able to change
the number of bytes used to store addresses in the file.
Solution:
HDF5 now throws an error instead of dying. In addition, it "reserves" address
space for the local heap and for object headers (which do not allocate space
immediately). This ensures that after the error occurs, there is enough address
space left to flush the entire file to disk, so no data is lost.
A more complete explanation is at /doc/html/TechNotes/ReservedFileSpace.html
Platforms tested:
sleipnir, copper (parallel), verbena, arabica, Windows (Visual Studio 7)
Solution:
Platforms tested:
Misc. update:
h5repack changes
Description:
there were some requests to change some minor h5repack features
h5repack only made a warning about a non available filter in verbose mode ( -v )
without -v it kept silent, and users sometimes missed this warning
the request was that it should print this warning always. so, the new format, is e.g
./h5repack -i test_szip.h5 -o out.h5
Warning: dataset </dset_szip> cannot be read, SZIP filter is not available
due to this, and to avoid a lot of these messages in the shell test script, I modified
the script h5repack.sh so that it detects the presence of all filters in the environment
(previously it only detected SZIP)
the test files were also divided in more files , to make the script code easier to
follow
Solution:
Platforms tested:
linux
AIX (no szip)
solaris (no szip, no gzip )
Misc. update:
h5dump new tests
Description:
added new tests for the print of array indices (nested objects, several ranks)
Solution:
Platforms tested:
linux
AIX
solaris
Misc. update:
Bug fix
Description:
The "shared" raw B-tree node can get freed before all the B-tree nodes
had been flushed out to disk and released by the cache.
Solution:
Implement a simple reference counting wrapper for objects in the library
and use it to hold the shared raw B-tree nodes so they aren't freed before all
references to them in memory are released.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir)
IRIX64 6.5 (modei4)
h5dump new tests
Description:
added more tests for the escape/not escape feature for string data (with vlen, with
compound, with char data)
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
h5dump new tests
Description:
added new tests for the -p option, superblock, file contents, fill values, array indices.
Solution:
Platforms tested:
linux
AIX
solaris
Misc. update:
Description:
Replaced the old metadata cache with a cache with a modified LRU
replacement policy. This should improve the hit rate.
Solution:
Since we want to flush cache entries in increasing address order, I
used the threaded binary B-tree code to store the cache entries.
There is a fair bit of overhead here, so we may want to consider
other options.
While the code is designed to allow the support of other replacement
algorithms, at present, only a modified version of LRU is supported.
The modified LRU algorithm requires that a user selectable portion
of the cache entries be clean. The clean entries are evicted first
when writes are not permitted. If the pool of clean entries is used
up, the cache grows beyond its user specified maximum size. The
cache can also exceed its maximum size if the combined size of the
protected (or locked) entries exceeds the maximum size of the cache.
Platforms tested:
eirene (serial, parallel, fp), h5committested
Misc. update:
Purpose:
HDF5 now supports SZIP with no encoder.
Description:
SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled.
Solution:
Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC.
Platforms tested:
Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++)
Misc. update:
Description: Added new API H5Fget_name and new test program called filename.c. This function
returns the name of the file by object ID(file, group, dataset, named datatype, and attribute)
which belongs to the file.
Platforms tested: h5committest and fuss.
Misc. update: MANIFEST and RELEASE.txt
h5dump output change, new tests
Description:
the storage layout output format the storage layout output format had some changes
same for the user defined filter
add an option (-y) for not printing the array indices (default is print indices )
the option for escaping non printable characters covers all characters (default is not escape)
(this might be not very portable, the test files are tstring.ddl and tstringe.ddl )
add tests for the new options
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
Description: This is the second effort to correct XML dumper after adding null
dataspace test for attribute and dataset. Since XML schema hasn't been updated
for null space, took out null space test from tdset.h5 and tattr.h5 and put it
into a seperate file, tnullspace.h5. Only h5dump tests this null space file;
XML dumper doesn't do it at this moment. We'll wait until XML schema is updated
first.
Platforms tested: h5committest and RH 8(fuss)
Misc. update: MANIFEST(added two new files in tools/testfiles, tnullspace.h5
and tnullspace.ddl)
Description:
added the code for print strings with new line and display the path of references (new source files h5tools_ref.c and .h )
added a test suite in testh5dump.sh.in for
( note : to create testh5dump.sh , one must redo ./configure; this detects the availability of filters
and generates testh5dump.sh accordingly)
1) storage layout
2) fill value
3) print reference with path
4) print strings with new lines
5) filters
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
Update shell scripts
Description:
Switch to generating the testh5dump.sh script at configure time, so we can
determine which filters are available to test.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too small to require h5committest
h5dump new version
Description:
added the changes already made for 1.6
support for dumping of
1) filters
2) storage layout
3) fill value
4) comments
5) superblock
6) file contents
7) array indices
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
Code optimization & bug fix
Description:
When dimension information is being stored in the storage layout message
on disk, it is stored as 32-bit quantities, possibly truncating the dimension
information, if a dimension is greater than 32-bits in size.
Solution:
Fix the storage layout message problem by revising file format to not store
dimension information, since it is already available in the dataspace.
Also revise the storage layout data structures to be more compartmentalized
for the information for contiguous, chunked and compact storage.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
Solaris 2.7 (arabica)
h5committest
New Feature
Description:
Add the data transform function, H5Pset_transform().
Platforms tested:
"h5committested".
Copper was down. Ran parallel tests in sol instead.
Misc. update:
new tests for h5repack
Description:
added more tests both to the test program and shell script that test
a variation of different filter converssions
Solution:
Platforms tested:
linux
Misc. update:
new tests for h5repack
Description:
added tests that do layout type to layout type conversion in a matrix of 9 between compact, contiguous and chunking
Solution:
Platforms tested:
linux
afs has problems; I could not telnet to sol and copper, arabica is really slow (meaning
waiting 1 minute for a typed character) and the writing of a file gave an error
arabica 181% afs: failed to store file (145)
afs: failed to store file (145)
Misc. update:
1) new function for tools library
2) new test script for h5repack
Description:
1) currently all the tools (h5dump, h5diff, etc) do not check if a filter is available
for reading some dataset that might have a filter not available on the current configuration (the behaviour
of the tools until now was to trigger a library error, saying that the dataset cannot be read
due to the lack of the filter)
Solution:
1) added a new function h5tools_canreadf that checks if a dataset can be read
depending on the availability of filters.
this function was added in calls for h5diff and h5repack.
instead of triggering the library error, a message is printed, saying that the dataset
cannot be read (the print is optional, it is on on verbose mode)
2) added a shell script that tests the commannd line tool behaviour of h5repack
the script does a series of runs of h5repack with several options on the same file (this file test4.h5
was added to the testfiles dir).
then, it runs the h5diff tool, with the input and output files , in each run.
the goal of the test is also to check item 1) . the binary file was saved with filters
that might not be available on other configurations
Platforms tested:
linux (all filters enabled)
linux (some filters disabled)
solaris (some filters disabled)
AIX (some filters disabled)
windows (all filters on and off )
Misc. update:
Bug fix/optimization
Description:
Address slowdown in MPI-I/O file metadata operations that was introduced
mid-stream. We now _require_ a POSIX compliant parallel file system for the
MPI-I/O file driver (as well as for the MPI-POSIX file driver).
Also optimized file open operation when the file is being created by
reducing the number of collective & syncronizing calls.
Additionally, refactor the MPI routines into a common place, eliminating
duplicated code.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committest
Purpose: Maintenance
Description: Fortran APIs MAC OS X port for IBM XL Fortran compiler
Solution: Brought back changes from 1.6 branch
Platforms tested: pommier, h5comittested; this time h5committest
complained about copperpp directory and didn't run;
tests on verbena and sol passed.
Misc. update:
Code cleanup
Description:
Refactor library testing framework (used for the testhdf5 & ttsafe tests)
to remove almost all of the duplicated code, moving the common code into a
new 'testframe.c' source file.
Platforms tested:
FreeBSD 4.9 (sleipnir) w & w/o thread-safety
h5committest
h5diff new feature
Description:
added compare for attributes
a new options flag (-a) was added to the options structure. it is 0 by default (no compare )
the output of the compare is the same that for datasets, and all the other flags also apply for attributes
(the memory compare is done in the same function diff_array)
all the other requirements for compare of datasets (type, space) are identical too
Platforms tested:
linux
solaris 2.7
IRIX
Misc. update:
Code cleanup
Description:
Removed "H5Git" routines, now that there are library routines which perform
the same functionality.
Platforms tested:
FreeBSD 4.9 (sleipnir)
Linux 2.4 (verbena) w/FORTRAN
too minor for h5committest
Description: The standard output from Error API test has some non-standard
message like path name or line number.
Solution: use sed to remove any non-standard information in testerror.sh
Platforms tested: h5committest
to avoid printing error messages.
Description: If enable-hdf5v1_6 is configured in, make some functions
compatible with v1.6. Error test program print out some error messages as
it succeeds.
Solution: Use #ifdef H5_WANT_H5_V1_6_COMPAT statements. Use shell script
to compare error test output with the standard one.
Platforms tested: h5committest
Purpose:
Bug fix
Description:
pablo build failed.
Solution:
Fixed the makefile so that the needed header files are again created on the
fly.
Platforms tested:
modi4, copper, verbena, arabica
Misc. update:
Purpose: Bug fix.
Description: Parallel make fails because Pablo IDs are not assigned.
Solution: Extracted the Pablo IDs into header files and added them to the
pablo subdirectory instead of creating them on the fly.
Platforms tested: h5committested by Albert
Misc. update: Have updated MANIFEXT
This is an omibus update to the h5dump program.
1. Fixes bug 888
2. updates XML output to 1.6
Description:
Solution:
Platforms tested:
verbena, arabica, copper (64)
Misc. update:
MANIFEST updated
Need to do release notes.
Update
Description:
Revamped the configuration system. The configurations for the Fortran
and C++ libraries are no longer separate from the "main"
configuration system. This involved removing the "configure*" and
"aclocal.m4" files from the fortran/ and c++/ subdirectories. Also
merging settings in the config/ subdirectories into the main config/
subdirectory.
Fortran header files had to be modified a little for Linux. It was
checking if it was a Linux machine by some #defines, however with the
-std=c99 switch, these defines weren't there. I added a check for
some other ones which should be there whether the -std=c99 switch is
used or not.
Platforms tested:
Verbena (Fortran & C++)
Sol (Fortran & C++)
Copper (Fortran & C++)
Modi4 (Parallel, Fortran, & C++)
Misc. update:
New tool.
Description:
A new tool to build the HDF5 library. This was historically the
makeh5 command. Added here so that it can be used in machines outside
of the HDF Groups. Having it in the source allows version specific
customization.
Platforms tested:
Did run h5committest which failed due to other source code problems.
Tested it by hand by ../hdf5/bin/buildhdf5 and passed as far as its
own functionality.
Misc. update:
Purpose: Maintenance
Description: I added three new configuration files in c++/config
directory for Cray SV1, T3E and T90IEEE
Solution: Modified MANIFEST to add new files.
Platforms tested:
Misc. update:
Purpose: Maintenance for 1.6 release
Description: I added INSTALL_Windows_withf90.txt file that was lost
in 1.5 branch.
Solution: Modified MANIFEST to include new file.
Platforms tested:
Misc. update:
Code cleanup
Description:
Limit the scope on more function prototypes/macros/typedefs.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not necessary.
New test
Description:
Added test to check for dangling objects being handled correctly by the
library.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committested
New feature/enhancement
Description:
Chunked datasets are handled poorly in several circumstances involving
certain selections and chunks that are too large for the chunk cache and/or
chunks with filters, causing the chunk to be read from disk multiple times.
Solution:
Rearrange raw data I/O infrastructure to handle chunked datasets in a much
more friendly way by creating a selection in memory and on disk for each chunk
in a chunked dataset and performing all of the I/O on that chunk at one time.
There are still some scalability (the current code attempts to
create a selection for all the chunks in the dataset, instead of just the
chunks that are accessed, requiring portions of the istore.c and fillval.c
tests to be commented out) and performance issues, but checking this in will
allow the changes to be tested by a much wider audience while I address the
remaining issues.
Platforms tested:
h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
Purpose: Maintenance for T3E
Description: Cray updated T3E to run unicosmk2.0.6.X
Configuration failed.
Solution: Created new unicosmk2.0.6.X files in the config and fortran/config
directories and added them to MANIFEST
Platforms tested: Cray T3E
Misc. update:
Feature Add
Description:
It's now possible to print out characters as actual characters
instead of their decimal equivalent numbers. It's the same thing that
h5ls does. The flag to do this is "--string" or "-r".
Platforms tested:
Verbena (Fortran & C++)
Arabica (Fortran)
Modi4 (Fortran & Parallel)
Misc. update:
Purpose: Catching up with 1.4 branch
Description: 1.5 didn't have unicos10.0.X configuration files
for both C and fortran.
Solution: Added the files and updated MANIFEST
Platforms tested: bin.chmanifest on arabica
Misc. update:
Addition
Description:
Added "h5fc" utility. This is the same as the "h5cc" utility but for
Fortran. It only handles static libraries since that's the only
library we build for Fortran.
Note! I've tested the code on the normal platforms but I haven't
tested the utility itself.
Platforms tested:
Arabica (Fortran)
Modi4 (Fortran & Parallel)
Burrwhite (Fortran & C++
Misc. update:
Purpose: New tool
Description: Pankaj created h5import tool long time ago. I am finally checking
the source code in for the 1.6 release.
Known problems: h5import test will crash if srcdir option
is used; will be fixed before the release.
Solution:
Platforms tested: vebena, arabica, modi4 were tested. h5import tests passed
I also tested on arabica with srcdir option to make sure that
there is a problem when srcdir is used. After that I disabled
h5import tests. Daily tests will build h5import, but will not run
h5import tests.
Misc. update:
added test files for h5diff
Description:
2 binary hdf5 files, generated by h5difftst.c located in /tools/h5diff
1 of several text files to be added , that have a predifined h5diff output , to be compared with
the output of the program rum by testh5diff.sh
Solution:
Platforms tested:
Misc. update:
Purpose: Bug fix.
Description: When I added support for filters, I forgot to add
fortan man pages file to the MANIFEST file.
When I run chkmanifest, I discovered that there was also
a typo ( ./dac/Makefile.in instead of ./doc/Makefile.in )
Solution: added ./doc/html/fortran/h5z_FORTRAN.html and chnages "a" to "o"
Platforms tested: arabica with bin/chkmanifest
Misc. update:
Purpose: Catching up with C library
Description: Added new files for PGI compilers support on Linux 2.4
config/linux-gnuaout
fortran/config/linux-gnuaout
Added new source and test files for filters
fortran/src/H5Zf.c
fortran/src/H5Zff.f90
fortran/test/tH5Z.f90
Solution:
Platforms tested: arabica(C and F90), burrwhite (pgcc and pgf90), modi4 (F90 and parallel)
Misc. update:
Bug fix
Description:
"make distclean" is failing currently in the doc/html/ed_libs and
doc/html/ed_styles directories for lack of a 'Dependencies' file.
Solution:
Add it.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Misc. update:
Purpose:
Maintenance
Description:
Added support for variable length datatypes.
Solution:
I added support for Fortran VL types to 1.4 branch before the
hdf5-1.4.5 release. Now I am bringing changes back to the 1.5 branch
Platforms tested:
arabica (fortran), eirene (fortran), modi4 (parallel, fortran)
New internal feature
Description:
Add internal API for building and working with heaps (H5HP). This will be
used for the LRU algorithm in the new metadata cache code.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Code cleanup
Description:
Clean up miscellaneous warnings which have crept into the code.
Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD.
Adjust gcc compiler flags to be more concise for production mode.
Refactor the H5O code so that there is a stronger boundary between code
in the H5O package and code in the library which just calls H5O routines.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2
Misc. update:
Update MANIFEST if you add or remove any file.
Code cleanup.
Description:
Break up the ~9350 line H5T.c module into smaller pieces, which contain
code for a particular feature or support for a datatype class.
This should make the "main" H5T code (still in H5T.c) easier to support,
as well as removing some of the "minor" routines from the user applications
which don't use them (my rough estimates show about 4% reduction (~30K on
a FreeBSD machine) in optimized, staticly-linked binaries for very simple
programs)
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Misc. update:
Update MANIFEST
New feature for developers.
Description:
Added "function stack" tracing to library. This allows developers (there
is no public API) to call H5FS_print within the library and get a listing
of the functions traversed to reach that point in the library. Eventually,
I may add support for reporting the parameters to each function also...
Mainly for debugging parallel I/O programs, but I think it will come in
handy in other cases also.
The function stack tracking is controlled with a configure switch:
--enable-funcstack, which defaults to enabled currently. When we branch
for 1.6, we should change the default setting on the branch to be disabled.
Also, added a destructor to the thread-specific keys when thread-safety is
turned on in the library. Otherwise, they were leaking memory and causing
difficult to debug errors in threaded programs (like the test/ttsafe test).
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) w/thread-safety enabled.
Misc. update:
Updated MANIFEST with new files added (src/H5FS.c & src/H5FDprivate.h)
Update release_docs/RELEASE with thread-safety bug fix.
Update
Description:
MANIFEST:
- Added .autom4te.cfg and H5FDfphdf5.[ch] files.
.autom4te.cfg
- When the new versions of libtool/autoconf come out, this file
will help get rid of the annoying autom4te.cache directory.
Platforms tested:
Linux
Bug fix
Description:
Currently, when the library encounters an object header message that isn't
know, it fails to open that object in the file.
Solution:
Allow the library to skip over the unknown object header message and
continue to process the remaining messages, in the hope that the skipped
message isn't important later. If it is important, it will be caught at
a higher level of the library.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
fortran/config/powerpc-ibm-aix5.x
To cover AIX 5.x.
Removed Files:
fortran/config/powerpc-ibm-aix4.2.1.0
fortran/config/powerpc-ibm-aix4.3.2.0
These files are outdated and have been replaced by powerpc-ibm-aix4.x.
Modified Files:
MANIFEST
Update it.
Platforms tested:
Only on LLNL blue. LLNL SP are the only machines that use
these files and that I have access.
Update
Description:
Added new files to the MANIFEST and added a new configure switch for
enabling building of the Flexible parallel HDF5 module.
Platforms tested:
Modi4 Eirene Arabica
Purpose:
Bug fix, code improvement
Description:
Fortran tests didn't cleanup created files. Also HDF5_PREFIX and
HDF5_PARAPREFIX were not used to specify location of the files.
There was a redundant file in the testpar directory that contained
an error reporting function used by both serial and parallel tests.
Solution:
Created library h5test_fortran.a that contains functions used by the
serial and parallel tests. It includes Fortran and C functions that
may be called from Fortran programs to report errors, to modify
file names and to cleanup files after run. Modified test code to
use new functions.
Platforms tested:
Linux 2.2 (eirene) serial
IRIX64 (modi4) parallel with HDF5_PREFIX and HDF5_PARAPREFIX set to $SCR
Solris 2.7 with mpich 1.2.4 with HDF5_PARAPREFIX set to /tmp/epourmal
Purpose:
Design for compact dataset
Description:
Compact dataset is stored in the header message for dataset layout.
Platforms tested:
arabica, eirene.
Code cleanup
Description:
Move get/set routines for each type of property list (file creation,
dataset creation, file access and dataset transfer) into their own source
code module.
Platforms tested:
FreeBSD 4.6 (sleipnir)
Compress I/O Test Add
Description:
This is the initial check-in of the compress I/O test. It will write
out compressed buffers to a file.
I need to implement a write of uncompressed data first for
comparison...
Platforms tested:
Linux
Update
Description:
Updated the way we used the AC_INIT macros so that it's current to
the new standard way and not deprecated anymore.
Also, added a test for some functions during parallel Fortran
configure. This required the creation of an H5config_fortran.h file
so that we can test for these...
Platforms tested:
Elena is going to test the changes to the fortran after she checks in
her other code. I will test on SDSC machines after getting her code.
The other changes were tested on Linux.
Autotools Update
Description:
I've updated autoconf, automake, and libtool to the latest/greatest
versions; 2.53, 1.6, and 1.4.2 resp.
Many changes come with the new versions:
- ltconfig is no longer used
- acconfig.h is no longer used (#define values are declared with
the macro)
- regeneration of all of the aclocal.m4, configure, and
H5config.h.in files.
- new config.{guess,sub} files
- new ltmain.sh file
Platforms tested:
AIX (blue), and Linux
Purpose:
Feature Add
Description:
Added support for dumping Group Comments. This involved a
modification of the DDL as well.
Solution:
Steal code from h5ls and put it in the h5dump. The ddl.html file was
updated as normal. And a test was created...
Platforms tested:
Dangermouse, Modi4, Kelgia
Removing the DPSS (gridstorage) driver source code.
Description:
The DPSS (using Grid-Storage) driver is retired.
Removed the configure option with-gridstorage from configure.in.
Cvs remove the following files
./src/H5FDdpss.c
./src/H5FDdpss.h
./test/dpss_read.c
./test/dpss_write.c
Regenerated Dependencies files (some had to be hand-edited since
'make depend' did not cover them.)
Removed reference to DPSS Virtual file driver from H5F.c.
Platforms tested:
modi4 (Parallel; -with-gass=...), eirene, arabica (fortran, cxx).
Purpose:
Feature Add
Description:
Added "install-example" and "install-all" to the Makefile system.
The behaviour of the "make install*" options:
make install - Installs binaries, libraries, include
files, and example programs.
make install-examples - Installs only the example programs.
The directories are:
${prefix}/doc/hdf5/examples/{c,c++,fortran}
make install-all - Install the binaries, libraries, include
files, example programs, and
documentation. The whole kit-n'-caboodle.
make uninstall-examples - Get rid of those example files (but not
the ${prefix}/doc/hdf5/examples/...
directories)
There's a new bin/ program which helps create directories which are
deeply nested called "mkdirs". It's a simple shell script.
Platforms tested:
Linux
New feature
Description:
Added perform programs to test the HDF5 library performance. Programs
are installed in directory perform/.
Platforms tested:
eirene
Purpose:
Bug Fix
Description:
The way we were generating Dependencies and .depend files was broken.
If the $srcdir or other macros began with a ".", then it would match
anything and cause problems since it would then overwrite the
beginning of the header file's path.
Solution:
Wrote a Perl script which can handle this type of weirdness better.
It's only used when the environment is a GNU one with a GCC
compiler...
Platforms tested:
Linux
Purpose:
Maintenance
Description:
Source directory has been rearranged. INSTALL*, HISTORY.txt and RELEASE.txt were moved to the release_docs directory. *.zip files were moved to the windows directory. README file was renamed to README.txt MANIFEST was updated to reflect those changes.
Purpose:
Feature
Description:
Changed Pablo source code to allow the user to build the instrumented
library to link with either the Trace Libraries or Pablo PCF.
Solution:
Added conditionally compiled code to routines in the pablo subdirectory.
Also added a new file PabloHDF_MPIO.c containing all of the MPI I/O code
for better code structure.
Platforms tested:
Solaris 32-bit, Irix 64
Purpose:
Adding two new files (doc/html/fortran/F90Flags.html, and
doc/html/fortran/h5_FORTRAN.html) to FORTRAN portion of
reference manual.
Platforms tested:
IE 5
