new feature
Description:
Added a force mode (-force) which turns off prompting, just do it.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? no
[If no, why not?] It is just a shell script. Tested it in eirene.
Misc. update:
Will check in the updated RELEASE and document files.
h5diff bugs and enhancements
Description:
added checking for not comparable datatypes
added checking for the same dimensionality
changed the functionality of -n option, to check up to n reported differences
changed the formula for -p to relative error with absolute value
fixed a bug on solaris, related to printf integer format (int datatype)
Platforms tested:
w2000, linux, solaris
removed -d , -g , -t and -v options and renamed -m to -d and -v to -l
added output for objects that are not in one file but in the other
Platforms tested:
windows, linux
Code cleanup
Description:
Clean up miscellaneous warnings which have crept into the code.
Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD.
Adjust gcc compiler flags to be more concise for production mode.
Refactor the H5O code so that there is a stronger boundary between code
in the H5O package and code in the library which just calls H5O routines.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2
Misc. update:
Update MANIFEST if you add or remove any file.
Bug Fix (Bug #806)
Description:
When compiling/installing with the command "make install" (without
performing a "make" beforehand) the tools weren't being made. This
was because the `install' command in the Makefiles only relied upon
public libraries, not private ones (which the one in the tools
directory is).
Solution:
Had the `install' command be dependent upon an "AUX_LIB" macro which
includes libraries which are needed but aren't distributed (that is,
they're staticly compiled).
Platforms tested:
Linux
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
bug fix
added more verbose help message
added support for all dataset types
added more error checking
added new output format (28 Jan version )
Platforms tested:
windows , linux
patch
Description:
Tflops interprets "$@" as "" when no parameter is given (e.g., the
case of missing file name). Changed it to use $@ till Tflops fixes it.
Platforms tested:
Tflops, burrwhite (pp), modi4(pp)
Code cleanup & bug fix
Description:
Cleanup another set of warnings on Windows and also fix mis-placed assertion
that caused the daily tests to fail.
Platforms tested:
IRIX64 6.5 (modi4) w/-n32
Code cleanup
Description:
Various code cleanups to allow the development branch to be compiled with
a C++ compiler (i.e. CC=g++ )
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) C++
Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
Purpose:
bug fix
Description:
Need to use HDopen, HDread for windows support.
off_t in windows is 32-bit; however, windows can support 64-bit file. So we
have to change off_t to int64 in order that >4G file can be supported.
Solution:
See above
Platforms tested:
linux2.2.18smp, windows 2000
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
2002-12-03 23:00:35 Robb Matzke <matzke@arborea.spizella.com>
* main: Replaced the H5LS_PREPEND_FILENAME compile-time symbol which was
always defined, with a runtime decision. If h5ls is
invoked with more than one argument then the file name
is displayed as part of the object name, otherwise the
file name is not displayed.
Renamed h5dumptst.c to h5dumpgentest.c.
Added a test to run h5dumpgentest to make it can run without error. Note
that this does not verify if it is generating the correct test files.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? YES
Misc. update:
Update MANIFEST if you add or remove any file. YES
Changed the file name nofilename.ddl to tnofilename.ddl, to be consistent
with the other file names.
Platforms tested:
eirene only since this is a very simple change.
Improvement
Description:
Moved the code parsing for missing file name in the parse_command_line
routine.
Added a test checking if the tool fails as expected when no file name is
given.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? YES
Misc. update:
Update MANIFEST if you add or remove any file. YES, added nofilename.ddl
Bug fix (ID 765 and part of ID 841)
Description:
"h5dump -H" would core dump. It was because the code was not
checking if there was any file argument after option parsing.
It referred to an non-existing filename argument.
There was also some very old but not commented code which allowed
the use of back slash to escape a filename begun with '-'. The
'--' option is better and more common practice.
Solution:
Added code to check if there is still a filename argument after parsing.
(This checking probably should goto inside the option_parsing routine.
Also removed that old code mentioned above.
Platforms tested:
eirene, arabica and modi4
updated copyright note.
Copied the UNUSED correction from v1.4.
Added an example to the usage of -- option.
Description:
[describe the bug, or describe the new feature, etc]
Solution:
[details about the changes, algorithm, etc...]
[Please as detail as you can since your own explanation is
better than others guessing it from the code.]
Platforms tested:
arabica (fortran), modi4 (pp, fortran),eirene (fortran, CC).
Possible Bug Fix
Description:
There seems to be a problem with the h5cc script picking up old
header files. This may have something to do with the CPPFLAGS macro
being set to some nefarious thing and the compiler looking in those
directories for the header files instead of the one where HDF5 is
actually installed.
Solution:
Put the -I$includedir first.
Platforms tested:
None...This is really triggered by the nightly tests. I'm kind of
hoping that this will fix things...
Bug fix
Description:
Old setup put $(srcdir)/h5redeploy in the install list but this
same list is used for clean too, thus the file is removed from
source. That is bad.
Solution:
Set it to use a cp to do it. Need to change the source version
name to something else because for one, it is confusing to use
the same name. For another, if the build is done in place (i.e.,
not using --srcdir), the source file is removed, again.
So, renamed it with the .in suffix. Who knows, it may take
more processing than just cp when more features are added to it.
Platforms tested:
Eirene
Bug Fix
Description:
On AIX machines, the number of script interpreters which ran the
testh5dump.sh script would strip too many of the quotes away, causing
problems if you're quoting something with a space in it.
Solution:
Just got rid of the spaces and used ',' instead.
Platforms tested:
AIX and Linux
Purpose:
user reported a bug on h5 to gif tool
Description:
h5 to gif did not read an HDF5 image correctly
Solution:
updated the program so that it reads the updated HDF5 image format to the latest specification
the debug version of MSVC was giving a failure on the write of the GIF file,
regarding the GIF file pointer
this write error is eliminated declaring that pointer a global variable
there is still an applicattion error on the exit of the program, only on the debug version of MSVC
Platforms tested:
w2000
Migrate from configure macros of XYZ_ABC to H5_XYZ_ABC
Description:
configure generates many macros definitions on the fly and
were stored in src/H5config.h which is included by H5public.h.
But other software that uses hdf5 may also run their own configure.
There can be a clash in macro name space. We decided awhile ago
to prepend all generated macros with "H5_" to avoid conflicts.
The process has started and this commit completes it (at least attempt
to).
Solution:
Many macros symbols (e.g. SIZEOF_xxx and HAVE_xxx were changed to
H5_SIZEOF_xxx and H5_HAVE_xxx). Then H5private.h no longer includes
H5config.h. This cuts H5config.h away from HDF5 source code.
Pending issues:
The module of fortran and pablo are to be resolved in a different
commit.
Platforms tested:
eirene (parallel), arabica (solaris 7 --enable-fortran, --enable-cxx)
Minor h5ls bugfix for compound types with array members.
(Similar changes shortly for 1.4 branch).
2002-04-04 16:39:09 Robb Matzke <matzke@arborea.spizella.com>
* display_cmpd_type: Removed code to display array dimensions from display_cmpd_type()
since this was a holdover from the days when hdf5
didn't have an array datatype and was duplicated by
display_array_type().
Code removal
Description:
Removed the HDF4 source files from the HDF5 tree. The directories
will remain. Use the "-P" option when doing a cvs checkout or update
to "prune" the empty directories from your personal tree.
Code Motion
Description:
Removal of HDF4 from the configure/Makefiles. This is a precursor to
the actual physical removal of the HDF4 tools from the HDF5 tree.
Platforms tested:
Arabica, Dangermouse
Bug Fix
Description:
Some -I paths weren't included in the h5cc script. That would cause
the compiler to fail if it was trying to find gass header files or
the like.
Solution:
Added the CPPFLAGS macro to the h5cc.in file so that it'll be there
when it's generated. This will also include some -D options which we
compiled the library with, like the LFS flags on Linux.
Also changed the configure* files so that it will "chmod" the created
h5cc file to 755 (executable) since that wasn't happening all the
time...
Platforms tested:
Linux
Bug Fix
Description:
There was a problem with having a lot of groups nested together. We
could only handle 1024 characters at most, but, in a parallel program
especially, it could occur that there were lots and lots of groups
and would be more than 1024.
Solution:
I made the "objname" part of the obj_t structure a pointer instead of
a fixed size. Added code to allocate/deallocate the memory we need
for it. Had to fix how the "prefix" was being handled in the h5dump
program. It was also set to only 1024 characters in length. I made it
dynamic.
Added a test case...Go me!
Platforms tested:
Linux, Solaris
Oops
Description:
I added files for testing the group comments dumping feature, but
didn't actually add it to the testh5dump.sh script.
Solution:
Added it.
Platforms tested:
Linux
Bug Fix
Description:
The library path was relying upon the "exec_prefix" variable.
However, we weren't including that into the h5cc script.
Solution:
Added it.
Platforms tested:
Linux
Purpose:
Small Fix
Description:
Changed how the list of drivers were listed from:
{ "driver1",
"driver2",
/* ... */
"drivern",
#ifdef FOO
"driverm"
#endif
};
to
{ "driver1"
,"driver2"
/* ... */
,"drivern"
#ifdef FOO
,"driverm"
#endif
};
since it's a nicer way of doing the same thing without the annoying
warning of an extraneous , if FOO isn't defined.
Platforms tested:
Dangermouse, Modi4, Kelgia
Purpose:
Feature Add
Description:
Added support for dumping Group Comments. This involved a
modification of the DDL as well.
Solution:
Steal code from h5ls and put it in the h5dump. The ddl.html file was
updated as normal. And a test was created...
Platforms tested:
Dangermouse, Modi4, Kelgia
Purpose:
Bug fix
Description:
The "make depend" command wasn't propagating down into the tools/
directories.
Solution:
Added the "depend:" part to the Makefile in the tools/ subdirectory.
Platforms tested:
Linux
Removing the DPSS (gridstorage) driver source code.
Description:
The DPSS (using Grid-Storage) driver is retired.
Removed the configure option with-gridstorage from configure.in.
Cvs remove the following files
./src/H5FDdpss.c
./src/H5FDdpss.h
./test/dpss_read.c
./test/dpss_write.c
Regenerated Dependencies files (some had to be hand-edited since
'make depend' did not cover them.)
Removed reference to DPSS Virtual file driver from H5F.c.
Platforms tested:
modi4 (Parallel; -with-gass=...), eirene, arabica (fortran, cxx).
Bug Fix.
Description:
The H5Rget_object_type function could not get the object type for dataset
region references.
Solution:
Added a new function, H5Rget_obj_type, to replace H5Rget_object_type.
The new function requires the reference type as an additional parameter,
in order to allow queries on different reference types to be performed
correctly.
Platforms tested:
FreeBSD 4.4. (sleipnir)
Purpose:
parameter file
Description:
handle parameters in various special cases.
Users donot even have to use this file if they don't have those special needs.
1) memory optimization (should always be set to 0 without handling huge SDS array conversion)
2) unlimited dimension with zero current size, users can use this file to define
their chunk size.
Solution:
Platforms tested:
RedHat Zoot 6.2
Purpose:
a bug fix
a feature added
Description:
1.conversion of unlimited dimension data with the current dimensional size 0
2. Use a parameter file to control some special cases:
1) To subdivide the huge array into hyperslabs, a memory buffer size has i
to be set.
2) when current dimensional size is 0, a default chunk size can be set.
Solution:
bug fixed 1. Old approach: the current dimensional size is set to H5S_UNLIMITED, which is
a huge number. The default chunk size is set as a *FIXED* default value.
2. New approach: the current dimensional size is set to the current value 0.
Users can provide the chunk size through a parameter file.
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
bug fixed
Description:
HDF4 Vdata is extensible, when converted, the new HDF4 data set should also be extensible.
Solution:
Make the HDF4 dataset converted from Vdata unlimited dimensions.
These three testing files should be updated accordingly.
Platforms tested:
Linux, RedHat 6.2
Backward Compatibility Fix
Description:
One of H5P[gs]et_buffer's parameters changed between v1.4 and the
development branch.
Solution:
Added v1.4 compat stuff around H5P[gs]et_buffer implementation and testing
to allow v1.4.x users to continue to use their source code without
modification.
These changes are for everything except the FORTRAN wrappers - I spoke with
Elena and she will make the FORTRAN wrapper changes.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Code cleanup
Description:
Windows is generating hundreds of warnings from some of the practices in
the library. Mostly, they are because size_t is 32-bit and hsize_t is
64-bit on Windows and we were carelessly casting the larger values down to
the smaller ones without checking for overflow.
Also, some other small code cleanups,etc.
Solution:
Re-worked some algorithms to eliminate the casts and also added more
overflow checking for assignments and function parameters which needed
casts.
Kent did most of the work, I just went over his changes and fit them into
the the library code a bit better.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Purpose:
A new feature
Description:
While testing h4toh5 utility with real NASA files, we find an example that the data array(one SDS) is so big that it exceeds the physical memory of some machine(>128 MB) and the conversion failed. Before the smart hyperslab operation is out, I am dividing the whole SDS into smaller hyperslabs with each hyperslab propotational to the original SDS array dimensions. For example, a three dimension array with 1000*1000*1000 elements, I can divide them into eight 500*500*500 pieces. I can read and write each piece and remember their starting and ending points. In this way, the memory allocation failure can be avoided; however, it may not be the efficient way.
I've tested this feature using SDS without chunking. It works fine. However, when testing SDS with chunking, it is extremely slow. This happens to be a bug in HDF5 library now. Quincey may fix this later and give me a more efficient way to handle the problem. Currently all my testing files are with UNLIMITED dimensions, so in HDF5 the chunking feature will be required.
SO by default, this feature will not be turned on.
Solution:
see the above
Platforms tested:
linux 2.2.18
Kludge
Description:
Since we're only about halfway through converting the internal use of
property lists from the "old way" to the generic property lists, we turned
off snapshots to avoid exposing lots of API changes to users, until the
APIs settled down.
Getting the snapshots rolling again seems to have become a priority, so
some changes are going to have to be made now that were going to be
postponed until we were completely finished with the conversion. This
requires that the old API functions be able to deal with both the old
and new property lists smoothly.
Solution:
Kludge together the property list code so that they can transparently handle
dealing with both the old and new property lists
Platforms tested:
FreeBSD 4.4 (hawkwind)
Code cleanup for better compatibility with C++ compilers
Description:
C++ compilers are choking on our C code, for various reasons:
we used our UNUSED macro incorrectly when referring to pointer types
we used various C++ keywords as variables, etc.
we incremented enum's with the ++ operator.
Solution:
Changed variables, etc.to avoid C++ keywords (new, class, typename, typeid,
template)
Fixed usage of UNUSED macro from this:
char UNUSED *c
to this:
char * UNUSED c
Switched the enums from x++ to x=x+1
Platforms tested:
FreeBSD 4.4 (hawkwind)
Purpose:
Bug Fix
Description:
Some so-called "operating systems" (*cough*Windows*cough*) can't
handle large string sizes.
Solution:
Replace the Usage string with individual strings which all call
fprintf() themselves.
Platforms tested:
Linux
Purpose:
Bug Fix
Description:
Object IDs command-line options weren't being picked up.
Solution:
The wrong flag was being checked for. Changed the flag from "v" to
"i", which is what the documentation says.
Platforms tested:
Linux
Purpose:
add another test file
Description:
Solution:
Platforms tested:
eirene, arabica
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
Add another test file
Description:
Solution:
Platforms tested:
eirene, arabica
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
a bug fix
Description:
change PIXEL_INTERLACE to INTERLACE_PIXEL and other interlace mode description
to fit for the image specification.
Solution:
Platforms tested:
eirene, sol2.7
Purpose:
1. fix a bug
2. turn off a feature
Description:
1. change the output of GRgetiminfo from NULL to &interlace_mode.
2. turn off the feature to change line-interleaved feature into
pixel-interleaved feature since inconsistent behaviour is found
in GR interface.
Solution:
see above
Platforms tested:
eirene, arabica
Purpose:
add a real raster-24 bit testing for interlace mode.
Description:
1. GR interfaces will never create an HDF4 file with interlace mode other than
pixel interleaved. DF24 interfaces can create HDF4 file with different interleaved.
There are inconsistent behaviors between GRreqimageil and GRreadimage, data read into the memory will not behave properly if a new interlace mode is asked.
2. Currently HDF5 image spec. supports pixel interleaved and plane interleaved.
We make a real image file to test whether the converter is doing the right thing.
Solution:
We use DF24 bit APIs to generate a real image file that can be tested by H5view.
Platforms tested:
linux and sol2.7
Code cleanup (sorta)
Description:
When the first versions of the HDF5 library were designed, I remembered
vividly the difficulties of porting code from a 32-bit platform to a 16-bit
platform and asked that people use intn & uintn instead of int & unsigned
int, respectively. However, in hindsight, this was overkill and
unnecessary since we weren't going to be porting the HDF5 library to
16-bit architectures.
Currently, the extra uintn & intn typedefs are causing problems for users
who'd like to include both the HDF5 and HDF4 header files in one source
module (like Kent's h4toh5 library).
Solution:
Changed the uintn & intn's to unsigned and int's respectively.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Improvement
Description:
The stdout and stderr were both redirected to an output file. This
works fine in tradition sequential Unix machines. But in some
parallel systems (like mpi-jobs in IBM SP), the stderr is merged
with stdout alright but not in the exact order as expected. This
is not deterministic in parallel jobs. So, the test output are
all there but the ordering maynot be as expected.
Solution:
Redirect stderr to separated file and append it to the stdout
file after test-command is executed. Then compare it with
the expected output. This eliminate the assumption that
stdout and stderr must merged in "chronical orders".
The .ddl file are updated by moving all stderr text to the end of the
file.
Platforms tested:
eirene.
Improvement
Description:
The stdout and stderr were both redirected to an output file. This
works fine in tradition sequential Unix machines. But in some
parallel systems (like mpi-jobs in IBM SP), the stderr is merged
with stdout alright but not in the exact order as expected. This
is not deterministic in parallel jobs. So, the test output are
all there but the ordering maynot be as expected.
Solution:
Redirect stderr to separated file and append it to the stdout
file after test-command is executed. Then compare it with
the expected output. This eliminate the assumption that
stdout and stderr must merged in "chronical orders".
Platforms tested:
tested in v1.4. Folded it into v1.5.
Purpose:
check-in the second time to update the handling of data transfer in h4toh5.
This will make up for the cvs conflict checking a couple hours ago.
Description:
Solution:
Platforms tested:
eirene
Purpose:
1) fix the implementation of image according to image specfication
2) fix two bugs of SDS implemention. the first one is
to handle the unlimited SDS with the first dimensional size set to 0.
the second one is to change the way how HDF5 dataset is written.
Description:
1) mapping 24-bit image to 3D arrays instead of 2D compound datatype.
2) previously forgot considering unlimited SDS with the size set to 0.
3) H5P_set_buffer seems not working well for a extremely small size.
Solution:
1) see above.
2) add a special case to deal with this.
3) don't use H5Pset_buffer.
Platforms tested:
RedHat Zoot 6.2
More code cleanups
Description:
Wrap up the code cleanups for changing the dataset transfer property lists
over to using the generic property list code.
Platforms tested:
IRIX64 6.5 (modi4)
Code cleanups, mostly..
Description:
Work on pacifying the SGI compiler to get the generic properties working
correctly with --enable-parallel and --enable-fortran. It's not quite
fixed yet, but I need to head home and these patches help... :-/
Platforms tested:
IRIX64 6.5 (modi4)
New Features!
Description:
Start migrating the internal use of property lists in the library from the
older implementation to the new generic property lists.
Currently, only the dataset transfer property lists are migrated to the
new architecture, all the rest of the property list types are still using
the older architecture.
Also, the backward compatibility features are not implemented yet, so
applications which use dataset transfer properties may need to make the
following changes:
H5Pcreate(H5P_DATASET_XFER) -> H5Pcreate_list(H5P_DATASET_XFER_NEW)
and
H5Pclose(<a dataset transfer property list>) -> H5Pclose_list(id)
This still may have some bugs in it, especially with Fortran, but I should
be wrapping up those later today.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Feature shift
Description:
Take out the v1.2.x compatibility stubs and put in the hooks for v1.4.x
compatibility when needed.
Platforms tested:
FreeBSD 4.3 (hawkwind)
Purpose:
Bug Fix
Description:
The way we were generating Dependencies and .depend files was broken.
If the $srcdir or other macros began with a ".", then it would match
anything and cause problems since it would then overwrite the
beginning of the header file's path.
Solution:
Wrote a Perl script which can handle this type of weirdness better.
It's only used when the environment is a GNU one with a GCC
compiler...
Platforms tested:
Linux
Purpose:
Bug Fix/Feature Add
Description:
Added new flag ("-f" and "--family") to allow user to specify which
file driver to use to open the file. If they don't specify anything,
then it defaults to the old behaviour of trying each driver in turn
until one actually opens the file.
If the driver the user specified doesn't succeed in opening the
file, then we do NOT try other file drivers.
Platforms tested:
Linux
Purpose:
Reformatting
Description:
Reformatted the code so that it's much clearer and conforms to the
HDF5 coding standards. Changed the function headers to use the ANSI
style instead of the KnR style. Kept the use of typedef's such as
"unsigned char" being "BYTE" and so on since of of this code is
copied from some other place and that's the style they use...I didn't
want to break things.
Platforms tested:
It compiles fine on Linux, but there aren't any tests for this
package, so...
Purpose:
Fixing a boo-boo
Description:
There was a problem with the generated Dependencies file. It listed
the H5pubconf.h header file as being in the $(top_srcdir) directory
when it's in the $(top_builddir) directory.
Solution:
Regenerated it.
Platforms tested:
Linux
Purpose:
a bug in the comment
Description:
The structure of HDF4 file is not correct in the orginal comment
Solution:
Correct the wrong comment and add more explanation
Platforms tested:
eirene
Bug Fix, Code Cleanup, Code Optimization, etc.
Description:
Fold in the hyperslab speedups, clean up compile warnings and change a
few things from using 'unsigned' or 'hsize_t' to use 'size_t' instead.
Platforms tested:
FreeBSD 4.3 (hawkwind), Solaris 2.7 (arabica), Irix64 6.5 (modi4)
Purpose:
Bug fix (by Bob McGrath)
Description:
On Solaris platforms palette was not written to the HDF5 file
Solution:
'\0' character was written outside the GroupName array. Apparently on Solaris it destroyed the condition value that determined if the palette should be written to the file.
Platforms tested:
Not tested yet.
Purpose:
Bug fix
Description:
One of the function calls (H5Sget_simple_extent_dims) used parameter with
the wrong type. That caused compilation errors on T3E.
Solution:
Used NULL since that argument is optional and was never used.
Platforms tested:
T3E(mcurie) and IRIX64 (modi4)
code warrior support
and some clean up
the macros file_seek and file_offset_t that were repeated over sevral files were put only in
H5private.h
H5private .h was updated for win32
vthe
Description:
Solution:
Platforms tested:
Purpose:
new features
Description:
1. add an option to convert HDF4 file without HDF4 specified attributes such as
HDF4_OBJECT_TYPE, HDF4_REF_NUM etc.
it can be done by inputting "h4toh5 -na input.hdf"
The default converter will still keep HDF4 specfied attributes.
2. Add compression features (gzip) for image too. Now the compressed HDF4 image
can be supported by using HDF5 gzip. Not sure whether tools can read it. Need to be tested.
3. Change SPACEPAD to NULLTERM for HDF4 dimensional name list. We can use variable length HDF5 string to represent these names, however currently H5dump and H5view cannot support variable length HDF5 string. converter will wait for other tools' update.
Solution:
Platforms tested:
eirene(Red Hat 6.2) and arabica(solaris 2.7)
Purpose:
update h4toh5 converter utility test files
Description:
better output of HDF4 dimensional name list
Solution:
Platforms tested:
eirene,arabica
Purpose:
update h4toh5 testing files
Description:
Now we are using fixed size for HDF5 dimensional name list and h5dump
output is appended with 000/000/.......................
It looks ugly and annoying.
Solution:
Use NULLTERM for SDS dimensional list names
Platforms tested:
RedHat 6.2(eirene) and solaris 2.7(arabica)
Purpose:
Bug fix
Description:
Rushabh added new directory tools/gifconv to CVS. He also modified
MANIFEST, but bin/chkmanifest complains this file.
Solution:
I will try to delete gif.h and then to add it once more.
Purpose:
Adding new feature
Description:
Added gif2h5 and h52gif conversion utilities
Solution:
The utilites follow the framework built for the gif2hdf and hdf2gif
utilities for hdf4. The main files modified were those that read the
H5 file and those that write H5 file. In the future, if you wish to
continue with the framework and extend it to .png or some other fileformat
the main files to edit will be the gif reader and writer.
One point to note with h52gif. You have to specify the exact location of
the image and the palette that it links to. You can choose not to specify
a palette (uniform grayscale chosen in this case) but you must specify
image location. In the future, someone could edit the readhdf.c source
to enable the reader to parse the hdf file and select all images with
corresponding palettes.
Platforms tested:
modi4 , eirene , hawkwind , arabica , Ren (NT 4.0) , Personal box (win2k)
Purpose:
Add definations of two new functions
Description:
Solution:
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
Add a constant(compression level for gzip)
Description:
For compression issue
Solution:
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
a bug fix
Description:
User can define "Real Vdata" as user-defined attribute. By using VSisattr, we can check this out. In order to keep this piece of information, We use "Vdata attribute" in the converted HDF5 file to distingush this kind of Vdata from independent Vdata.
Solution:
see above
Platforms tested:
eirene(Linux)
Purpose:
a bug fix
Description:
When Vsisattr is true, this Vdata still needs to be converted as an independent
real "Vdata", We will add object type of this vdata as "Vdata attribute".
Solution:
erease the evaluation of Vsisattr call.
Platforms tested:
Linux(eirene)
Purpose:
bug fix Adding more features
Description:
Bugs: 1) hdf4 dimensional scale data can be none, but the dim name can still defined by users, so number of hdf4 dimensional names and number of object reference may be different
Previously, this problem is not considered.
2) SDcheckempty will return true when fill value is set to HDF4 SDS, and then fill value information is lost
3) check whether SDS have fill value set although SDcheckempty return true.
Use H5Psetfillvalue and H5Dcreate in HDF5 part, still needs to wait for the new development of HDF5 and also need to investigate whether this part of code has bugs.
New features: compressed SDS will get compressed with gzip when it is converted. That will save some space.
[describe the bug, or describe the new feature, etc]
Solution:
See above and design document
Platforms tested:
eirene(linux)
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
small bug fix
Description:
If specifying --stride, it was checking for the wrong short-form of
the command-line parameters.
Solution:
Changed the 'T' to 'S' which is the new short form for the stride
option.
Platforms tested:
Linux
Update
Description:
If the count for a subset isn't specified, then we default to the
remainder of the dataset.
Solution:
Check for the count to be specified. If not, then find the dimensions
of the dataset and subtract from the the "start" parameter.
Platforms tested:
Linux
Update
Description:
Updated the way the subsetting data is retrieved. It now does it one
row of blocks at a time. It may still run out of memory, but this is
at least a good first step.
Also, the start parameter defaults to (0, 0, ...) if it isn't
specified.
Platforms tested:
Linux
Update
Description:
Replaced
#include <hdf5_file.h>
with
#include "hdf5_file.h"
so that gcc can pick up our files more easily without picking up
system header files (which we don't care about being in the
dependencies list).
Platforms tested:
Linux
