Lots of performance improvements & a couple new internal API interfaces.
Description:
Performance Improvements:
- Cached file offset & length sizes in shared file struct, to avoid
constantly looking them up in the FCPL.
- Generic property improvements:
- Added "revision" number to generic property classes to speed
up comparisons.
- Changed method of storing properties from using a hash-table
to the TBBT routines in the library.
- Share the propery names between classes and the lists derived
from them.
- Removed redundant 'def_value' buffer from each property.
- Switching code to use a "copy on write" strategy for
properties in each list, where the properties in each list
are shared with the properties in the class, until a
property's value is changed in a list.
- Fixed error in layout code which was allocating too many buffers.
- Redefined public macros of the form (H5open()/H5check, <variable>)
internally to only be (<variable>), avoiding innumerable useless
calls to H5open() and H5check_version().
- Reuse already zeroed buffers in H5F_contig_fill instead of
constantly re-zeroing them.
- Don't write fill values if writing entire dataset.
- Use gettimeofday() system call instead of time() system when
checking the modification time of a dataset.
- Added reference counted string API and use it for tracking the
names of objects opening in a file (for the ID->name code).
- Removed redundant H5P_get() calls in B-tree routines.
- Redefine H5T datatype macros internally to the library, to avoid
calling H5check redundantly.
- Keep dataspace information for dataset locally instead of reading
from disk each time. Added new module to track open objects
in a file, to allow this (which will be useful eventually for
some FPH5 metadata caching issues).
- Remove H5AC_find macro which was inlining metadata cache lookups,
and call function instead.
- Remove redundant memset() calls from H5G_namei() routine.
- Remove redundant checking of object type when locating objects
in metadata cache and rely on the address only.
- Create default dataset object to use when default dataset creation
property list is used to create datasets, bypassing querying
for all the property list values.
- Use default I/O vector size when performing raw data with the
default dataset transfer property list, instead of querying for
I/O vector size.
- Remove H5P_DEFAULT internally to the library, replacing it with
more specific default property list based on the type of
property list needed.
- Remove redundant memset() calls in object header message (H5O*)
routines.
- Remove redunant memset() calls in data I/O routines.
- Split free-list allocation routines into malloc() and calloc()-
like routines, instead of one combined routine.
- Remove lots of indirection in H5O*() routines.
- Simplify metadata cache entry comparison routine (used when
flushing entire cache out).
- Only enable metadata cache statistics when H5AC_DEBUG is turned
on, instead of always tracking them.
- Simplify address comparison macro (H5F_addr_eq).
- Remove redundant metadata cache entry protections during dataset
creation by protecting the object header once and making all
the modifications necessary for the dataset creation before
unprotecting it.
- Reduce # of "number of element in extent" computations performed
by computing and storing the value during dataspace creation.
- Simplify checking for group location's file information, when file
has not been involving in file-mounting operations.
- Use binary encoding for modification time, instead of ASCII.
- Hoist H5HL_peek calls (to get information in a local heap)
out of loops in many group routine.
- Use static variable for iterators of selections, instead of
dynamically allocation them each time.
- Lookup & insert new entries in one step, avoiding traversing
group's B-tree twice.
- Fixed memory leak in H5Gget_objname_idx() routine (tangential to
performance improvements, but fixed along the way).
- Use free-list for reference counted strings.
- Don't bother copying object names into cached group entries,
since they are re-created when an object is opened.
The benchmark I used to measure these results created several thousand
small (2K) datasets in a file and wrote out the data for them. This is
Elena's "regular.c" benchmark.
These changes resulted in approximately ~4.3x speedup of the
development branch when compared to the previous code in the
development branch and ~1.4x speedup compared to the release
branch.
Additionally, these changes reduce the total memory used (code and
data) by the development branch by ~800KB, bringing the development
branch back into the same ballpark as the release branch.
I'll send out a more detailed description of the benchmark results
as a followup note.
New internal API routines:
Added "reference counted strings" API for tracking strings that get
used by multiple owners without duplicating the strings.
Added "ternary search tree" API for text->object mappings.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
Other platforms/configurations tested?
FreeBSD 4.7 (sleipnir) serial & parallel
Solaris 2.6 (baldric) serial
Purpose:
bug fix
Description:
Need to use HDopen, HDread for windows support.
off_t in windows is 32-bit; however, windows can support 64-bit file. So we
have to change off_t to int64 in order that >4G file can be supported.
Solution:
See above
Platforms tested:
linux2.2.18smp, windows 2000
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
2002-12-03 23:00:35 Robb Matzke <matzke@arborea.spizella.com>
* main: Replaced the H5LS_PREPEND_FILENAME compile-time symbol which was
always defined, with a runtime decision. If h5ls is
invoked with more than one argument then the file name
is displayed as part of the object name, otherwise the
file name is not displayed.
Renamed h5dumptst.c to h5dumpgentest.c.
Added a test to run h5dumpgentest to make it can run without error. Note
that this does not verify if it is generating the correct test files.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? YES
Misc. update:
Update MANIFEST if you add or remove any file. YES
Changed the file name nofilename.ddl to tnofilename.ddl, to be consistent
with the other file names.
Platforms tested:
eirene only since this is a very simple change.
Improvement
Description:
Moved the code parsing for missing file name in the parse_command_line
routine.
Added a test checking if the tool fails as expected when no file name is
given.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? YES
Misc. update:
Update MANIFEST if you add or remove any file. YES, added nofilename.ddl
Bug fix (ID 765 and part of ID 841)
Description:
"h5dump -H" would core dump. It was because the code was not
checking if there was any file argument after option parsing.
It referred to an non-existing filename argument.
There was also some very old but not commented code which allowed
the use of back slash to escape a filename begun with '-'. The
'--' option is better and more common practice.
Solution:
Added code to check if there is still a filename argument after parsing.
(This checking probably should goto inside the option_parsing routine.
Also removed that old code mentioned above.
Platforms tested:
eirene, arabica and modi4
updated copyright note.
Copied the UNUSED correction from v1.4.
Added an example to the usage of -- option.
Description:
[describe the bug, or describe the new feature, etc]
Solution:
[details about the changes, algorithm, etc...]
[Please as detail as you can since your own explanation is
better than others guessing it from the code.]
Platforms tested:
arabica (fortran), modi4 (pp, fortran),eirene (fortran, CC).
Possible Bug Fix
Description:
There seems to be a problem with the h5cc script picking up old
header files. This may have something to do with the CPPFLAGS macro
being set to some nefarious thing and the compiler looking in those
directories for the header files instead of the one where HDF5 is
actually installed.
Solution:
Put the -I$includedir first.
Platforms tested:
None...This is really triggered by the nightly tests. I'm kind of
hoping that this will fix things...
Bug fix
Description:
Old setup put $(srcdir)/h5redeploy in the install list but this
same list is used for clean too, thus the file is removed from
source. That is bad.
Solution:
Set it to use a cp to do it. Need to change the source version
name to something else because for one, it is confusing to use
the same name. For another, if the build is done in place (i.e.,
not using --srcdir), the source file is removed, again.
So, renamed it with the .in suffix. Who knows, it may take
more processing than just cp when more features are added to it.
Platforms tested:
Eirene
Bug Fix
Description:
On AIX machines, the number of script interpreters which ran the
testh5dump.sh script would strip too many of the quotes away, causing
problems if you're quoting something with a space in it.
Solution:
Just got rid of the spaces and used ',' instead.
Platforms tested:
AIX and Linux
Purpose:
user reported a bug on h5 to gif tool
Description:
h5 to gif did not read an HDF5 image correctly
Solution:
updated the program so that it reads the updated HDF5 image format to the latest specification
the debug version of MSVC was giving a failure on the write of the GIF file,
regarding the GIF file pointer
this write error is eliminated declaring that pointer a global variable
there is still an applicattion error on the exit of the program, only on the debug version of MSVC
Platforms tested:
w2000
Migrate from configure macros of XYZ_ABC to H5_XYZ_ABC
Description:
configure generates many macros definitions on the fly and
were stored in src/H5config.h which is included by H5public.h.
But other software that uses hdf5 may also run their own configure.
There can be a clash in macro name space. We decided awhile ago
to prepend all generated macros with "H5_" to avoid conflicts.
The process has started and this commit completes it (at least attempt
to).
Solution:
Many macros symbols (e.g. SIZEOF_xxx and HAVE_xxx were changed to
H5_SIZEOF_xxx and H5_HAVE_xxx). Then H5private.h no longer includes
H5config.h. This cuts H5config.h away from HDF5 source code.
Pending issues:
The module of fortran and pablo are to be resolved in a different
commit.
Platforms tested:
eirene (parallel), arabica (solaris 7 --enable-fortran, --enable-cxx)
Minor h5ls bugfix for compound types with array members.
(Similar changes shortly for 1.4 branch).
2002-04-04 16:39:09 Robb Matzke <matzke@arborea.spizella.com>
* display_cmpd_type: Removed code to display array dimensions from display_cmpd_type()
since this was a holdover from the days when hdf5
didn't have an array datatype and was duplicated by
display_array_type().
Code removal
Description:
Removed the HDF4 source files from the HDF5 tree. The directories
will remain. Use the "-P" option when doing a cvs checkout or update
to "prune" the empty directories from your personal tree.
Code Motion
Description:
Removal of HDF4 from the configure/Makefiles. This is a precursor to
the actual physical removal of the HDF4 tools from the HDF5 tree.
Platforms tested:
Arabica, Dangermouse
Bug Fix
Description:
Some -I paths weren't included in the h5cc script. That would cause
the compiler to fail if it was trying to find gass header files or
the like.
Solution:
Added the CPPFLAGS macro to the h5cc.in file so that it'll be there
when it's generated. This will also include some -D options which we
compiled the library with, like the LFS flags on Linux.
Also changed the configure* files so that it will "chmod" the created
h5cc file to 755 (executable) since that wasn't happening all the
time...
Platforms tested:
Linux
Bug Fix
Description:
There was a problem with having a lot of groups nested together. We
could only handle 1024 characters at most, but, in a parallel program
especially, it could occur that there were lots and lots of groups
and would be more than 1024.
Solution:
I made the "objname" part of the obj_t structure a pointer instead of
a fixed size. Added code to allocate/deallocate the memory we need
for it. Had to fix how the "prefix" was being handled in the h5dump
program. It was also set to only 1024 characters in length. I made it
dynamic.
Added a test case...Go me!
Platforms tested:
Linux, Solaris
Oops
Description:
I added files for testing the group comments dumping feature, but
didn't actually add it to the testh5dump.sh script.
Solution:
Added it.
Platforms tested:
Linux
Bug Fix
Description:
The library path was relying upon the "exec_prefix" variable.
However, we weren't including that into the h5cc script.
Solution:
Added it.
Platforms tested:
Linux
Purpose:
Small Fix
Description:
Changed how the list of drivers were listed from:
{ "driver1",
"driver2",
/* ... */
"drivern",
#ifdef FOO
"driverm"
#endif
};
to
{ "driver1"
,"driver2"
/* ... */
,"drivern"
#ifdef FOO
,"driverm"
#endif
};
since it's a nicer way of doing the same thing without the annoying
warning of an extraneous , if FOO isn't defined.
Platforms tested:
Dangermouse, Modi4, Kelgia
Purpose:
Feature Add
Description:
Added support for dumping Group Comments. This involved a
modification of the DDL as well.
Solution:
Steal code from h5ls and put it in the h5dump. The ddl.html file was
updated as normal. And a test was created...
Platforms tested:
Dangermouse, Modi4, Kelgia
Purpose:
Bug fix
Description:
The "make depend" command wasn't propagating down into the tools/
directories.
Solution:
Added the "depend:" part to the Makefile in the tools/ subdirectory.
Platforms tested:
Linux
Removing the DPSS (gridstorage) driver source code.
Description:
The DPSS (using Grid-Storage) driver is retired.
Removed the configure option with-gridstorage from configure.in.
Cvs remove the following files
./src/H5FDdpss.c
./src/H5FDdpss.h
./test/dpss_read.c
./test/dpss_write.c
Regenerated Dependencies files (some had to be hand-edited since
'make depend' did not cover them.)
Removed reference to DPSS Virtual file driver from H5F.c.
Platforms tested:
modi4 (Parallel; -with-gass=...), eirene, arabica (fortran, cxx).
Bug Fix.
Description:
The H5Rget_object_type function could not get the object type for dataset
region references.
Solution:
Added a new function, H5Rget_obj_type, to replace H5Rget_object_type.
The new function requires the reference type as an additional parameter,
in order to allow queries on different reference types to be performed
correctly.
Platforms tested:
FreeBSD 4.4. (sleipnir)
Purpose:
parameter file
Description:
handle parameters in various special cases.
Users donot even have to use this file if they don't have those special needs.
1) memory optimization (should always be set to 0 without handling huge SDS array conversion)
2) unlimited dimension with zero current size, users can use this file to define
their chunk size.
Solution:
Platforms tested:
RedHat Zoot 6.2
Purpose:
a bug fix
a feature added
Description:
1.conversion of unlimited dimension data with the current dimensional size 0
2. Use a parameter file to control some special cases:
1) To subdivide the huge array into hyperslabs, a memory buffer size has i
to be set.
2) when current dimensional size is 0, a default chunk size can be set.
Solution:
bug fixed 1. Old approach: the current dimensional size is set to H5S_UNLIMITED, which is
a huge number. The default chunk size is set as a *FIXED* default value.
2. New approach: the current dimensional size is set to the current value 0.
Users can provide the chunk size through a parameter file.
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
bug fixed
Description:
HDF4 Vdata is extensible, when converted, the new HDF4 data set should also be extensible.
Solution:
Make the HDF4 dataset converted from Vdata unlimited dimensions.
These three testing files should be updated accordingly.
Platforms tested:
Linux, RedHat 6.2
Backward Compatibility Fix
Description:
One of H5P[gs]et_buffer's parameters changed between v1.4 and the
development branch.
Solution:
Added v1.4 compat stuff around H5P[gs]et_buffer implementation and testing
to allow v1.4.x users to continue to use their source code without
modification.
These changes are for everything except the FORTRAN wrappers - I spoke with
Elena and she will make the FORTRAN wrapper changes.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Code cleanup
Description:
Windows is generating hundreds of warnings from some of the practices in
the library. Mostly, they are because size_t is 32-bit and hsize_t is
64-bit on Windows and we were carelessly casting the larger values down to
the smaller ones without checking for overflow.
Also, some other small code cleanups,etc.
Solution:
Re-worked some algorithms to eliminate the casts and also added more
overflow checking for assignments and function parameters which needed
casts.
Kent did most of the work, I just went over his changes and fit them into
the the library code a bit better.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Purpose:
A new feature
Description:
While testing h4toh5 utility with real NASA files, we find an example that the data array(one SDS) is so big that it exceeds the physical memory of some machine(>128 MB) and the conversion failed. Before the smart hyperslab operation is out, I am dividing the whole SDS into smaller hyperslabs with each hyperslab propotational to the original SDS array dimensions. For example, a three dimension array with 1000*1000*1000 elements, I can divide them into eight 500*500*500 pieces. I can read and write each piece and remember their starting and ending points. In this way, the memory allocation failure can be avoided; however, it may not be the efficient way.
I've tested this feature using SDS without chunking. It works fine. However, when testing SDS with chunking, it is extremely slow. This happens to be a bug in HDF5 library now. Quincey may fix this later and give me a more efficient way to handle the problem. Currently all my testing files are with UNLIMITED dimensions, so in HDF5 the chunking feature will be required.
SO by default, this feature will not be turned on.
Solution:
see the above
Platforms tested:
linux 2.2.18
Kludge
Description:
Since we're only about halfway through converting the internal use of
property lists from the "old way" to the generic property lists, we turned
off snapshots to avoid exposing lots of API changes to users, until the
APIs settled down.
Getting the snapshots rolling again seems to have become a priority, so
some changes are going to have to be made now that were going to be
postponed until we were completely finished with the conversion. This
requires that the old API functions be able to deal with both the old
and new property lists smoothly.
Solution:
Kludge together the property list code so that they can transparently handle
dealing with both the old and new property lists
Platforms tested:
FreeBSD 4.4 (hawkwind)
Code cleanup for better compatibility with C++ compilers
Description:
C++ compilers are choking on our C code, for various reasons:
we used our UNUSED macro incorrectly when referring to pointer types
we used various C++ keywords as variables, etc.
we incremented enum's with the ++ operator.
Solution:
Changed variables, etc.to avoid C++ keywords (new, class, typename, typeid,
template)
Fixed usage of UNUSED macro from this:
char UNUSED *c
to this:
char * UNUSED c
Switched the enums from x++ to x=x+1
Platforms tested:
FreeBSD 4.4 (hawkwind)
Purpose:
Bug Fix
Description:
Some so-called "operating systems" (*cough*Windows*cough*) can't
handle large string sizes.
Solution:
Replace the Usage string with individual strings which all call
fprintf() themselves.
Platforms tested:
Linux
Purpose:
Bug Fix
Description:
Object IDs command-line options weren't being picked up.
Solution:
The wrong flag was being checked for. Changed the flag from "v" to
"i", which is what the documentation says.
Platforms tested:
Linux
Purpose:
add another test file
Description:
Solution:
Platforms tested:
eirene, arabica
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
Add another test file
Description:
Solution:
Platforms tested:
eirene, arabica
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
a bug fix
Description:
change PIXEL_INTERLACE to INTERLACE_PIXEL and other interlace mode description
to fit for the image specification.
Solution:
Platforms tested:
eirene, sol2.7
Purpose:
1. fix a bug
2. turn off a feature
Description:
1. change the output of GRgetiminfo from NULL to &interlace_mode.
2. turn off the feature to change line-interleaved feature into
pixel-interleaved feature since inconsistent behaviour is found
in GR interface.
Solution:
see above
Platforms tested:
eirene, arabica
Purpose:
add a real raster-24 bit testing for interlace mode.
Description:
1. GR interfaces will never create an HDF4 file with interlace mode other than
pixel interleaved. DF24 interfaces can create HDF4 file with different interleaved.
There are inconsistent behaviors between GRreqimageil and GRreadimage, data read into the memory will not behave properly if a new interlace mode is asked.
2. Currently HDF5 image spec. supports pixel interleaved and plane interleaved.
We make a real image file to test whether the converter is doing the right thing.
Solution:
We use DF24 bit APIs to generate a real image file that can be tested by H5view.
Platforms tested:
linux and sol2.7
Code cleanup (sorta)
Description:
When the first versions of the HDF5 library were designed, I remembered
vividly the difficulties of porting code from a 32-bit platform to a 16-bit
platform and asked that people use intn & uintn instead of int & unsigned
int, respectively. However, in hindsight, this was overkill and
unnecessary since we weren't going to be porting the HDF5 library to
16-bit architectures.
Currently, the extra uintn & intn typedefs are causing problems for users
who'd like to include both the HDF5 and HDF4 header files in one source
module (like Kent's h4toh5 library).
Solution:
Changed the uintn & intn's to unsigned and int's respectively.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Improvement
Description:
The stdout and stderr were both redirected to an output file. This
works fine in tradition sequential Unix machines. But in some
parallel systems (like mpi-jobs in IBM SP), the stderr is merged
with stdout alright but not in the exact order as expected. This
is not deterministic in parallel jobs. So, the test output are
all there but the ordering maynot be as expected.
Solution:
Redirect stderr to separated file and append it to the stdout
file after test-command is executed. Then compare it with
the expected output. This eliminate the assumption that
stdout and stderr must merged in "chronical orders".
The .ddl file are updated by moving all stderr text to the end of the
file.
Platforms tested:
eirene.
Improvement
Description:
The stdout and stderr were both redirected to an output file. This
works fine in tradition sequential Unix machines. But in some
parallel systems (like mpi-jobs in IBM SP), the stderr is merged
with stdout alright but not in the exact order as expected. This
is not deterministic in parallel jobs. So, the test output are
all there but the ordering maynot be as expected.
Solution:
Redirect stderr to separated file and append it to the stdout
file after test-command is executed. Then compare it with
the expected output. This eliminate the assumption that
stdout and stderr must merged in "chronical orders".
Platforms tested:
tested in v1.4. Folded it into v1.5.
Purpose:
check-in the second time to update the handling of data transfer in h4toh5.
This will make up for the cvs conflict checking a couple hours ago.
Description:
Solution:
Platforms tested:
eirene
Purpose:
1) fix the implementation of image according to image specfication
2) fix two bugs of SDS implemention. the first one is
to handle the unlimited SDS with the first dimensional size set to 0.
the second one is to change the way how HDF5 dataset is written.
Description:
1) mapping 24-bit image to 3D arrays instead of 2D compound datatype.
2) previously forgot considering unlimited SDS with the size set to 0.
3) H5P_set_buffer seems not working well for a extremely small size.
Solution:
1) see above.
2) add a special case to deal with this.
3) don't use H5Pset_buffer.
Platforms tested:
RedHat Zoot 6.2
More code cleanups
Description:
Wrap up the code cleanups for changing the dataset transfer property lists
over to using the generic property list code.
Platforms tested:
IRIX64 6.5 (modi4)
Code cleanups, mostly..
Description:
Work on pacifying the SGI compiler to get the generic properties working
correctly with --enable-parallel and --enable-fortran. It's not quite
fixed yet, but I need to head home and these patches help... :-/
Platforms tested:
IRIX64 6.5 (modi4)
New Features!
Description:
Start migrating the internal use of property lists in the library from the
older implementation to the new generic property lists.
Currently, only the dataset transfer property lists are migrated to the
new architecture, all the rest of the property list types are still using
the older architecture.
Also, the backward compatibility features are not implemented yet, so
applications which use dataset transfer properties may need to make the
following changes:
H5Pcreate(H5P_DATASET_XFER) -> H5Pcreate_list(H5P_DATASET_XFER_NEW)
and
H5Pclose(<a dataset transfer property list>) -> H5Pclose_list(id)
This still may have some bugs in it, especially with Fortran, but I should
be wrapping up those later today.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Feature shift
Description:
Take out the v1.2.x compatibility stubs and put in the hooks for v1.4.x
compatibility when needed.
Platforms tested:
FreeBSD 4.3 (hawkwind)
Purpose:
Bug Fix
Description:
The way we were generating Dependencies and .depend files was broken.
If the $srcdir or other macros began with a ".", then it would match
anything and cause problems since it would then overwrite the
beginning of the header file's path.
Solution:
Wrote a Perl script which can handle this type of weirdness better.
It's only used when the environment is a GNU one with a GCC
compiler...
Platforms tested:
Linux
Purpose:
Bug Fix/Feature Add
Description:
Added new flag ("-f" and "--family") to allow user to specify which
file driver to use to open the file. If they don't specify anything,
then it defaults to the old behaviour of trying each driver in turn
until one actually opens the file.
If the driver the user specified doesn't succeed in opening the
file, then we do NOT try other file drivers.
Platforms tested:
Linux
Purpose:
Reformatting
Description:
Reformatted the code so that it's much clearer and conforms to the
HDF5 coding standards. Changed the function headers to use the ANSI
style instead of the KnR style. Kept the use of typedef's such as
"unsigned char" being "BYTE" and so on since of of this code is
copied from some other place and that's the style they use...I didn't
want to break things.
Platforms tested:
It compiles fine on Linux, but there aren't any tests for this
package, so...
Purpose:
Fixing a boo-boo
Description:
There was a problem with the generated Dependencies file. It listed
the H5pubconf.h header file as being in the $(top_srcdir) directory
when it's in the $(top_builddir) directory.
Solution:
Regenerated it.
Platforms tested:
Linux
Purpose:
a bug in the comment
Description:
The structure of HDF4 file is not correct in the orginal comment
Solution:
Correct the wrong comment and add more explanation
Platforms tested:
eirene
Bug Fix, Code Cleanup, Code Optimization, etc.
Description:
Fold in the hyperslab speedups, clean up compile warnings and change a
few things from using 'unsigned' or 'hsize_t' to use 'size_t' instead.
Platforms tested:
FreeBSD 4.3 (hawkwind), Solaris 2.7 (arabica), Irix64 6.5 (modi4)
Purpose:
Bug fix (by Bob McGrath)
Description:
On Solaris platforms palette was not written to the HDF5 file
Solution:
'\0' character was written outside the GroupName array. Apparently on Solaris it destroyed the condition value that determined if the palette should be written to the file.
Platforms tested:
Not tested yet.
Purpose:
Bug fix
Description:
One of the function calls (H5Sget_simple_extent_dims) used parameter with
the wrong type. That caused compilation errors on T3E.
Solution:
Used NULL since that argument is optional and was never used.
Platforms tested:
T3E(mcurie) and IRIX64 (modi4)
code warrior support
and some clean up
the macros file_seek and file_offset_t that were repeated over sevral files were put only in
H5private.h
H5private .h was updated for win32
vthe
Description:
Solution:
Platforms tested:
Purpose:
new features
Description:
1. add an option to convert HDF4 file without HDF4 specified attributes such as
HDF4_OBJECT_TYPE, HDF4_REF_NUM etc.
it can be done by inputting "h4toh5 -na input.hdf"
The default converter will still keep HDF4 specfied attributes.
2. Add compression features (gzip) for image too. Now the compressed HDF4 image
can be supported by using HDF5 gzip. Not sure whether tools can read it. Need to be tested.
3. Change SPACEPAD to NULLTERM for HDF4 dimensional name list. We can use variable length HDF5 string to represent these names, however currently H5dump and H5view cannot support variable length HDF5 string. converter will wait for other tools' update.
Solution:
Platforms tested:
eirene(Red Hat 6.2) and arabica(solaris 2.7)
Purpose:
update h4toh5 converter utility test files
Description:
better output of HDF4 dimensional name list
Solution:
Platforms tested:
eirene,arabica
Purpose:
update h4toh5 testing files
Description:
Now we are using fixed size for HDF5 dimensional name list and h5dump
output is appended with 000/000/.......................
It looks ugly and annoying.
Solution:
Use NULLTERM for SDS dimensional list names
Platforms tested:
RedHat 6.2(eirene) and solaris 2.7(arabica)
Purpose:
Bug fix
Description:
Rushabh added new directory tools/gifconv to CVS. He also modified
MANIFEST, but bin/chkmanifest complains this file.
Solution:
I will try to delete gif.h and then to add it once more.
Purpose:
Adding new feature
Description:
Added gif2h5 and h52gif conversion utilities
Solution:
The utilites follow the framework built for the gif2hdf and hdf2gif
utilities for hdf4. The main files modified were those that read the
H5 file and those that write H5 file. In the future, if you wish to
continue with the framework and extend it to .png or some other fileformat
the main files to edit will be the gif reader and writer.
One point to note with h52gif. You have to specify the exact location of
the image and the palette that it links to. You can choose not to specify
a palette (uniform grayscale chosen in this case) but you must specify
image location. In the future, someone could edit the readhdf.c source
to enable the reader to parse the hdf file and select all images with
corresponding palettes.
Platforms tested:
modi4 , eirene , hawkwind , arabica , Ren (NT 4.0) , Personal box (win2k)
Purpose:
Add definations of two new functions
Description:
Solution:
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
Add a constant(compression level for gzip)
Description:
For compression issue
Solution:
Platforms tested:
eirene
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
a bug fix
Description:
User can define "Real Vdata" as user-defined attribute. By using VSisattr, we can check this out. In order to keep this piece of information, We use "Vdata attribute" in the converted HDF5 file to distingush this kind of Vdata from independent Vdata.
Solution:
see above
Platforms tested:
eirene(Linux)
Purpose:
a bug fix
Description:
When Vsisattr is true, this Vdata still needs to be converted as an independent
real "Vdata", We will add object type of this vdata as "Vdata attribute".
Solution:
erease the evaluation of Vsisattr call.
Platforms tested:
Linux(eirene)
Purpose:
bug fix Adding more features
Description:
Bugs: 1) hdf4 dimensional scale data can be none, but the dim name can still defined by users, so number of hdf4 dimensional names and number of object reference may be different
Previously, this problem is not considered.
2) SDcheckempty will return true when fill value is set to HDF4 SDS, and then fill value information is lost
3) check whether SDS have fill value set although SDcheckempty return true.
Use H5Psetfillvalue and H5Dcreate in HDF5 part, still needs to wait for the new development of HDF5 and also need to investigate whether this part of code has bugs.
New features: compressed SDS will get compressed with gzip when it is converted. That will save some space.
[describe the bug, or describe the new feature, etc]
Solution:
See above and design document
Platforms tested:
eirene(linux)
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
small bug fix
Description:
If specifying --stride, it was checking for the wrong short-form of
the command-line parameters.
Solution:
Changed the 'T' to 'S' which is the new short form for the stride
option.
Platforms tested:
Linux
Update
Description:
If the count for a subset isn't specified, then we default to the
remainder of the dataset.
Solution:
Check for the count to be specified. If not, then find the dimensions
of the dataset and subtract from the the "start" parameter.
Platforms tested:
Linux
Update
Description:
Updated the way the subsetting data is retrieved. It now does it one
row of blocks at a time. It may still run out of memory, but this is
at least a good first step.
Also, the start parameter defaults to (0, 0, ...) if it isn't
specified.
Platforms tested:
Linux
Update
Description:
Replaced
#include <hdf5_file.h>
with
#include "hdf5_file.h"
so that gcc can pick up our files more easily without picking up
system header files (which we don't care about being in the
dependencies list).
Platforms tested:
Linux
Update
Description:
Replaced "#include <hdf5_file.h>" with the equivalent, but better
#include "hdf5_file.h" so that gcc can pick up our header files more
easily.
Platforms tested:
Linux
Purpose:
a bug fix
Description:
when creating sds dimensional scale dataset, I accidently created two sds dimensional scale dataset with the same name, (say using name "dim1" for both rank 2 and rank 3).
hdf4 library doesn't give me complaints, hdp dumpsds doesn't generate complaints. h4toh5 converter since assumed that dimensional scale name has to be unique, simply skip if finding the same dimensional name, so accidently the result from hdp dumpsds are the same as the result from h5dump until I implement library API and find this bug.
Solution:
make unique sds dimensional scale name for this test file. Will also need to modify testfiles later.
suggestions: somebody check hdf4 library to disallow the same dimensional scale nameOused for the same sds object.
Platforms tested:
eirene, and this check in will not affect daily test.
Update
Description:
Applied fix to h5dumper which was applied to the 1.4 branch. Done
just before creating subdirectories in the 1.4 branch.
Platforms tested:
Linux
Updated
Description:
Updated test results for the h5dumper's XML option. The tool now
conforms with the fixes implemented in the 1.4 branch.
Platforms tested:
Linux
Feature Add
Description:
This is the first (alpha) version of h5tools which does subsetting
according to the subsetting document. There's still some niceties
which need to be worked out, but this is the ground work.
Platforms tested:
Linux
Bug Fix.
Description:
Kent is correct about the malloc error. It's harmless if it's not
needed anyway.
Solution:
Removed the #ifdef's since all platforms should use this.
Platforms tested:
Linux
Purpose:
fixing the format of dumper output for windows test
Description:
New dumptest includes the feature that an error of the output will be put into stderr in linux
On windows, that stderr will not be redirected into the output file and the comparsion between the expected file and the dumper output is not correct.
Solution:
change fprintf(stderr .....) at h5tool_util.c into fprintf(stdout ......) so that it fits the comparison.
Platforms tested:
windows 2000
Purpose:
a bug fix on windows(possible on other platforms)
Description:
not allocating enough space for a string at dump_all for debug version
a string tmp is defined at dump_all(....),
The memory that is allocated to tmp is malloc(strlen(prefix)+strlen(name)+1);
However, there is one testing case : strlen(prefix) is 0 and
tmp is allocated in the following:
strcat(tmp,"/");
strcat(tmp,name);
....
free(tmp);
the program fails when freeing tmp for debug (dll) version on windows 2000
Solution:
For windows platform:
allocate memory strlen(prefix)+strlen(name)+2
Platforms tested:
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
a bug fix
Description:
uninitialize the start and edge value for test_ras8 and test_ras24 functions
Solution:
initialize
Platforms tested:
eirene,arabica
Update
Description:
Due to code motion, some functions which were being called were moved
to another module. Included only that header file.
Platforms tested:
Linux
Update
Description:
Updated to reflect the new naming of APIs in the h5tools library.
I.e., things which were once named "h5dump_*" now have the "h5tools_"
prefix instead.
Platforms tested:
Linux
Code Cleanup
Description:
Moved some code out which wasn't specific to the h5tools into the
h5tools_utils and h5tools_str modules. Also started on adding the
subsetting feature to the dumping.
Platforms tested:
Linux
Code Movement
Description:
Moved some functions which worked only with the h5tools_str_t
structure from the h5tools module to the h5tools_str module.
Platforms tested:
Linux
New Modules
Description:
Separated the "generic" functions out of the h5tools module into
their own separate module to make things a bit nicer to deal with.
Platforms tested:
Linux
Bug Fix
Description:
For some reason, it was trying to cd to the examples/ and pablo/
directories to do a distclean. Only problem was, they aren't here.
Solution:
Removed those from the `for' loop.
Platforms tested:
Linux
More Subsetting
Description:
Wanted to do a check-in of the current subsetting stuff. The
command-line parsing was already in there. I now added the feature to
the h5dump_t structure and it now outputs the new SUBSET DDL stuff in
the correct format. It doesn't yet do the actual subsetting, but
we're getting there...
Platforms tested:
Linux
Resubmit of Changes
Description:
Previous changes to the h5dumper were lost. This patch includes
better memory management of XML formatted strings along with the bug
fixes for the XML code.
Solution:
Merged the XML patch with the previous code.
Platforms tested:
Linux
Bug Fix
Description:
Some platforms (gondolin) don't necessarily have the vsnprintf
function.
Solution:
Changed it to HDvsnprintf and added a header for H5private to include
it...
Platforms tested:
Linux
Bug Fix
Description:
We were trying to build the talign test program before the library
was actually built.
Solution:
Moved talign test to the lib directory since it belongs with the
library anyway.
Platforms tested:
Linux
Purpose:
Bug fix.
Description:
Order of elements in groups is wrong. (Need to find out why
h5gen accepted this at all.)
Note that test output had to be corrected in some cases.
Solution:
Fixed xml_dump_group to do the right order to match the DTD
Platforms tested:
Linux, solaris.
Fix
Description:
Left in the separate subdirectories fro the small programms.
Solution:
removed them and replaced with the misc/ subdirectory
Platforms tested:
Linux
Code Movement
Description:
Moved the tools into their own separate directories (except for these
small tools which are REALLY small, not tested, or documented,
really...They just get put into the misc/ directory).
Platforms tested:
Linux and Kelgia
Purpose:
fix bug in XML output
Description:
OBJ-XID, Parents, etc. were messed up in several cases.
Solution:
Fixed. Updated tests to have correct results in them
Note: tvldtype.h5.xml is a junk file,
Platforms tested:
Solaris 2.7
Purpose:
fix bug in XML output
Description:
OBJ-XID, Parents, etc. were messed up in several cases.
Solution:
Fixed. Updated tests to have correct results in them
Platforms tested:
Solaris 2.7
Bug fix
Description:
h4toh5test was being installed because it was put in the PUB_PROGS
which is specifically for public programs to be installed.
Solution:
Moved it away from the PUB_PROGS list to the PROGS list.
Platforms tested:
eirene.
Purpose:
Fixing Makefile.in for compiling h4toh5test.c at the right option
Without including hdf4 lib, h4toh5test will not be generated.
Description:
Solution:
Platforms tested:
arabica
Feature Add
Description:
Added support for the h4toh5test program which needs to be
conditionally compiled in only if HDF4 is specified.
Platforms tested:
Linux
Feature Add
Description:
This is the first phase of adding subsetting to the dumper. The
command-line parsing should be pretty much set right now.
NOTE: The revision before this one can be used to back port to the
1.4 branch when the XML stuff is folded into that branch for
release.
Platforms tested:
Linux
Purpose:
adding new test hdf4files to be tested by h4toh5 converter
Description:
Solution:
more typical test files are used
Platforms tested:
[machines you have tested the changed version. This is absolute
important. Test it out on at least two or three different platforms
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Purpose:
source code to generate hdf files for h4toh5 converter
Description:
Solution:
see the comments of the code. this file is handled similar with h5dumptst.c
Platforms tested:
eirene, arabica
Purpose:
bug fix
Description:
two bugs: 1. different order of conventions of sds and image in hdf4 is
considered at h4toh5image.c file.
add palette colormodel attribute and correct names of
a few attributes. see detail at:
http://hdf.ncsa.uiuc.edu/HDF5/H5Image/ImageDetails.htm
2. convert hdf internal vdata structure(chunking table)
into a new hdf5 dataset.
Solution:
1. change h5 data space order for image.
h5dims[0] = edge[1]-edge[0];
h5dims[1] = start[1]-start[0];
2. at h4toh5main.c, check the vdata class name and if the
first few characters match with HDF_CHK_TBL; ignore converting
this vdata.
3. add a few attributes at h4toh5pal.c
platforms:
linux and sun 5.7(arabica)
such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and
64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
Update
Description:
Updated the code so that error messages are sent to stdout instead of
to /dev/null. Albert says that it was supposed to be doing this
anyway, so...
Also did some formatting changes.
Solution:
Changed the "2>/dev/null" to "2>&1"
Platforms tested:
Linux
Code cleanup
Description:
Cleaned the code up a bit getting rid of some warnings about "const"
being lost from passing a const char * to a function, etc.
Also force the error messages to stderr instead of simply printing
them to stdout. The same behaviour can be achieved by running h5dump
like:
h5dump ... 2>&1
however, the error reporting before this change wasn't consistent in
its use. I also made some of the error messages clearer (giving names
of the failed types and, when there's an "internal error", reporting
the line number the error occurred on so that we can debug).
Solution:
Converted to "fflush(stdout); fprintf(stderr,...);" for all error
reportings.
Platforms tested:
Linux
Improvement
Description:
The mkdir testfiles and the testing of h5dump have their stderr
redirected to /dev/null. Thus no way to see error messages when happen.
Removed the redirection
Platforms tested:
Eirene (linux).
document
Description:
Added comments in how to use the program and instructions
that the corresponding may need updates if new batch of binary
files are created.
Platforms tested:
Eye-balled only. (Just a few lines of comments.)
Purpose:
windows bug fix
Description:
need extra header files for windows
Solution:
add #ifdef win32 #endif loops
Platforms tested:
windows 2000, confirmed at eirene
Name change
Description:
Changed the names of the testh5toh4 and testh4toh5 scripts to the
"canonical" form -- testh4toh5.sh and testh5toh4.sh.
Platforms tested:
Linux
Bug fix
Description:
I dropped the boat on this one. I assigned prefix to NULL after it'd
been allocated...My bad.
Solution:
Stopped doing that.
Platforms tested:
Linux
Code cleanup
Description:
Changed so that the streaming stuff is inserted if > 1.2 version.
Also modified init_prefix function so that it's sane.
Platforms tested:
Linux
Annoying code cleanup
Description:
This code could all be on one line, but wasn't for some reason.
Solution:
Made it into one printf() call.
Platforms tested:
Linux
Code Cleanup
Description:
Cleaned up where structures were. Added comments to new functions.
Removed strcpy(prefix, "") and made it prefix = NULL; since that's
what it's doing anyway.
Platforms tested:
Linux
Bug fix
Description:
Compiler on sangamon (HP/UX 10.20) is having problems with the statement
"#ifdef 0".
Solution:
Changed statement to "#if 0" instead.
Platforms tested:
Eyeballed...
Code cleanup.
Description:
Fixed _lots_ (I mean _tons_) of warnings spit out by the gcc with the
extra warnings. Including a few show-stoppers for compression on IRIX
machines.
Solution:
Changed lots of variables' types to more sensible and consistent types,
more range-checking, more variable typecasts, etc.
Platforms tested:
FreeBSD 4.2 (hawkwind), IRIX64-64 (modi4)
New Feature
Description:
Changed the command line flags in the h5dumper so that they accept
both short and long flags. The flag syntax for some have changed
(I.e., object ids are no longer -v but -i and -header is now -H or
--header, etc.) A new function is added called get_options which can
be used for all other tools as well.
Had to change these output files to accept the correct flags.
Platforms tested:
Linux
New Feature
Description:
Changed the command line flags in the h5dumper so that they accept
both short and long flags. The flag syntax for some have changed
(I.e., object ids are no longer -v but -i and -header is now -H or
--header, etc.) A new function is added called get_options which can
be used for all other tools as well.
Platforms tested:
Linux
2000-12-28 12:03:41 Robb Matzke <matzke@llnl.gov>
* main: If the file cannot be opened then immediately try the next
argument. This prevents multiple error messages from
the same file.
Purpose:
bug fix
Description:
1. zeroing out memory before using the memory buffer
2. before wrongly using Vquerysize API of HDF4 to obtain
the real file size of record type in HDF4. Actually Vquerysize will
return the real data type size in Memory. That will cause the object
reference difference on Cray which file size is not equal memory size.
Solution:
1. using h4toh5ZeroingMemory function to zero out memory.
2. calcuating the real file size of vdata record in the converter instead of
using Vquerysize HDF4 API.
Platforms tested:
Cray J90, arabica, eirene
Purpose:
bug fix
Description:
Zeroing out all memory buffer before it is being used.
Solution:
using h4toh5ZeroMemory function
Platforms tested:
linux(eirene), arabica(sun 2.7)
Purpose:
kind of bug fix for windows
Description:
Use HDfree() instead of free() so that it will not cause exceptions on windows dll
Solution:
Platforms tested:
windows NT 4.0 and 5.0, confirmed at Linux box.
Bug Fix
Description:
Stumbled across a problem when working on the J90 h5dump problem where
I noticed that some of the output for named compound datatypes wasn't being
displayed.
Solution:
Generated correct DDL files.
Platforms tested:
FreeBSD 4.2 (hawkwind) & Cray J90 (killeen)
Bug Fix
Description:
Stumbled across a problem when working on the J90 h5dump problem where
I noticed that some of the output for named compound datatypes wasn't being
displayed.
Solution:
Corrected output to display missing information.
Platforms tested:
FreeBSD 4.2 (hawkwind) & Cray J90 (killeen)
Purpose:
a bug fix for windows
Description:
ignore checking the permission of file and directory on windows
Solution:
using ifndef WIN32 endif to ignore checking this feature.
Will find how windows deal with file checking stuff later.
Platforms tested:
WIN 2000, and confirmed at eirene(Linux)
Purpose:
turn off "checking file and directory permission right" on windows platform
Description:
Adding #ifndef WIN32 #endif block at the corresponding test-file and test-dir functions.
Solution:
See above.
Platforms tested:
Win NT 4.0,5.0; and confirm the result the same on eirene and arabica
Bug Fix.
Description:
Attributes whose datatype was an object or dataset region reference weren't
dumping correctly.
Solution:
Pass along attribute ID to the code which prints the attributes, so the
reference types can be dereferenced and printed.
Platforms tested:
FreeBSD 4.2 (hawkwind) & IRIX64 6.5 (modi4)
Following up on added tests.
Description:
Added the code to generate the vldata and array test files added into
the h5dump test suite recently. This doesn't have any affect on the
library or tools, it's just for future reference.
Platforms tested:
FreeBSD 4.2 (hawkwind)
Purpose:
bug fix (zeroing memory for memory allocating for the name of dimensional list)
Description:
At modi4, test failed due to garbage read for name of dimensional list.
Solution:
zeroing out memory before writing the name.
Platforms tested:
modi4, eirene(Linux)
Code cleanup/bug fix
Description:
Variable length datatypes were leaking memory when being dumped.
Solution:
Reclaim the VL data after each element has been dumped out.
Platforms tested:
FreeBSD 4.2 (hawkwind) & Solaris 2.6 (baldric)
Re-enabling features.
Description:
Re-add the tests which had been core-dumping on a couple of platforms, now
that the underlying bug is fixed.
Platforms tested:
FreeBSD 4.2 (hawkwind) & Solaris 2.6 (baldric)
Bug fix.
Description:
Data dumping routines weren't correctly handling scalar dataspaces when
dumping them. Under some circumstances, it would get into infinite loops.
Solution:
Small patches to avoid problems when the number of dimensions is zero.
Platforms tested:
FreeBSD 4.2 (hawkwind)
Bug workaround
Description:
tarray4.h5 and tdvltypes4.h5 are causing core dumps on modi4
Solution:
Commented out those tests for now. I'll take a look at them after the code
freeze for the beta-release is finished.
Platforms tested:
O2K (modi4)
Bug fixes and misc. code fixing.
Description:
Updated to reflect current DDL document. Also changed VL data to be able
to be able also dump VL data of any other datatype.
Platforms tested:
FreeBSD 4.1.1 (hawkwind)
Bug fix
Description:
The code for determining what a string should print out if it was
declared as H5T_STR_NULLTERM (C strings), H5T_STR_SPACEPAD
(Fortran strings), or H5T_STR_NULLPAD (print null characters to
the end of the line) was mucked. A user had a problem with it and
suggested a change, but the change didn't seem to work properly.
Also, if the string was H5T_STR_SPACEPAD, it could have stopped
when encountering a NULL even if it hadn't gotten to the end of
the string.
Solution:
Reworked the code to make it more understandable what's happening
and to add in support for H5T_STR_SPACEPAD which may have been
missing before.
Platforms tested:
Linux
Bug fix
Description:
If TESTH5TOH4 or TESTH4TOH5 wasn't set, then it would mess up the
testing process by adding a ``#'' to the for statement (since
there was a line-continuation in the definition of the macro).
Solution:
Placed the macro expansions all on one line so that there's no
junk characters picked up by accident.
Platforms tested:
OSF1 (Gondolin)
Bug fix
Description:
With the changing of the spelling error in the h5ls usage
statement, these testfiles puked.
Solution:
Corrected them in there.
Platforms tested:
Linux
Code cleaning
Description:
Use the variable to rid the warnings of "variable set but not used".
Platforms tested:
Linux and modi4 -64. Just compile only.
Bug fix
Description:
The output of floating point dumps wasn't necessarily standard.
The h5ls utility does it in a better way.
Solution:
Changed the output parameters from %g to %1.*g and added the
appropriate FLT_DIG/DBL_DIG parameter for the `*' in the above.
Platforms tested:
Linux
Bug fix
Description:
I was writing things out to the HDF file in big-endian format
without doing any conversions on the data or anything like that.
This was causing tests to fail on most machines
Solution:
Removed the big-endian craziness...But, this kinda resulted in a
bug in the HDF dumper which Albert and Robb suggested ways of
fixing.
Platforms tested:
Linux
pathc
Description:
TFLOPS coredumped on h5ls because when h5ls calls ioctl(TIOCGWINSZ),
the ioctl() routine coredumped inside.
Solution:
It is likely that TFLOPS does not support window-size in the
compute nodes. Still, it should not coredump. Bypass it for
TFLOPS for now.
Platforms tested:
TFLOPS & modi4 (-64)
Bug fix
Description:
[Most] SGIs failed on dumping VL data during the daily tests they
seem to be initializing variables differently, exposing a bug in dumping
datasets with scalar dataspaces.
Also, clean up code to get rid of compiler warnings.
Solution:
Initilialize variable correctly.
Platforms tested:
SGI IRIX 6.5 (paz)
Bug fix.
Description:
VL datatype dumping was not working correctly on most machines because
the "native" version of the variable-length type wasn't being generated
for the printing process.
Re-enabled VL dumping test
Solution:
Generate "native" version of the VL datatype to read in for printing,
also some code cleanup in the VL dumping algorithm.
Platforms tested:
FreeBSD 4.1.1 (hawkwind) & Solaris 2.6 (baldric)
