Purpose:
New Feature; Optimization; Clean-up
(Merged from 1.4 branch)
Description:
There is no symbolic constant to pass to functions that take an
optional object ID for when the caller wants to indicate no object
ID. In the past the caller always passed a negative integer.
GPFS performs poorly.
The h5ls tool decides whether to list the file name in the output
based on a compile-time choice, which isn't always optimal at run
time.
Solution:
Added a symbolic constant H5I_INVALID_HID.
Added code to tell the mmfsd of GPFS to forego byte range token
prefetching.
h5ls decides whether to print the file name at runtime based on
the number of objects being listed.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
2002-08-09 11:58:59 Robb Matzke <matzke@arborea.spizella.com>
* rewrite_func: Add trace info after any kind of FUNC_ENTER* macro.
Bug fix
Description:
Much of the VL-string testing code was recently disabled accidentally.
Solution:
Re-enable it and add test for writing/reading large string attributes.
Platforms tested:
FreeBSD 4.7 (sleipnir) - small enough to not need triple-check
Purpose:
Feature; Optimization
Description:
Clients pass `-1' or make their own #define for HDF5 functions
that take an optional object ID.
Blue's GPFS is slow for typical SAF restart dumps.
Solution:
Added a #define for H5I_INVALID_HID
Added GPFS-specific code to H5FDmpiposix.c that tells mmfsd to
forego byte range token prefetching. This code can be compiled
into the library by defining USE_GPFS_HINTS. The plan is to either
generalize this so it's detected during configure and turned
on/off at runtime, or to move it up into DSL/SAF with the new HDF5
functions to that return the low-level file handle.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
2002-09-05 12:24:28 Robb Matzke <matzke@arborea.spizella.com>
*: Added #define for H5I_INVALID_HID. An invalid object handle is
actually any integer that isn't currently in use as a
handle, and non-positive integers are never used as a
handle. The #define is -1, and is mostly so that HDF5
users can pass a symbolic name instead of a negative
number to functions that take optional object ID
arguments.
Purpose:
Feature; Optimization
Description:
Clients pass `-1' or make their own #define for HDF5 functions
that take an optional object ID.
Blue's GPFS is slow for typical SAF restart dumps.
Solution:
Added a #define for H5I_INVALID_HID
Added GPFS-specific code to H5FDmpiposix.c that tells mmfsd to
forego byte range token prefetching. This code can be compiled
into the library by defining USE_GPFS_HINTS. The plan is to either
generalize this so it's detected during configure and turned
on/off at runtime, or to move it up into DSL/SAF with the new HDF5
functions to that return the low-level file handle.
Platforms tested:
SuSE Linux (arborea), gcc and mpich-1.2.4
SunOS (baldric), gcc
Purpose:
New feature to H5Dget_offset
Description:
If user block is set, H5Dget_offset should be able to return the absolute
offset from the beginning of file.
Platforms tested:
eirene, arabica
Purpose:
New feature
Description:
Add deflate and shuffle macro at H5pubconf.h on windows.
Solution:
Platforms tested:
windows 2000
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Improvement
Description:
The fortran test code did not exit with error like C programs do.
Without the appropriate exit code, make or shell could not tell the
fortran test program has encountered errors. So, make continues on
even when some fortran tests have failed.
Solution:
Consultant advised to use "CALL exit(code)" to simulate what C programs
do. Though this is not standard Fortran, he has not seen it failed to
work in most fortran compilers thought some of them, like intel compiler,
may need to link in an extra library.
Platforms tested:
Tested on burrwhite (linux 2.4), eirene (linux 2.2), sol (Solaris 8, serial
and mpich parallel), modi4.
Did not use the standard commit test since this is a purely fortran fix.
Cleanup
Description:
Removed the option (--enable-parallel=mpich). It did not work.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}? All passed.
Update
Description:
Cleaned up the debug printing of the generic property list. It was
printout NULL property values for non-existant properties and putting
multiple properties into one group.
Platforms tested:
Eirene (too small change for full testing)
Bug fix
Description:
Array testing routine is creatint huge arrays on the function stack
which causes a segmentation fault on Linux & FreeBSD when threadsafe
support is enabled.
Solution:
Allocate data for test dynamically instead of automatically.
In general, this should be the preferred method for all data arrays.
Platforms tested:
FreeBSD 4.7 (sleipnir) w/threadsafe enabled.
Code cleanup
Description:
Finish checkin of H5A API cleanups with header files I forgot on the first
checkin pass.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Code Cleanup & New Tests
Description:
tmisc.c:
Switched from using H5_HAVE_COMPRESSION flag in favor of
H5_HAVE_FILTER_DEFLATE.
dsets.c:
Switched from using H5_HAVE_COMPRESSION flag in favor of
H5_HAVE_FILTER_DEFLATE.
Refactored I/O filter tests to allow new filters to be [more] easily
added.
Added tests for shuffle & deflate+shuffle I/O filters (if the filter(s)
are enabled).
Added test for creating a new dataset with a filter that is not
available.
Added test for attempting to read a dataset created with a filter that
is not available.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Code Cleanup & New Feature
Description:
H5config.h.in:
Removed H5_HAVE_COMPRESSION & H5_HAVE_FILTER_GZIP flags.
Added H5_HAVE_FILTER_DEFLATE flag.
H5Z.c:
H5Zprivate.h:
H5Zpublic.h:
Switched from using H5_HAVE_COMPRESSION flag in favor of
H5_HAVE_FILTER_DEFLATE.
Added H5Zunregister & H5Zfilter_avail API functions.
Changed a numeric constant (256) to a symbolic constant
(H5Z_FILTER_RESERVED).
Automatically add the shuffling filter to the list of available filters
(when it is enabled).
Moved prototypes for H5Z_filter_deflate & H5Z_filter_shuffle from the
public header into the private header.
H5Zdeflate.c:
Switched from using H5_HAVE_COMPRESSION & H5_HAVE_FILTER_GZIP flags in
favor of H5_HAVE_FILTER_DEFLATE.
Cleaned up formatting & error reporting a bit.
H5Zshuffle.c:
Rewrote shuffling algorithm to be more efficient.
Added error checking & reporting.
Added standard Pablo information.
Added standard function header comment.
Added FUNC_ENTER & FUNC_LEAVE macros.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Code cleanup, Feature enhancement
Description:
Improve hashing algorithm to not generate as many clashes.
Avoid using H5MD_realloc when changing the size of a chunk on disk.
Instead, use H5MF_free followed by H5MF_alloc, which avoids copying the
information from the space for the previous chunk to the space for the
new chunk, since we are going to overwrite the chunk with new data anyway.
This change should both improve I/O performance in situations where
compressed chunks are being overwritten as well as reduce the space used
in the file during that situation.
Cleaned up error handling in a few cases.
Removed old debugging printf()s.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
Bug Fix
Description:
Free blocks of space in the file which were merged with a block before them
were not updating the free list information correctly, allowing space in
the file to over-allocated and potentially corrupted.
Solution:
Correct address of free block during merging.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir)
New feature
Description:
Added TIMESTAMP and event markers. Easier to tell the chronical events.
Turned of echo mode. Added an option to turn it on.
Platforms tested:
Only in eirene.
Purpose:
add "shuffle" filter note
Description:
Solution:
Platforms tested:
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Purpose:
Add a testing case for shuffling algorithm.
Description:
A control test:
Since shuffling 8-bit array should not change the original result; so we use a
8-bit array to test whether after shuffling the array keeps the same values.
Solution:
Platforms tested:
eirene,arabica,modi4
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
Purpose:
Adding internal shuffle filter
Description:
With the combination of shuffling filter with general compression
algorithm, the compression ratio may be improved without adding much
encoding and decoding time for many real NASA datasets(especially
floating data) and other application
datasets(See techNotes).
Solution:
SHuffle the bytes within the data to utilize the locality.
Platforms tested:
arabica , eirene, modi4
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
