feature
Description:
h5repack support for scaleoffset compression
Checking in early to help debug the filter.
Solution:
Added messages and command line to handle new scale offset filter.
Note: TESTS ARE DISABLED FOR NOW. The filter is not
complete, repack tests may fail due to know problems.
PLEASE DO NOT MESS WITH THE SCALEOFFSET TESTS AT THIS TIME.
They will be enabled when the filter is ready.
Platforms tested:
verbena,copper,shanti
Misc. update:
MANIFEST
Fix SZIP filter to dynmically detect encoder.
Description:
Solution:
See:
http://hdf.ncsa.uiuc.edu/RFC/SZIP/Szip_dynamic_12_Oct.pdf
Changes to h5repack tests, contingent on detecting SZIP encoder.
Note new program:
testh5repack_detect_szip
Checks fo rencoder, prints out "yes" or "no". Used by hrepack.sh
to detect encoder. Can also be used for windows tests. This is
only used as part of the tests.
Had to modify Makefile to build and clean this program.
h5repack test
Description:
modified a test file generation contents, for more easy debugging
(generated just one reference dataset and one region reference )
Solution:
Platforms tested:
linux
aix
solaris
Misc. update:
new test
Description:
added a test that generates and copies a file with a dataset with fill value
(this is to test the property list function H5Pequal)
Solution:
Platforms tested:
linux
solaris
aix
Misc. update:
bug fix
Description:
the option CHUNK:NONE (remove chunking ) was not setting the layout to contiguous
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
Purpose: Small bug fix
Description: When SZIP filter is present but encoding is not enabled
test_misc21 and h5repack tests failed.
Solution: Those tests should not run in this situation at all.
Used conditonal compilation to disable the tests.
Platforms tested: sol (today I will enable the daily tests with the szip library
that doesn't have encoder for few other platforms)
Misc. update:
h5diff and h5repack changes
Description:
h5diff
introduced the following four modes of output:
Normal mode: print the number of differences found and where they occured
Report mode: print the above plus the differences
Verbose mode: print the above plus a list of objects and warnings
Quiet mode: do not print output (h5diff always returns an exit code of 1 when differences are found)
h5repack
added an extra parameter for SZIP filter (coding method)
the new syntax is
-f SZIP=<pixels per block,coding>
(pixels per block is a even number in 2-32 and coding method is 'EC' or 'NN')
Example of use:
./h5repack -i file1 -o file2 -f SZIP=8,NN -v
updated usage messages, test scripts and files accordingly
Solution:
Platforms tested:
linux
AIX
solaris
Misc. update:
h5repack changes
Description:
there were some requests to change some minor h5repack features
h5repack only made a warning about a non available filter in verbose mode ( -v )
without -v it kept silent, and users sometimes missed this warning
the request was that it should print this warning always. so, the new format, is e.g
./h5repack -i test_szip.h5 -o out.h5
Warning: dataset </dset_szip> cannot be read, SZIP filter is not available
due to this, and to avoid a lot of these messages in the shell test script, I modified
the script h5repack.sh so that it detects the presence of all filters in the environment
(previously it only detected SZIP)
the test files were also divided in more files , to make the script code easier to
follow
Solution:
Platforms tested:
linux
AIX (no szip)
solaris (no szip, no gzip )
Misc. update:
Purpose:
HDF5 now supports SZIP with no encoder.
Description:
SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled.
Solution:
Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC.
Platforms tested:
Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++)
Misc. update:
new test
Description:
add a test that generates a file with "holes" and then uses h5repack to clean
the empty space. it works
Solution:
Platforms tested:
linux
Misc. update:
bug fix, new tests
Description:
added a couple of #ifdefs to check if a specific filter is available for read
added more tests
the h5tools check filters had a mismatched filter variable
Solution:
Platforms tested:
linux, with several filter configurations (all, some , none )
Misc. update:
new tests for h5repack
Description:
added more tests both to the test program and shell script that test
a variation of different filter converssions
Solution:
Platforms tested:
linux
Misc. update:
new tests for h5repack
Description:
added tests that do layout type to layout type conversion in a matrix of 9 between compact, contiguous and chunking
Solution:
Platforms tested:
linux
afs has problems; I could not telnet to sol and copper, arabica is really slow (meaning
waiting 1 minute for a typed character) and the writing of a file gave an error
arabica 181% afs: failed to store file (145)
afs: failed to store file (145)
Misc. update:
bug fix
Description:
the synntax of the input of h5repack conatined double quotes and spaces, which
were causing problems on the parsing in AIX paralell
Solution:
replaced the spaces by =
that is, instead of -f "GZIP 6"
we have now
-f GZIP=6
Platforms tested:
linux
solaris
AIX paralell
Misc. update:
code clean, change test script
Description:
removed soem debugging messages
changed the chunking command line call to a non square dimensionality
Solution:
Platforms tested:
linux
Misc. update:
1) new function for tools library
2) new test script for h5repack
Description:
1) currently all the tools (h5dump, h5diff, etc) do not check if a filter is available
for reading some dataset that might have a filter not available on the current configuration (the behaviour
of the tools until now was to trigger a library error, saying that the dataset cannot be read
due to the lack of the filter)
Solution:
1) added a new function h5tools_canreadf that checks if a dataset can be read
depending on the availability of filters.
this function was added in calls for h5diff and h5repack.
instead of triggering the library error, a message is printed, saying that the dataset
cannot be read (the print is optional, it is on on verbose mode)
2) added a shell script that tests the commannd line tool behaviour of h5repack
the script does a series of runs of h5repack with several options on the same file (this file test4.h5
was added to the testfiles dir).
then, it runs the h5diff tool, with the input and output files , in each run.
the goal of the test is also to check item 1) . the binary file was saved with filters
that might not be available on other configurations
Platforms tested:
linux (all filters enabled)
linux (some filters disabled)
solaris (some filters disabled)
AIX (some filters disabled)
windows (all filters on and off )
Misc. update:
bug fix in H5Zshuffle.c
add more tests to h5repack that exposed the bug
Description:
when creating a dataset with the shuffle filter and duplicating it in a new dataset (file)
the call to H5Z_set_local_shuffle failed. this is because the value of cd_nelmts of the filter
structure is set to 1 (H5Z_SHUFFLE_TOTAL_NPARMS) when the original dataset is created, but when
the new dataset is created there is a checking instruction that fails if the value of
cd_nelmts is not 0 (its original value, H5Z_SHUFFLE_USER_NPARMS)
Solution:
just remove that check condition, since the value of cd_nelmts is not used anyway.
if we decide that the value of cd_nelmts is necessary, then the H5O_pline_copy function
must be changed to update this value (a different update for each filter)
Platforms tested:
linux
solaris
AIX
Misc. update:
new library function H5Pdelete_filter
deletes one or all filters from a dataset creation property list
this was done for the NONE option of h5repack, added tests for this feature
added a test for the new function in /test/dsets.c
Description:
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
added h5repack and h5diff support for copying and differences of references to dataset regions
modified the behaviour in the diff of attributes, when a difference in name is detected
in the attribute cycle (number of attributes of object), instead of exiting the
cycle, rather continue
Description:
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
h5repack change
Description:
changed the default size of chunks when defining a new dataset to apply a filter
this size is now equal to the current dimensions. previously , it was set to half this
value, and it was causing write calls to fail on some files (probably a bug)
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
Bug fix
Description:
When two property lists are compared, the H5Pequal routine was just
comparing the raw information for the property values. This causes problems
when the raw information contains pointers to other information.
Solution:
Allow a 'compare' callback to be registered for properties, so that a user
application get perform the comparison itself, allowing for "deep" compares of
the property value.
This was exported to the H5Pregister & H5Pinsert routines in the development
branch, but not the release branch.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
h5repack new features
Description:
added support/ tests for contiguous and compact layout processing
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
h5repack new features
Description:
added checking routines for the filters that were applied to the output file
added tests for szip filter
Solution:
Platforms tested:
linux
solaris
(IRIX is not available)
:
Misc. update:
h5repack new features
Description:
addded the copy routines for hard links
changed the traversal structure to store information about the hard links
added tests
Solution:
Platforms tested:
linux
solaris
IRIX
Misc. update:
