Bug Fixes
Description:
Fixes for several bugs, including dumping of excess output to a temporary file, fix for printing
hsize_t datatype, and the long awaited fix for intermixed output.
Solution:
Fix 1: Overflow file
Previously, any output that a worker task made was buffered locally in memory, up to a point. Any
output beyond the size of the buffer (used to be 10k) was discarded. Now, the memory buffer size has been
changed to 1k and any output beyond this amount is sent a temporary file. This way, no output is lost
and memory usage is kept under control. The temporary file is deleted as soon as a worker task finishes
sending its contents to the manager.
Fix 2: hsize_t printing
Printing of the hsize_t datatype used to be handled by %Hu passed to HDfprintf. However, there is no corresponding HDvsnprintf that
is able to print hsize_t types. These are now printed with the aid of H5_PRINTF_LL_WIDTH.
Fix 3: Intermixed output fix
Intermixed output would occur on some machines (although I haven't seen it happen for a while) due to the unpredictability of the underlying network
and the speed at which various message would travel. This has been fixed by having all output send to the manager
for printing. The worker tasks no longer print the output themselves upon receipt of a token, but instead
send that data to the manager.
Platforms tested:
heping, eirene, tg-login (the only place that seems to still experience intermixed output every now and then)
Misc. update:
Automake version upgrade
Description:
Upgraded automake version from 1.6.2 to 1.9.5.
Changed bin/reconfigure script to use automake 1.9.5.
Changed configure.in and Makefiles to use new FCFLAGS and FC variables
instead of FFLAGS and F9X.
Automake and configure should now do the lion's share of the work
supporting Fortran 9X; macros in acsite.m4 are now mostly unused (will
be cleaned later).
Altered how configure handles pmake; now root-level Makefile.in is
processed by bin/reconfigure to have a .MAKEFLAGS target, since
automake no longer allows us to define unused variables.
Configure now always checks for C++ compiler even if it is not
used, since automake thinks this is the Right Thing To Do and
will break otherwise.
Platforms tested:
Sol, copper, heping, mir, sleipnir, eirene, pommier, kelgia, modi4.
Bug fix
Description:
The GASS VFL driver header file was bringing in the <string.h> header file,
which several other source code modules needed also, but weren't including
explicitly themselves.
Solution:
Add includes for <string.h> to files which actually need them.
Platforms tested:
FreeBSD 4.11 (sleipnir) w/C++ as CC
Configuration not tested by h5committest...
Bug fix.
Description:
MPI_LONG_LONG, which is not standard yet, was used to pass the
nubmer of differences found. This was needed because number of
differences is defined as type hsize_t which can be arbitary large
such that there is no MPI type that matches it. The value is passed
between processes as an array of bytes in order to be portable. But this
may not work in non-homogeneous MPI environments.
This fix was actually Leon's idea.
Platforms tested:
Tested in QSC in which this failed.
Removed GASS configuration and software packages associated with
it. The following software are no longer configurable.
checking for main in -lcrypto... yes
checking for SSL_get_version in -lssl... yes
checking globus_common.h usability... yes
checking globus_common.h presence... yes
checking for globus_common.h... yes
checking for globus_module_activate in -lglobus_common... yes
checking for main in -lglobus_gass_cache... yes
checking for main in -lglobus_gaa... yes
checking for main in -lglobus_gss... yes
checking for main in -lglobus_gss_assist... yes
checking for main in -lglobus_io... yes
checking for main in -lglobus_gass_transfer_assist... yes
checking for main in -lglobus_gass_transfer... yes
checking for globus_gass_open in -lglobus_gass_file... yes
Platforms tested:
h5committested (but copper was down.) tested in modi4 too.
bug fix.
Description:
When MPE library is used, it prints two extra message lines that
interfere with the expected output.
Solution:
Filter out those two lines of text from stderr.
Also added a provision to print the whole generated output when failure
is detected.
Platforms tested:
Tested in copper, serial, parallel and parallel with MPE.
Misc. update:
bug fix/new feature.
Description:
(committing changes made by Leon.)
ph5diff now is a real program by itself. It has its
own main (ph5diff_main.c).
Codes common to h5diff and ph5diff are in h5diff_common.c.
Removed the hack that hide failures from testh5diff.sh.
Platforms tested:
heping(serial, pp), sol (pp), copper(pp).
Misc. update:
Purpose:
Bug fix
Description:
modi4 dies during build with strange errors.
The root cause of these is a two-year-old hack in HDF5's libtool
script that only takes effect on IRIX.
Solution:
Edited the libtool hack (by editing ltmain.sh) to correct a bug in
the hack.
Also made sure that compiler-specific DEFAULT_LIBS are used
when linking.
Platforms tested:
sleipnir, copper, modi4, sol
Bug fix.
Description:
ph5diff is still being fixed.
Changed this to report all failures but will not flag them as
errors so that daily tests can continue. Will be fixed soon.
Platforms tested:
Tested in sol (pp) where it has most failures.
Misc. update:
Purpose:
Bug fix
Description:
Fixed the error on copper where output would match, except the worker tasks are forcefuly terminated at the end.
Solution:
If an exit code other than 0 was returned, the
parallel environment would assume that there
was an error and would terminate all other
tasks. Because h5repack relies on the return
value of h5diff, it was necessary to return
values other than 0. Fixed by having parallel
runs of h5diff always return 0, whereas the
return value of serial runs is unchanged.
Platforms tested:
copper
Misc. update:
Bug fix
Description:
Arabica exhibited strange errors when linker found wrong versions of
header files. This happened because include directories were
given to linker in the wrong order.
Solution:
Move include directories from AM_CFLAGS variable to INCLUDES
variable to put them before CPPFLAGS variable. Trust me, it works.
This bug may also have contributed to strange errors on other platforms
(kelgia?).
Platforms tested:
copper, sleipnir, arabica.
(h5dump broke while building on arabica, but this happened in
a clean checkout, too).
Bug fix
Description:
pmake (on modi4, for instance) complains about undefined variables
if it is run without the -V flag, which turns those errors to warnings.
Solution:
Added test to configure.in to see if $MAKE will allow Makefiles
with undefined variables. If not, sets -V flag in AM_MAKEFLAGS.
Ensured that all custom make targets use AM_MAKEFLAGS.
Also defined all variables that caused errors in top-level Makefile.am.
This means that pmake can be used to build hdf5 *from the top level
only*. To run make from a subdirectory, still need to use -V flag
(or use make or gmake).
Platforms tested:
modi4, heping, copper, sleipnir
bug fix.
Description:
Need eval before the RUNCMD command because some machines like
AIX, has RUNPARALLEL in the style as
MP_PROCS=3 MP_TASKS_PER_NODE=3 poe ./a.out
that throws the shell script off.
Platforms tested:
Tested in copper (pp) where it failed before.
Tested in heping pp too.
Misc. update:
Bug fixed.
Description:
Heping used to have a network limit that if too many rsh connections
occurred in a second, it stopped accepting connections for 30 seconds.
That caused testph5diff to fail when too many mpi jobs are executed
repidly. A sleep 2 was in to go around this problem. Heping's limit
has been raised, thus no more need for this hack. Sleep 2 is removed.
Platforms tested:
Tested in heping (pp).
Misc. update:
Correction to previous print_manager_output bugfix
Description:
Although the previous bugfix did fix all the errors, it was not the correct way
to fix the problem. Since the manager will only buffer output when run in
parallel mode, in serial mode this function should not even be called.
Solution:
Wrapped print_manager_output in #define's to prevent it from being included in
serial builds.
Platforms tested:
heping (serial + parallel)
Misc. update:
Purpose:
bug fix
Description:
a string with 2 characters was declared as having size 2. changed to size 3 (+ null character)
this was causing compiler warnings with g++
Solution:
Platforms tested:
linux
free bsd (g++)
Misc. update:
Fixed numerous ph5diff bugs.
Description:
Fixed manager output printing
Fixed out of order output printing
Fixed test script execution problem
Temporary fix for large amounts of output overflowing buffer.
Solution:
The manager task buffers its output. However, since the manager task
never gets a print token, this output was lost. Solution: new function called
print_manager_output that prints buffered output is called in places where the
manager buffers its output.
printf was apparently buffering output. This means that a task would sometimes
print even after it had given up its print token. Added fflush() call after
printf() calls, which seems to have fixed the problem.
calling rsh multiple times in succession seems to overwhelm something in Linux,
as it begins to refuse new connections until the old ones reset. Since each
call to mpirun in the test script starts up 4 rsh sessions, the test script
eventually is unable to run any further tests. Solution: Added a short delay in the
testscript between successive calls to mpirun to allow old connections to reset.
The 10k output buffer was of insufficient size to hold the large amounts of
output generated by some of the tests. Since code to buffer to a file has not
been implemented yet, a temporary fix was to increase the size of the output
buffer to 50k.
Platforms tested:
heping
Misc. update:
Bug fix
Description:
h5diff sometimes will 'exit(status)' but if it is in parallel mode,
it needs to close up the workers and the MPI environment before exit(..).
Solution:
Created h5diff_exit() that will just exit in serial mode but does all
the shutdown properly if in parallel mode.
Platforms tested:
tested in heping, serial and parallel.
Misc. update:
Bug fix
Description:
h5diff_main.c contained some debug printf statment that produced
unexpected output. Commented them out.
testh5diff.sh: implemented code to test ph5diff tests but
ph5diff has some errors that generated unexpected output.
Temporary hacked to skip tests if run by daily test account.
Platforms tested:
Tested in heping, serial and parallel.
new features
Description:
Added a parallel option to testh5diff.sh to do parallel h5diff tests.
Added testph5diff.sh to invoke testh5diff.sh with parallel mode.
Platforms tested:
Tested in heping (serial and pp).
Features
Description:
Added the option to build ph5diff and run testphdf5.sh when
parallel mode is enabled.
Platforms tested:
tested in heping, both serial and parallel.
Misc. update:
New feature
Description:
Add basic code for new B-tree implementation. They don't do much yet,
aren't hooked up to anything yet and the format may change, but I'd like to
start getting them into the daily tests.
Platforms tested:
FreeBSD 4.11 (sleipnir) w/parallel
Too minor to require h5committest
Bug Fix
Description:
Permanent fix for the incompatibilities between h5diff and h5repack.
Solution:
h5diff now contains the code to run both parallel and serial diffs.
Depending on how the binary is called, it will run either the serial or
the parallel versions respectively.
Platforms tested:
heping(serial + parallel), copper.
Misc. update:
Bug fix
Description:
Found the permanant fix to automake/CVS dependency problem
Solution:
Added AM_MAINTAINER_MODE macro to configure.in.
Now automake will never try to regenerate Makefiles, Makefiles.in,
configure, H5config.h, etc. when they are out of date, nor will it
print any warnings.
Developers should be very very careful to use reconfigure script,
and can add --enable-maintainer-mode flag to configure on heping
to regenerate these files correctly.
Platforms tested:
heping sleipnir copper
Bug fix
Description:
Configuration files' timestamps were incorrect.
Solution:
Update Makefiles.in. Also added correct paths to autotools on heping,
so heping build should be able to re-generate configuration files even
if they are still broken.
Platforms tested:
sleipnir
Bug fix
Description:
Dependencies between configure files (aclocal, configure.in, configure,
Makefiles.am and .in) are still causing Makefiles to try to run autotools
during build.
Solution:
Committed all Makefiles.in to update their timestamps.
As a temporary measure, forcibly prevented automake from running
autotools during build by redefining the variables it uses.
Platforms tested:
sleipnir (No changes to Makefile content)
Bug fix
Description:
Different compilers use different flags to include Fortran module files
Solution:
Changed boilerplate to use configure variable rather than hardcoded -M flag.
Platforms tested:
sleipnir, sol, copper
Configuration feature
Description:
HDF5 now uses automake to generate Makefiles
Solution:
Makefile.in files are now generated from Makefile.am files.
To reconfigure (after chaning a Makefile.am or configure.in):
/bin/sh bin/reconfigure.sh
Platforms tested:
Many
Temporary daily tests fix.
Description:
The newly added ph5diff is not quite compatible with the h5repack tool.
Disabled the ph5diff build (reverted back to building serial h5diff)
to allow daily tests to work while ph5diff is fixed.
Platforms tested:
Tested in heping both serial and parallel modes.
Misc. update:
Bug fix: Temporary fix for h5repack failures in all parallel builds.
Description:
The parallel additions to h5diff interfered with h5repack.
Solution:
Added a second set of "parallel" functions to h5diff.c. h5repack uses the serial versions, whereas h5diff will use the parallel versions.
Also, h5diff will now be smart about when to enter parallel mode. If is run with mpirun with more than 1 task, it will enter parallel mode. Otherwise, it will stay in serial mode as before.
Platforms tested:
heping (serial and parallel)
Misc. update:
New feature.
Description:
Added new tool ph5diff. (Code done by Leon Arber.)
Code is changed but test is not working yet. For now,
it skipped all tests.
Platforms tested:
Tested in heping, serial and parallel modes.
h5diff and h5repack changes
Description:
h5diff
introduced the following four modes of output:
Normal mode: print the number of differences found and where they occured
Report mode: print the above plus the differences
Verbose mode: print the above plus a list of objects and warnings
Quiet mode: do not print output (h5diff always returns an exit code of 1 when differences are found)
h5repack
added an extra parameter for SZIP filter (coding method)
the new syntax is
-f SZIP=<pixels per block,coding>
(pixels per block is a even number in 2-32 and coding method is 'EC' or 'NN')
Example of use:
./h5repack -i file1 -o file2 -f SZIP=8,NN -v
updated usage messages, test scripts and files accordingly
Solution:
Platforms tested:
linux
AIX
solaris
Misc. update:
Bug fixes
Description:
Updated dependencies
Fixed error with C++ compiler builds of main library
Added H5Pset_data_transform to MPE info
Platforms tested:
FreeBSD 4.9 (sleipnir)
Code cleanup
Description:
Clean up lots of warnings based on those reported from the SGI compilers
as well as gcc.
Platforms tested:
SGI O3900, IRIX64 6.5 (Cheryl's SGI machine)
FreeBSD 4.9 (sleipnir) w/ & w/o parallel
h5committest
bug fix continuation
Description:
the atof return value on a hexadecimal input is different
on some systems; before checking for the atof return value
do a character check for the first 2 characters of the string input
Solution:
Platforms tested:
linux
solaris
Misc. update:
bug fix (sort of)
Description:
apparently linux and other systems (solaris) return a different value from atof if the
argument is in hexadecimal .
this return value was used to test if the argument to -p and -d was a valid floating point
number
for now , commented the calls in the script with -p <hexa> until we find a portable solution
Solution:
Platforms tested:
linux
solaris
windows
Misc. update:
bug fix
Description:
the -p option was not parsing correctly if the input was in the exponential format
Solution:
add a new parse test for floating point numbers
Platforms tested:
linux
Misc. update:
code clean, bug fix
Description:
the attributes of the root group were not being compared
removed compiler warnings on IRIX and solaris
Solution:
added a special function to compare the attributes of the root group
Platforms tested:
linux
IRIX 6.5
solaris 2.7
Misc. update:
h5diff new features
Description:
added comparison for attributes
adeded comparison for all dataset datatypes
added tests for the new features
changed the output format
Solution:
Platforms tested:
linux
solaris 5.7
IRIX 6.5 (64)
Misc. update:
h5diff new feature
Description:
added compare for attributes
a new options flag (-a) was added to the options structure. it is 0 by default (no compare )
the output of the compare is the same that for datasets, and all the other flags also apply for attributes
(the memory compare is done in the same function diff_array)
all the other requirements for compare of datasets (type, space) are identical too
Platforms tested:
linux
solaris 2.7
IRIX
Misc. update:
