* commit '54957d37f5aa73912763dbb6e308555e863c43f4':
Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes.
Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh.
Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1
Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
Clean up warnings (from 2774 -> 1560, with my standard debug build)
Tested on:
MacOSX/64 10.11.5 (amazon) w/serial, parallel & production
(h5committest forthcoming)
gcc 4.9.2 was used to create the warning list
- implicit casts
- shadowed variables
- various enum issues
- other minor fixes (comments, unused macros, etc.)
Tested on: h5committest
Address HDFFV-7942 - h5diff: incorrect result for comparing attribute data with different type size in same class
Description:
When comparing attribute data values with same type class but different size, the result was incorrect. It was due to the size difference and got truncated.
Fixed to match up the smaller type size to big type size like what dataset does.
Tested:
jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows (32-LE cmake), cmake (jam)
HDFFV-7770 - h5diff segmentation fault on JPSS files
Description:
Fixed segfault over non-comparable attribute with different dimention or
rank, along with '-c' option to display details.
Tested:
jam (linux32-LE), koala (linux64-LE), heiwa (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE), Windows, Cmake
Bring changes from Coverity branch to trunk:
r19975:
Fixed potential mem leak at H5O_attr_open_by_name
r19980:
Fix coverity issue 792.
Free tmp_env_prefix in H5Lexternal.c line 365 if it is not NULL but its contents are 0 when it goes out of scope.
r20039:
Eliminate warnings about nested extern and implicit declarations of parallel_print and address Coverity defects 712-781 by #including h5tools_utils.h in h5diff_array.c, h5diff_attr.c, h5diff_dset.c and h5diff_util.c.
r20046:
Purpose: Address TOCTOU warnings in h5jam and h5unjam
Description: Coverity is afraid that the state of the input file could change
between the call to stat() and the call to open(). This is called a time-of-
check time-of-use (TOCTOU) vulnerability. Modified stat calls to fstat which
uses an open file pointer so it (hopefully) won't complain any more.
r20047:
Addressed coverity issues 135-137, 462-464. Local pointers that needed to be freed in case of error were moved out of a switch statement in src/H5Tnative.c, set to NULL, and checked before freeing.
Tested on:
Mac OS X/32 10.6.6 (amazon) w/debug & production
(h5committested on Coverity branch)
Bring Coverity changes from branch to trunk:
r19161:
Fixed the part for matching the subset info with dataset
r19189:
BZ1646: h5dump does not check number of dimensions for subsetting parameters against the dataset
Changed subset_t structure from holding hsize_t pointers to holding new subset_d pointers, which hold the original hsize_t pointer + len. this len is then checked against dataset ndims in the handle_dataset function of h5dump.
Changed all references to use new data structure.
Added tests for each subset parameter.
r19190:
Added new h5dump ddl files
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(h5committested on branch)
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
formatted code
bug fix: the 1.6 branch did not have a test for the existence of long double type on print_type (print name of datatype)
tested: windows, linux
Finish deprecating last H5G symbol (H5G_obj_t) - yay!
Lots of misc. library fixes to remove confusion between links and
objects. The tools could still use another pass, to remove h5trav_type_t type
and make the correct distinction between links & objects.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
AIX/32 5.3 (copper) w/FORTRAN, w/parallel, in production mode
Mac OS X/32 10.4.10 (amazon) in debug mode
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
Users can create external links using H5L_create_external(). These links
point to an object in another HDF5 file. Users can alter the behavior of
external links or create new kinds of links by registering callbacks
using the H5L interface.
Added tests, tools support, etc.
Also a number of other, minor changes have been made (some restructuring of
the H5L interface, for instance).
Additional documentation and examples are forthcoming.
Code cleanup
Description:
Trim trailing whitespace in Makefile.am and C/C++ source files to make
diffing changes easier.
Platforms tested:
None necessary, whitespace only change
new feature
Description:
added the printout of the compression ratio for filters in h5repack, after the filter name, obtained with H5Dget_storage_size, before and after applying the filter
Solution:
Platforms tested:
linux
solaris
AIX
Misc. update:
bug fix
Description:
1) added a more explainative usage message
2) the percent relative error for the integer type (division) was being done using integer arythmetic; use floating point arythmetic instead
3) added a new test for integer percent
Solution:
Platforms tested:
linux (32,64)
AIX
solaris
Misc. update:
bug fix
Description:
the compare check for the datatype sign was not done in the correct place, causing invalid
comparisons to be made
Solution:
put it on the correct place
Platforms tested:
linux 32, 64
AIX
Misc. update:
New feature
Description:
Check in baseline for compact group revisions, which radically revises the
source code for managing groups and object headers.
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
This initiates the "unstable" phase of the 1.7.x branch, leading up
to the 1.8.0 release. Please test this code, but do _NOT_ keep files created
with it - the format will change again before the release and you will not
be able to read your old files!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
Solution:
There's too many changes to really describe them all, but some of them
include:
- Stop abusing the H5G_entry_t structure and split it into two separate
structures for non-symbol table node use within the library: H5O_loc_t
for object locations in a file and H5G_name_t to store the path to
an opened object. H5G_entry_t is now only used for storing symbol
table entries on disk.
- Retire H5G_namei() in favor of a more general mechanism for traversing
group paths and issuing callbacks on objects located. This gets us out
of the business of hacking H5G_namei() for new features, generally.
- Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t
- Lots more...
Platforms tested:
h5committested and maybe another dozen configurations.... :-)
Bug fix
Description:
ph5diff had been hanging in Tflops. Found out that vsnprintf ph5diff
uses was a local coded that did not limit printing according to the size
argument. That resulted in buffer overflow and other problems.
Solution:
Added some sort of size checking in the home-grown vsnprintf and
had ph5diff checked for error return of vsnprintf. Leon also
revamped the ph5diff manager's way of handling communications with
the workers. That eliminated all but the last case of hanging.
Platforms tested:
Tflops.
Misc. update:
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Bug fix for parallel case for new "Some objects were not comparable" patch to h5diff.
Description:
The parallel h5diff wouldn't print out "Some objects were not comparable" because the worker
tasks were not communicating the not_cmp flag of the diff_opt_t struct back to the manager, who ultimately
prints everything.
Also, some miscellaneous fixes for error printing. Some errors were printed out with printf instead of fprintf(stderr,...).
In parallel environments, this can result in output getting lost.
Solution:
Had the worker tasks pass along the not_cmp flag to the manager when they sent along
the number of differences they found.
Platforms tested:
heping (pp), sol (pp)
Misc. update:
bug fix
Description:
when 2 objects were not comparable, the final print information for the non verbose mode printed "0 differences found"
Solution:
replaced instead with a Summary message that says
"Some objects were not comparable"
Platforms tested:
linux
solaris
Misc. update:
Description:
while compiling in mir with pgcc -O2 a condition
if (name ) {
do_something(name);
}
is executed when name is NULL
Solution:
avoid the error by checking the NULL pointer inside do_something(name);
Platforms tested:
linux (mir with pgcc 6.0)
Misc. update:
Bug fix
Description:
Typo in #ifdef to check for snprintf functionality.
Solution:
That should have been H5_VSNPRINTF_WORKS.
Platforms tested:
heping, modi4
Misc. update:
Purpose:
Bug fix.
Description:
ph5diff fails on modi4 due to the way snprintf works on IRIX.
Solution:
The C99 standard says that, if there isn't enough room in the string,
snprintf should return the number of characters that
would have been written to the output string if there were enough room.
The snprintf on modi4 would return the number of characters that is was able to write
succesfully to the string if space ran out. The ph5diff logic that checks if
the output buffer was full did not handle this sort of return value correctly.
Used VSNPRINTF_WORKS from configure test to check how snprintf works and do
the logic accordingly.
Platforms tested:
modi4
Misc. update:
Bug Fixes
Description:
Fixes for several bugs, including dumping of excess output to a temporary file, fix for printing
hsize_t datatype, and the long awaited fix for intermixed output.
Solution:
Fix 1: Overflow file
Previously, any output that a worker task made was buffered locally in memory, up to a point. Any
output beyond the size of the buffer (used to be 10k) was discarded. Now, the memory buffer size has been
changed to 1k and any output beyond this amount is sent a temporary file. This way, no output is lost
and memory usage is kept under control. The temporary file is deleted as soon as a worker task finishes
sending its contents to the manager.
Fix 2: hsize_t printing
Printing of the hsize_t datatype used to be handled by %Hu passed to HDfprintf. However, there is no corresponding HDvsnprintf that
is able to print hsize_t types. These are now printed with the aid of H5_PRINTF_LL_WIDTH.
Fix 3: Intermixed output fix
Intermixed output would occur on some machines (although I haven't seen it happen for a while) due to the unpredictability of the underlying network
and the speed at which various message would travel. This has been fixed by having all output send to the manager
for printing. The worker tasks no longer print the output themselves upon receipt of a token, but instead
send that data to the manager.
Platforms tested:
heping, eirene, tg-login (the only place that seems to still experience intermixed output every now and then)
Misc. update:
Correction to previous print_manager_output bugfix
Description:
Although the previous bugfix did fix all the errors, it was not the correct way
to fix the problem. Since the manager will only buffer output when run in
parallel mode, in serial mode this function should not even be called.
Solution:
Wrapped print_manager_output in #define's to prevent it from being included in
serial builds.
Platforms tested:
heping (serial + parallel)
Misc. update:
Fixed numerous ph5diff bugs.
Description:
Fixed manager output printing
Fixed out of order output printing
Fixed test script execution problem
Temporary fix for large amounts of output overflowing buffer.
Solution:
The manager task buffers its output. However, since the manager task
never gets a print token, this output was lost. Solution: new function called
print_manager_output that prints buffered output is called in places where the
manager buffers its output.
printf was apparently buffering output. This means that a task would sometimes
print even after it had given up its print token. Added fflush() call after
printf() calls, which seems to have fixed the problem.
calling rsh multiple times in succession seems to overwhelm something in Linux,
as it begins to refuse new connections until the old ones reset. Since each
call to mpirun in the test script starts up 4 rsh sessions, the test script
eventually is unable to run any further tests. Solution: Added a short delay in the
testscript between successive calls to mpirun to allow old connections to reset.
The 10k output buffer was of insufficient size to hold the large amounts of
output generated by some of the tests. Since code to buffer to a file has not
been implemented yet, a temporary fix was to increase the size of the output
buffer to 50k.
Platforms tested:
heping
Misc. update:
Bug Fix
Description:
Permanent fix for the incompatibilities between h5diff and h5repack.
Solution:
h5diff now contains the code to run both parallel and serial diffs.
Depending on how the binary is called, it will run either the serial or
the parallel versions respectively.
Platforms tested:
heping(serial + parallel), copper.
Misc. update:
fix bugs.
Description:
The three variables, g_nTasks, outBuff, and outBuffOffset, were
declared in ph5diff.h. That means they are declared in various
*.c files that include it, thus having the appearance of multiple
defined. Most C compilers are forgiving of this error but
the g++ rejects this practice.
Solution:
Make it only an extern declaration in ph5diff.h but really
defined them in h5diff_util.c
Platforms tested:
sleipnir using g++ since this only failed in Daily test.
Also testes in heping, serial and parallel modes.
Description:
the print_paralell h5diff function had a call to vsnprintf.
this causes a linking error on Windows
Solution:
replaced with HDvsnprintf
Platforms tested:
windows (vs6)
linux
Misc. update:
Bug fix: Temporary fix for h5repack failures in all parallel builds.
Description:
The parallel additions to h5diff interfered with h5repack.
Solution:
Added a second set of "parallel" functions to h5diff.c. h5repack uses the serial versions, whereas h5diff will use the parallel versions.
Also, h5diff will now be smart about when to enter parallel mode. If is run with mpirun with more than 1 task, it will enter parallel mode. Otherwise, it will stay in serial mode as before.
Platforms tested:
heping (serial and parallel)
Misc. update:
New feature.
Description:
Added new tool ph5diff. (Code done by Leon Arber.)
Code is changed but test is not working yet. For now,
it skipped all tests.
Platforms tested:
Tested in heping, serial and parallel modes.
