General shared library improvements for CYGWIN / AIX
Description:
Shared libraries are disabled on both CYGWIN and AIX due
to inability to build them correctly. Part of the problem
in both of these situations is the lack of the libtool
flag -no-undefined, which tells libtool that all needed
symbols are defined at link time (a requirement on these
systems) and that it's okay to build shared libraries.
Another problem are lack of dependencies between wrapper
libraries and core C HDF5 library.
This patch addresses both of these by fixing configure to
add in -no-undefined flag for libtool during linking and
adds automake dependencies in the Makefile.am files.
After testing, both CYGWIN and AIX now generate shared
libraries, but there are still some test failures in each.
(cache_api, dt_arith, and testerror.sh on CYGWIN, and
fortran tests on AIX).
Even though the shared libraries are not quite perfect,
this is a general improvement to what we had before, so
I'm applying the patch anyways. Note that default behavior
of shared libraries on these systems being disabled has
NOT been changed and requires the use of the
--enable-unsupported to attempt to build them.
We will need to address the test failures in each
architecture prior to formally supporting shared
libraries on each.
Tested:
h5committested & CYGWIN tested (on bangan)
(AIX tested by Albert on bp-login2)
Add "silent make" mode configure option.
Description:
Automake 1.11 has a new option available that allows for a
silent make mode. This functionality needs to be explicitly
enabled in configure.in via the use of the automake macro
AM_SILENT_RULES, which is what this commit is adding.
This introduces a new configure option:
--{en|dis}able-silent-rules
This option is on by default, and simplies compile and link
line outputs when building the library. Disabling this option
will print full "verbose" output (i.e., full compile and
linking lines for each target).
Tested:
This was tested on jam & h5committested
- Revise shared Fortran library disabling scenarios in configure
- Improve configure output summary
Description:
Shared Fortran libraries are not supported on Mac, but were being
disabled by configure in a way that also forced the C libraries
to be static-only. This has been fixed, so now only shared Fortran
is disabled while shared C can remain.
This prompted two additional changes:
1. While working on the check that addresses whether or not
shared Fortran libraries are allowed, removed old and no
longer needed check(s) that disable shared Fortran
libraries with HP, Intel 8, PGI, and Absoft compilers.
(Essentially, Mac is the only situation in which Fortran
shared are disabled by configure.)
2. Having two different states of libraries (i.e. shared C
library with static-only Fortran library) was not apparent
in the configure summary, which labeled all libraries as
either shared and/or static. I've added lines to both the
C++ and Fortran output sections to list shared/static-ness
of these libraries specifically.
Additionally, I've made sure that the new --enable-unsupported
configure option correctly overrides configure if it tries to
disable a shared library.
Tested:
jam, fred, & h5committest
Bring changes from Coverity branch to trunk:
r19975:
Fixed potential mem leak at H5O_attr_open_by_name
r19980:
Fix coverity issue 792.
Free tmp_env_prefix in H5Lexternal.c line 365 if it is not NULL but its contents are 0 when it goes out of scope.
r20039:
Eliminate warnings about nested extern and implicit declarations of parallel_print and address Coverity defects 712-781 by #including h5tools_utils.h in h5diff_array.c, h5diff_attr.c, h5diff_dset.c and h5diff_util.c.
r20046:
Purpose: Address TOCTOU warnings in h5jam and h5unjam
Description: Coverity is afraid that the state of the input file could change
between the call to stat() and the call to open(). This is called a time-of-
check time-of-use (TOCTOU) vulnerability. Modified stat calls to fstat which
uses an open file pointer so it (hopefully) won't complain any more.
r20047:
Addressed coverity issues 135-137, 462-464. Local pointers that needed to be freed in case of error were moved out of a switch statement in src/H5Tnative.c, set to NULL, and checked before freeing.
Tested on:
Mac OS X/32 10.6.6 (amazon) w/debug & production
(h5committested on Coverity branch)
Bug 1707 is that H5Eset_auto causes a seg fault when an application uses -DH5_USE_16_API with
the 1.8 library to compile.
I created a branch off the trunk called set_auto to continue working on the problem.
Tested on jam - backing out, simple.
through H5Eset_auto. I changed the union in the structure H5E_auto_t. Another change is to
make H5Eget_auto fail if H5Eset_auto is called to set the printing function. I'll write a
document for it.
Tested on heiwa, jam, and amani.
The property change in configure.in, config, and Makefile.am came from the merge of the 1.8
library change.
Corrected use/name of source folder aliases.
Duplicated FindMPI.cmake so that non-c++ compiler is found first (recommemded commands did not work).
Tested: local linux with mpich
Bring Coverity changes from branch to trunk:
r19161:
Fixed the part for matching the subset info with dataset
r19189:
BZ1646: h5dump does not check number of dimensions for subsetting parameters against the dataset
Changed subset_t structure from holding hsize_t pointers to holding new subset_d pointers, which hold the original hsize_t pointer + len. this len is then checked against dataset ndims in the handle_dataset function of h5dump.
Changed all references to use new data structure.
Added tests for each subset parameter.
r19190:
Added new h5dump ddl files
Tested on:
Mac OS X/32 10.6.4 (amazon) w/debug & production
(h5committested on branch)
Fixed file cleanup during make distclean / make check.
Description:
The h5jam Makefile and the h5importtestutil.sh script both included cleanup
steps that removed "*.txt" from the directory. This resulted in the new
CMake specific configuration file, CMakeLists.txt, getting blown away by
make distclean (for h5jam) and make check (for h5import).
I've removed the *.txt from the h5jam Makefile as the directory looks to be
getting cleaned appropriately without it, and I've modified the h5import
script h5importtestutil.sh to point to the specific .txt files to clean.
Tested:
jam
Enable tools lib to be built as a dll on windows. Added two get/set functions for progname and d_status.
Also add windows import/export declarations to functions.
Updated error_mesg() and warn_mesg() to remove progname argument and use get functions
Tested:
Windows, linux
Improve configure's large-file support control.
Description:
Modified configure to now attempt to add defines necessary for
supporting largefiles on all systems, instead of solely on linux. This
is in response to user requests to enable largefile support on Solaris
by default, as well as to give extra control on AIX (instead of just
jamming the necessary flag into the config files).
The old --enable-linux-lfs flag has been removed in favor of the
--enable-largefile flag (enabled by default), which can be used on all
platforms.
On systems where large files cannot be supported in this manner,
configure will report as such.
Tested:
h5committest
AIX (NCSA's blue_print machine)
duty, liberty, and linew.
Autotool Upgrade / Libtool Bug Fix
Description:
Updated autotools referenced in bin/reconfigure as follows:
Autoconf 2.64 --> Autoconf 2.65
Automake 1.11 --> Automake 1.11.1
Libtool 2.2.6a --> Libtool 2.2.6b-mcg
The referenced libtool version is a custom version of 2.2.6b. It
has been tweaked to fix a bug in libtool that occurs
when using PGI 10.0 compilers. A check incorrectly categorizes
the C++ compiler as version 1.0 instead of 10.0, and the link
line is subsequently set up incorrectly and fails to compile.
A patch has been made available and will be included in the next
release of libtool, but in the meantime I've applied the patch to a custom
installation as indicated above. This bin/reconfigure now references
the custom installation, and the resulting configure script will correctly
categorize the PGI 10.0 C++ compiler.
Ran bin/reconfigure to regenerate configure and makefiles.
Tested:
jam and amani with PGI 10.0 compilers.
h5committest
Bring revisions from Coverity fixing branch to trunk:
r18184:
Fixed Coverity issue 373. Allocated memory freed in line 762 in case of error.
r18185:
Fixed Coverity issues 357 & 358. Added check for NULL pointer before use.
r18186:
Fix coverity item 65. Added code to h5unjam to correctly handle failures in
read() and write, and also to correctly handle writes that write less than
requested.
r18187:
Fix coverity items 115 and 116. Added code to H5Tenum.c to correctly close
opened datatypes in case of failure.
r18188:
Fixed Coverity issue 46. Check that dataset->shared is not null when freeing
memory after error.
r18190:
Fix coverity item 95. Added code to H5T_create_vlen to correctly close
allocated datatype in case of failure.
r18191:
Fixed Coverity error 59. Checked sfirst for -1 value before use in line 10533.
r18192:
Fix Coverity items 121 and 28
Added Asserts:
121: assert that all dimensions of count have values greater than zero.
28: assert curr_span pointer is not null before dereference.
Note: still need too add checks in hyperslab APIs that fail
when count values are zero, and appropriate tests.
r18194:
Fixed Coverity issues 61 & 62. Checked variable snpoints for value < 0 in line
218.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug & production
(already daily tested on coverity branch)
The mixed use of RUNTEST (original) and RUNTESTS (new) caused
confusion. E.g., the timings in test/ was still using the old
$RUNTEST. It made more sense to use $RUNTEST which is used
by the dejagnu feature of automake. So, I changed all
$RUNTEST or $RUNTESTS to $RUNEXEC.
config/commence.am & config/conclude.am are the two files
that got changes. Also fixed an error in test/Makefile.am.
The rest are changed by bin/reconfigure.
Tested: h5committested.
Bring Coverity changes into the trunk: (also other minor cleanups)
r17955:
Fix Coverity item 24. Add missing error condition to
H5AC_ext_config_2_int_config.
r17956:
Fix Coverity item 24. Improve error checking in H5A_compact_build_table_cb.
r17957:
Fix Coverity item 150. Fix warning in H5A_compact_build_table_cb.
r17958:
Fix Coverity item 117. Fix error handling in H5B_shared_new.
r17959:
Fix Coverity item 209. Added an assertion for leaf->shared in
H5B2_cache_leaf_dest.
r17960:
Fix Coverity item 208. Added an assertion for internal->shared in
H5B2_cache_internal_dest.
r17961:
Fix Coverity item 89. Reworked the code to avoid array overrun in
H5C__autoadjust__ageout__insert_new_marker.
r17962:
Fix for coverity Resource_leak 195,203,204,205.
r17963:
Fix Coverity item 44. Prevented potential NULL dereference in H5D_btree_debug.
r17964:
Fix Coverity issues #197, 198 & 199: memory not being released. (Also
clean up other resource leaks in nearby and/or similar code).
r17965:
Fix Coverity issue #151: release resources on error
r17966:
Fix Coverity issue #187: Remove leftover code remnant from prior bugfix
which was causing resource leak of open files.
r17967:
Fixed Coverity issues # 193 & 194. Removed unnecessary memory allocation and
added comparison of length of path parameter to the size of the destination
buffer in h5import.h/h5import.c.
r17968:
Fix Coverity item 144. Fixed memory leak on error in H5D_chunk_copy.
r17969:
Fix for coverity Resource_leak #196.
r17970:
Coverity 167-173:
Initialized pointer of buffers.
In error handling, closed types and free memory.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.6.2 (amazon) in debug mode
Mac OS X/32 10.6.2 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
CFLAGS overhaul
Description:
Modified the way configure handles CFLAGS.
(note: all changes also apply to FCFLAGS, CXXFLAGS, CPPFLAGS, and LDFLAGS).
1. The configure process will now always preserve a user's CFLAGS
environment variable setup. Any additional flags necessary for compilation
added at configure time will be passed into the Makefiles as AM_CFLAGS,
which is an automake construct to be used in addition to CFLAGS.
This will allow a user to have the final say, as CFLAGS will always appear
later in the compile line than AM_CFLAGS. Additionally, setting CFLAGS
during make will no longer completely erase all flags set by configure,
since they're maintained in AM_CFLAGS.
2. Additionally, where possible, flags previously being assigned directly
into CFLAGS (and thus propagating into h5cc) have now been redirected into
H5_CFLAGS, so they're used ONLY for compiling hdf5, and not embedded into
the h5cc wrapper script as well.
*Note that H5_CFLAGS ultimately is assigned into AM_CFLAGS for use in the
Makefiles. Complete description of changes and build process will be
included in a Configure Document that Elena and I are working on.
3. Removed unsupported config files. This includes:
config/dec-osf*
config/hpux11.00
config/irix5.x
config/powerpc-ibm-aix4.x
config/rs6000-ibm-aix5.x
config/unicos*
4. Modified configure summary to display additional values. Specifically,
appropriate AM_* variables are being shown, as well as H5_FCFLAGS and
H5_CXXFLAGS, which were for some reason not already present.
Tested:
- H5committest
- Tested on all THG / NCSA machines, using several combinations of the more
prominent configure options (c++, fortran, szip, threadsafe, parallel,
et cetera). (Thanks to Quincey for rysnc testing setup!)
- With regards to new automated testing, anything *necessary* for
compilation will be caught by the daily tests as it stands now. (i.e.,
if LDFLAGS is not properly set when szip is used, linking will fail).
Additionally, with regards to which flags get into h5cc, if any
*necessary* flags have been improperly removed, then daily tests should
fail during make installcheck. Additional machine-specific desired
behaviors and/or checks may have to be set up separately within the
daily tests, so this is something to work on.
Updating autotools
Description:
Installed new autotools and used them to reconfigure HDF5.
- Automake upgraded to 1.11
- Autoconf upgraded to 2.64
- bin/reconfigure script edited to use new versions (on jam), and
run to generate new configure script and Makefile.in's.
- configure.in script edited to add "_cv_" to all AC_CACHE_VAL strings
(in order to comply with new autoconf standard).
Tested:
Tested on machines jam, smirom, liberty, linew.
Tested w/ features c++, fortran, parallel.
Tested w/ compilers gcc, pgcc, icc.
Further testing via Daily Tests should catch any other outliers. Upon
passing DT's, I'll propogate the new tools into 1.8, hdf4, et cetera.
Bring r17266 from 1.8 branch to trunk:
Clean up various compiler warnings from IRIX64 build
Tested on:
Mac OS X/32 (amazon) debug & production
(yes, I know it's not an IRIX64 system :-)
Too minor to require h5committest
Fix BZ #1583
Description:
The --enable-static-exec flag was broken. Configure was adding the
-all-static linker flag to an environment variable which was subsequently
never used. (looks like it was used in 1.6 Makefile.ins, but never
moved to 1.8's Makefile.ams when we added automake support).
To fix, I've added this environment varibale to the link lines of all the
tools. Now, when --enable-static-exec is invoked, it will correctly
generate statically linked executables in the installed bin directory.
Tested:
I've tested manually to ensure that the generated execs are static when
the flag is used and dynamic when not. I've also tested when szip is
used, as this was a use case mentioned in the bug report.
Also: ran full tests with make check on jam and smirom for good measure.
- Updated bin/reconfigure to use latest version of automake (1.10.2).
Re-generated Makefile.in's by running bin/reconfigure.
- Added libtool version numbers to c++, fortran, hl, hl c++, and hl fortran
libraries.
Tested:
jam, liberty, smirom
Bug Fix
Description:
Fixing BZ #1381. The --includedir=DIR configure option, which is used
to specify the installation location of C header files, did not work
correctly as the path was hard-coded in config/commence.am. I'm presuming
this is because an older version of automake didn't know where to put
c header files. In any case, removing this line now defaults the includedir
to the same directory that it is currently hard-coded to, and also fixes
the configure flag to allow customization of this value.
Tested:
jam, liberty
the way h5ls prints types, it starts searching for NATIVE types first. One solution would be h5ls not to detect these native types, using for example the same print datatype function that h5dump does, that would make the output look the same on all platforms ("32-bit little-endian integer" would be printed instead). Drawback, this "native" information would not be available. Other solution is to have not one but 2 expected outputs and make the shell script detect the endianess and compare with one output or other
tested: h5committest
Description:
- Remove need to set LD_LIBRARY_PATH when using shared szip library.
- Libtool 2.2.6a is now used to generate libraries.
- 'make check install' dependency bug is fixed, and should no longer
break the build.
- removed hard coding of shell in config/commence.am, as this causes
problems on Solaris with the new version of libtool.
- RELEASE.txt with appropriate changes.
Tested:
- kagiso, smirom, linew (merged from 1.8, pretty quick tests)
Description:
As part of our Windows cleanup, we try to remove windows-specific tweaks in the source code. There are many instances where Windows code is introduces via ifdef's. We re-evaluate whether they are still required, and found that many of them are not. Others we change to "feature"-specific code, rather than Windows-specific.
Tested:
VS2005 on WinXP
VS.NET on WinXP
h5committest (kagisopp, smirom, linew)
Description:
In library code, we try not to use system calls directly, but instead use the HD{function} macro instead. This way, we can map special versions of the call on particular systems. Previously, it was all done in H5private.h. However, in an effort to clean up platform-specific definitions, we move all of the Windows macros into a separate file, win32defs.h. This way, we can use the non-Posix versions that Visual Studio sends warnings about.
Some macros are set specifically in the platform-specific header files. Then, any macros left unset will be set by the "default" implementation in H5private.h.
This checkin also cleans up various source files to use the HD* macros when possible.
Tested:
VS2005 on WinXP
VS.NET on WinXP
h5committest (kagiso, linew, smirom)
Description:
The fortran Makefile.am used HDF_FORTRAN to indicate it is part of the
Fortran API source so that conclude.am will give fortran api prefix in the
test output. The symbox HDF_FORTRAN is also used in configure for a different
purpose (indicated --enable-fortran). They conflicted.
Similar problem for the symbol HDF_CXX.
Solution:
Changed all the involved Makefile.am to use "FORTRAN_API" instead. It is
a more appropriate name. Same for CXX_API.
Along the way, discovered that the Makefile.am of hl/fortran/test and
hl/cxx/test did not have those symbols at all. Added them in.
Platform tested:
Kagiso only. It is a trivia change.
Improvement.
Description:
src/libhdf5.settings was the initial configure summary and is installed.
Then configure is changed to dump a summary of the configure settings to
the output and also append it to src/libhdf5.settings. That created
two different output formats and duplicated information. This is the
initial attempt to clean up this confusion and unify the output format.
It is decided to use the src/libhdf5.settings template as the unified means.
This requires more macros symbols be defined. The following symbols are
all related to generating the src/libhdf5.settings file.
AC_SUBST(EXTERNAL_FILTERS)
AC_SUBST(MPE) MPE=no
AC_SUBST(STATIC_EXEC) STATIC_EXEC=no
AC_SUBST(HDF_FORTRAN) HDF_FORTRAN=no
AC_SUBST(FC) HDF_FORTRAN=no
AC_SUBST(HDF_CXX) HDF_CXX=no
AC_SUBST(CXX) HDF_CXX=no
AC_SUBST(HDF5_HL) HDF5_HL=yes
AC_SUBST(GPFS) GPFS=no
AC_SUBST(LINUX_LFS) LINUX_LFS=no
AC_SUBST(INSTRUMENT) INSTRUMENT=no
AC_SUBST(CODESTACK) CODESTACK=no
AC_SUBST(HAVE_DMALLOC) HAVE_DMALLOC=no
AC_SUBST(DIRECT_VFD) DIRECT_VFD=no
AC_SUBST(THREADSAFE) THREADSAFE=no
AC_SUBST(STATIC_SHARED)
AC_SUBST(enable_shared)
AC_SUBST(enable_static)
AC_SUBST(UNAME_INFO) UNAME_INFO=`uname -a`
The src/libhdf5.settings.in has CONDITIONAL's added to it too. The
untrue conditions turned into a "#" and these lines are cleaned by the
post processing script.
Platform tested:
h5committest on kagiso, smirom and linew.
Description: Applying update to autotools that was applied to 1.8 a couple
of weeks ago to the trunk.
Updated bin/reconfigure script to reflect the new versions of
libtool and automake in the /home1/packages/ directory.
Rearranged configure.in script. When using libtool 2.2.2, the
libtool script doesn't generate until later in the configuration
process, so I had to move a test that parsed through the libtool
script to a point after where it was actually being generated.
Ran libtoolize on the project, and ran bin/reconfigure to
regenerate configure and Makefile.in's throughout.
Tested: kagiso, smirom, linew (h5committest)
