Purpose:
Remove H5_MPI_SPECIAL_COLLECTIVE_IO_WORKS and
H5_MPI_COMPLEX_DERIVED_DATATYPE_WORKS #defines from source.
Description:
Two advanced parallel functionalities, special collective IO and
complex derived datatypes, are not supported by older
implementations of mpi, and thus our code limits the use of these
features with #ifdefs and has checks in configure to set them (or
not). Unfortunately, configure can't actually run a parallel check
to see if these features are working (nor not) so it resorts to
looking in the config files where they are explicity enabled or
disabled based on versions of mpi, sytems being built on, or for
no documented reason at all (i.e. just set to on or off as some
'default'). Overriding these settings is easy if need be, provided
it is known that it needs to be done to get improved performance,
and oftentimes it is not.
Most new MPI implementations successfully handle the functionality
requested when these #defines are set, and many of the "turn these
features off" cases in the config files are for old (> 5 years)
versions of MPI and retired systems (such as NCSA's tungsten).
Therefore, the decision has been made to remove the support for
these old versions of MPI and systems that cannot handle these
behaviors. The #ifdefs and supporting setup in the config/ files
and configure script has been removed, and the code executed when
these options were not set removed from the source.
In passing, this commit also cleans up some whitespace issues in
both t_mpi.c and H5Dmpio.c. Furthermore, in t_mpi.c, the special
collective IO test was not getting regularly run due to it being
written to work only with four processes (we regularly test with
six, previously with three), and thus it failed when actually run
due to an out of bounds data buffer assignment. It has been
modified to run at any number of processes greater than four, and
the memory problem has been fixed so the test passes.
Tested:
jam, h5committest, ember
CFLAGS overhaul
Description:
Modified the way configure handles CFLAGS.
(note: all changes also apply to FCFLAGS, CXXFLAGS, CPPFLAGS, and LDFLAGS).
1. The configure process will now always preserve a user's CFLAGS
environment variable setup. Any additional flags necessary for compilation
added at configure time will be passed into the Makefiles as AM_CFLAGS,
which is an automake construct to be used in addition to CFLAGS.
This will allow a user to have the final say, as CFLAGS will always appear
later in the compile line than AM_CFLAGS. Additionally, setting CFLAGS
during make will no longer completely erase all flags set by configure,
since they're maintained in AM_CFLAGS.
2. Additionally, where possible, flags previously being assigned directly
into CFLAGS (and thus propagating into h5cc) have now been redirected into
H5_CFLAGS, so they're used ONLY for compiling hdf5, and not embedded into
the h5cc wrapper script as well.
*Note that H5_CFLAGS ultimately is assigned into AM_CFLAGS for use in the
Makefiles. Complete description of changes and build process will be
included in a Configure Document that Elena and I are working on.
3. Removed unsupported config files. This includes:
config/dec-osf*
config/hpux11.00
config/irix5.x
config/powerpc-ibm-aix4.x
config/rs6000-ibm-aix5.x
config/unicos*
4. Modified configure summary to display additional values. Specifically,
appropriate AM_* variables are being shown, as well as H5_FCFLAGS and
H5_CXXFLAGS, which were for some reason not already present.
Tested:
- H5committest
- Tested on all THG / NCSA machines, using several combinations of the more
prominent configure options (c++, fortran, szip, threadsafe, parallel,
et cetera). (Thanks to Quincey for rysnc testing setup!)
- With regards to new automated testing, anything *necessary* for
compilation will be caught by the daily tests as it stands now. (i.e.,
if LDFLAGS is not properly set when szip is used, linking will fail).
Additionally, with regards to which flags get into h5cc, if any
*necessary* flags have been improperly removed, then daily tests should
fail during make installcheck. Additional machine-specific desired
behaviors and/or checks may have to be set up separately within the
daily tests, so this is something to work on.
Use a slightly less efficient method of computing the log2() on SGI IRIX64,
in order to avoid a compiler bug when optimizations are turned on.
Tested on:
SGI IRIX64 6.5 (atlantia)
work-around solution for special cases in collective IO
Description:
For collective IO
if some processes don't have any contributions to IO, even with the new SGI compiler(7.4.4m), the testing program will be hanged. Previously only thought it didn't work with old compiler. It turned out not to be true.
Solution:
Turned off the special flag: H5_MPI_SPECIAL_COLLECTIVE_IO_WORKS
Platforms tested:
UCAR IRIX 6.5
this file only affects IRIX 6.x system. No need to test at other platforms.
Misc. update:
Purpose:
H5CC feature
Description:
Made H5_CFLAGS change to several more configuration files.
Solution:
Moved nonessential flags from CFLAGS to H5_CFLAGS for several more
platforms. If my guesses about which flags were nonessential were
wrong, we'll have Daily Test errors.
Platforms tested:
mir, cray, copper, juniper (OS X)
Description: In the past we had Fortran support only for 64bit mode
on IRIX64-6.5. This feature was hardcoded in the IRIX configuration file.
New automatic Fortran datatypes configuration should support both
32bit and 64bit modes. I finally verified that it worked.
Solution: Removed -64 flag from the configuration file.
Platforms tested: UCAR IRIX64-6.5 in 32 and 64bit modes.
Misc. update:
Purpose:
bug fix
Description:
modi4's sgi compiler is too old to support collective IO when there are no contributions from
some processes.
Solution:
Disable mpi_special_collective_io_work macro.
Platforms tested:
NCAR SGI
MODI4
COOPER
Misc. update:
Description:
Add source code copyright notice; comment symbols vary by type of file.
Platforms tested:
No changes that should cause any problems. Nonetheless, the tree will
be checked out fresh and tests run immediately post-commit.
Code cleanup
Description:
Removed Fortran type sizes from config/* files.
Solution:
These lines weren't doing anything anymore; they've been replaced by automatic
type generation. This is just housekeeping to remove them from
the config/* files.
Platforms tested:
mir, sol, modi4
Description: Added 2 new configure options, --enable-exception and --enable-accuracy.
--enable-exception lets the library check whether user's exception handling
functions are present during compiler data conversions and use them if they are.
When it's disabled, this step is skipped to improve conversion speed. This
step isn't implemented yet for soft conversions because there would be little
gain in speed.
--enable-accuracy guarantees data accuracy during data conversions. It means
the library will choose compiler conversions only if the accurate data is
secured. Otherwise, the library will go for the library's own conversions. If
this option is disabled, the library uses compiler conversions in favor of
their speed as long as they work even if data can be incorrect.
Platforms tested: h5committest and fuss. Some systems may fail after this
checkin.
improvement.
Description:
hdf5_mpi_complex_derived_datatype_works is hardset to no but
newer IRIX64 systems have it working. Changed the code to
detect the compiler version and set hdf5_mpi_complex_derived_datatype_works
to no for older compilers only.
Platforms tested:
modi4 for the no effect.
Misc. update:
Purpose:
Fix the daily test failure of collective irregular test in modi4.
Description:
The original test passed with new SGI compiler. Albert verified this from
the SGI machine at DOE. The compiler version number at modi4 is 7.3.1.3m, the compiler version number at DOE is 7.4.2m. compiler at modi4 is old and SGI may fix some bugs during the updating of their compiler.
Solution:
At configuration file IRIX.6.x, set the macro MPI_COMPLEX_DERIVED_DATATYPE to no. In that way, HDF5 library won't call derived datatype routine in the test.
Platforms tested:
modi4
Misc. update:
Automake version upgrade
Description:
Upgraded automake version from 1.6.2 to 1.9.5.
Changed bin/reconfigure script to use automake 1.9.5.
Changed configure.in and Makefiles to use new FCFLAGS and FC variables
instead of FFLAGS and F9X.
Automake and configure should now do the lion's share of the work
supporting Fortran 9X; macros in acsite.m4 are now mostly unused (will
be cleaned later).
Altered how configure handles pmake; now root-level Makefile.in is
processed by bin/reconfigure to have a .MAKEFLAGS target, since
automake no longer allows us to define unused variables.
Configure now always checks for C++ compiler even if it is not
used, since automake thinks this is the Right Thing To Do and
will break otherwise.
Platforms tested:
Sol, copper, heping, mir, sleipnir, eirene, pommier, kelgia, modi4.
Bug fix
Description:
This checkin addresses two bugs.
Firstly, I used the wrong semantics when detecting if $MAKE is defined to
the empty string or undefined.
Secondly, pmake on IRIX does not work well with automake's dependency tracking
if the build is not an in-place build.
Solution:
Fixed semantics in configure.in.
Disabled automatic dependency tracking automatically on IRIX. This is not
the optimal fix, but it should work fine as long as no one is doing HDF5
development on IRIX, and it is much easier than trying to detect the exact
situations in which dependency tracking fails (or than trying to fix dependency
tracking!).
Platforms tested:
heping, modi4 (pmake and gmake)
Description: For HP-UX 11.00, the compiler generates 'floating exception'
when converting 'long double' to most of integer types.
Solution: Define a macro for all other systems except HP-UX 11.00. Hard set
this macro to 'no' in config/hpux11.00 to skip this test for HP-UX 11.00.
Platforms tested: modi4, kelgia, fuss
Description: Start to support software conversion between long double and
all integers.
Solution: No major changes to the algorithm. Changes to configure is to
exclude SGI for long double to integers test because there're too many problems
in their compiler.
Platforms tested: h5committest, modi4, fuss, Teragrid, arabica
Misc. update: RELEASE.txt
Bug fix.
Description:
MIPSpro Compiler 7.4.x starts to support C99 features
but stdint.h has a "guard" in it that will #error if
cc is invoked without asking for C99 standard. But it
does not result with a non-zero compiler exit code, thus
configure thought it is okay to use header file stdint.h.
This causes problems later.
Solution:
Default to use the C99 compiler if available.
Platforms tested:
Tested in Sandia tesla machine which is a new SGI machine.
Code cleanup
Description:
Reduce compiler warnings on SGI IRIX
Platforms tested:
FreeBSD 4.10 (sleipnir)
SGI IRIX6 (Cheryl's machine)
Too minor to require full h5committest
Bug fix (sorta)
Description:
The SGI machines have problems accurately (and consistently) converting
unsigned long values to float and double values, so put in a bit of a hack in
the datatype conversion test code to allow them to get "close enough". This
hack is enabled at configure time by a flag which should only be set on machines
with this problem.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
Bug Fix
Description:
The FFLAGS, CXXFLAGS, and CPPFLAGS variables weren't carrying
user-defined values into the Makefiles.
Solution:
Changed the flags from FOO="..." to FOO="$FOO ..."
Platforms tested:
Copper
Verbena
Arabica
Misc. update:
Update
Description:
Revamped the configuration system. The configurations for the Fortran
and C++ libraries are no longer separate from the "main"
configuration system. This involved removing the "configure*" and
"aclocal.m4" files from the fortran/ and c++/ subdirectories. Also
merging settings in the config/ subdirectories into the main config/
subdirectory.
Fortran header files had to be modified a little for Linux. It was
checking if it was a Linux machine by some #defines, however with the
-std=c99 switch, these defines weren't there. I added a check for
some other ones which should be there whether the -std=c99 switch is
used or not.
Platforms tested:
Verbena (Fortran & C++)
Sol (Fortran & C++)
Copper (Fortran & C++)
Modi4 (Parallel, Fortran, & C++)
Misc. update:
Replaced the individual setting of gcc flags with the combined
gnu-flags file.
Platforms tested:
Tested in modi4 with different combinations of gcc. No needed to
run three platforms tests since this is for irix machines only.
Misc. update:
Bug fix
Description:
Old config assume compiler is the SGI compiler if it is not gcc.
It would apply all SGI cc options to the $CFLAGS. This would cause
problem if a non-cc non-gcc compiler is used.
Solution:
Verify the compiler is indeed the SGI compiler before applying all those
cc specific options to the $CFLAGS.
Platforms tested:
modi4, both parallel serial, gcc or cc, -64 or n32.
Problems:
libtool generated compiler linking commands with dependency_libs
appended to any dynamic library used but only "-lz -lmfhdf ..." were
appended but "-L...", if used, are not appended. So, the "-lz ..."
information is incomplete and solaris cc does not allow even if the
"-L..." information is provide in later part of the command.
Solution:
Included $LDFLAGS in the building of $(LIB).
irix5.x:
irix6.x:
Added "-s" to PROD_CFLAGS so that production code are striped for
smaller sizes.
Platform Tested:
Solaris 2.6
----------------------
This extensive change is the virtual file layer implementation. I've
ported and tested the sec2, family, and core drivers and only ported
the mpio driver (Albert will test it). So if you need MPIO I would
recommend sticking with the previous version for a while.
You will get a few compile warnings about split and stdio drivers not
being implemented and possibly tracing information not inserted in
some of the drivers. You can safely ignore them but I plan to fix
them.
I'm still working on the split driver because I just realized that it
needs a part of the VFL that isn't written yet.
Documentation is being updated also because there were some minor
changes (mostly just name changes). It should be available on my web
site later this week.
./MANIFEST
./src/Makefile.in
./src/hdf5.h
./src/H5Flow.c [REMOVED]
./src/H5Fstdio.c [REMOVED]
./src/H5Fsec2.c [REMOVED]
./src/H5Fsplit.c [REMOVED]
./src/H5Fmpio.c [REMOVED]
./src/H5Ffamily.c [REMOVED]
./src/H5Fcore.c [REMOVED]
./src/H5MFpublic.h [REMOVED]
./src/H5FD.c [NEW]
./src/H5FDcore.c [NEW]
./src/H5FDcore.h [NEW]
./src/H5FDfamily.c [NEW]
./src/H5FDfamily.h [NEW]
./src/H5FDmpio.c [NEW]
./src/H5FDmpio.h [NEW]
./src/H5FDprivate.h [NEW]
./src/H5FDpublic.h [NEW]
./src/H5FDsec2.c [NEW]
./src/H5FDsec2.h [NEW]
Removed/added files for virtual file layer.
./bin/trace
./src/H5.c
Removed unused public datatypes and added new VFL public
datatypes.
Changed an error message.
./config/BlankForm
./config/dec-flags
./config/gnu-flags
./config/hpux10.20
./config/hpux9.03
./config/irix5.x
./config/irix6.x
./config/solaris2.x
./config/unicosmk
Removed the H5F_OPT_SEEK and H5F_LOW_DFLT constants from the
configuration since they're no longer applicable. The default
file driver is always the sec2 driver and it always optimizes
calls to lseek() or lseek64().
./config/depend.in
C preprocessor errors generated during automatic dependency
building are sent to /dev/null to prevent them from appearing
twice in the make output.
./src/H5AC.c
./src/H5B.c
./src/H5D.c
./src/H5F.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5Oattr.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Oshared.c
./src/H5T.c
./src/H5detect.c
./test/ohdr.c
Changed H5F_ADDR_UNDEF to HADDR_UNDEF to be more consistent
with the `haddr_t' datatype which is now a public type.
./src/H5D.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Tconv.c
./test/cmpd_dset.c
./test/dsets.c
./test/overhead.c
./test/tselect.c
./test/tvltypes.c
The H5P_DATASET_XFER constant was changed to H5P_DATA_XFER
because the properties apply to all types of I/O operations,
not just datasets.
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5R.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Smpio.c
./src/H5Spoint.c
./src/H5Sprivate.h
./test/big.c
./test/h5test.c
./test/istore.c
./testpar/t_dset.c
./testpar/t_file.c
./tools/h5debug.c
./tools/h5ls.c
Modified to work with the virtual file layer by calling H5FD_*
functions instead of H5F_low_* functions and by passing file
access and data transfer properties by object ID instead of
pointer.
Changed H5D_transfer_t to H5FD_mpio_xfer_t since the
COLLECTIVE vs. INDEPENDENT transfer mode is specific to the
MPIO file driver.
Moved MPIO-specific stuff into the MPIO driver.
./src/H5B.c
./src/H5D.c
./src/H5Fprivate.h
The H5F_mpio_* private functions were renamed and placed in
the H5FDmpio driver except those which appeared in H5Smpio.c.
./src/H5E.c
./src/H5Epublic.h
Added major error number H5E_VFL for virtual file layer
related errors.
./src/H5F.c
./src/H5Fprivate.h
Changed the logic that controls whether the boot block is
written. Instead of assuming that the first call to write the
boot block is only to allocate space, I've added a function
argument which makes this explicit.
Changed the way files are compared so that a driver-defined
comparison function can be called. Files which belong to
different drivers are always considered different.
Removed H5F_driver_t since file drivers are now identified by
object ID instead of a special non-user-extendible datatype.
Removed all the hard-coded low-level file properties which
have been replaced by the various file drivers.
./src/H5I.c
./src/H5Iprivate.h
Added the H5I_inc_ref() which was removed a few months ago
since we finally have a use for it.
./src/H5Ipublic.h
Added the H5I_VFL object ID type to identify file drivers in
the virtual file layer.
./src/H5MF.c
./src/H5MFprivate.h
Moved all the allocation/deallocation code into the virtual
file layer which allows file drivers to override much of it.
./src/H5P.c
./src/H5Ppublic.h
Moved file driver-specific code into the various file driver
files.
The H5Pcopy() and H5Pclose() functions make calls into the
virtual file driver to manage the memory for driver-specific
file access and data transfer properties.
./src/H5private.h
./src/H5public.h
The `haddr_t' type is now public.
./test/tfile.c
Added a few more comments.
OS cannot handle the 64 bit instruction sets. Let the local system
default takes care of preferred machine code.
Tested on both IRIX and IRIX64 platforms.
----------------------
./INSTALL
Reorganized and added some additional examples.
./MANIFEST
./aclocal.m4 [NEW]
./configure.in
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
./bin/ltconfig [NEW]
./bin/ltmain.sh [NEW]
Added tests to determine how to compile shared libraries and
how to link programs with them before the libraries are
installed. Also how to install and uninstall shared
libraries. The configure step also prints the names of the
config files it's trying to load for easier debugging.
./bin/config.guess
./bin/config.sub
Replaced with a newer version from GNU. The changes we made
to that file to report `irix6.x' and `FreeBSD' without version
numbers have been incorporated into configure.in instead. In
the future, do not change these two files (see the top of
configure.in instead). By the way, this update was required to
get shared libraries working.
./config/linux [REMOVED]
./config/linux-gnulibc1 [NEW]
./config/linux-gnu [NEW]
./config/alpha-dec [REMOVED]
./config/alpha-dec-osf4.0 [REMOVED]
./config/dec-osf4.x [NEW]
./config/irix5.3 [REMOVED]
./config/irix5.x [NEW]
./config/irix64 [REMOVED]
./config/freebsd
Moved config files around to agree with output from the new
config.guess. The linux file was split into gnu (RedHat),
gnulibc1, and gnulibc2 versions. The alpha-dec file was
removed (I think it was unused) and the alpha-dec-osf4.0 was
changed to dec-osf4.x. The irix5.3 file renamed to irix5.x and
the irix64 file was renamed to irix6.x. The freebsd file was
changed to point to linux-gnulibc1. These changes were tested
on:
Linux 2.0
Linux 2.1
FreeBSD 3.2
Irix 5.3
Irix64 6.2
Irix64 6.4
HP/UX 10.20
OSF1 4.0
./config/alphaev56-dec-osf4.x
./config/irix64
Added warnings similar to linux/freebsd about using compilers
with known bugs.
./config/commence.in
./config/conclude.in
./src/Makefile.in
./test/Makefile.in
./testpar/Makefile.in
./tools/Makefile.in
Added definitions for shared libraries. This has been tested
on the following systems:
Linux 2.0
Linux 2.1
FreeBSD 3.2
Irix 5.3
Irix64 6.2
Irix64 6.4
HP/UX 10.20 static only
OSF1 4.0
If you want to disable use of shared libraries (you probably
do for development purposes since it takes a lot longer to
compile and because you have to run dynamically linked
programs in a special way if the library hasn't been
installed) then add `--disable-shared' to the configure
command line. This is all documented in the INSTALL file.
./bin/release
Temprarily commented out the MANIFEST checking when running
under svf since svf is about to be replaced by a newer
version. This change only affects error checking during the
release process.
./Makefile.dist
./Makefile.in
Added `make check' which does the same thing as `make _test'
since the former is endorsed by the GNU coding style and
people are used to it. The old `make _test' still works too
(and so does `make test' if you use GNU make).
----------------------
./INSTALL
Added instructions for using other compilers on Irix platforms.
./configure.in
./configure [REGENERATED]
./src/H5config.h.in [REGENERATED]
./src/H5private.h
The presence of <sys/sysinfo.h> and <sys/proc.h> is only
checked on DEC/Alpha because there are too many problems
including these headers on other systems.
./config/irix6.x
./config/irix64
Removed `-ansi' and added a comment. Converting an ASCII UTC date
and time to a time_t value is not defined by ANSI or Posix but
is available if we don't use `-ansi'. This allows the dataset
modification time messages to be read properly by
H5Gget_objinfo().
./src/H5Tconv.c
Fixed a compiler warning.
./test/dtypes.c
./test/hyperslab.c
./test/istore.c
Fixed compiler warnings in printf() statements.
----------------------
./INSTALL
Added warnings and a disclaimer about GNU, DEC, Irix64, and NT
compilers that generate incorrect machine code.
./configure.in
./src/H5private.h
Detects and includes <sys/param.h> which is needed on FreeBSD
before <sys/proc.h> even though we only really using anything
from <sys/proc.h> on the DEC Alpha.
./config/irix64
Turned off warnings for duplicate definitions from the linker
because -lnsl on irix has the same stuff in it as -lc.
./config/irix6.x
Split up the CC and CFLAGS settings like with irix64 so that
compilers besides `-n32' can be used.
./bin/snapshot
The snapshots are tagged with names like hdf5-1_1_52 which is
similar to the way the releases are tagged (cvs doesn't allow
dots in tags).
./test/dtypes.c
./src/H5private.h
./src/H5Tconv.c
Fixed some alignment violations on the DEC when using high
optimization levels. The DEC incorrectly optimizes certain
memcpy() and memmove() calls when the source argument is not
word aligned if the call looks like it's copying an atomic
data type.
./test/hyperslab
Worked around code generation bugs in the Irix64 Mongoose 7.00
compiler by casting some `unsigned long' values to `unsigned'
in an expression.
./src/H5Ocomp.c
Fixed a place where %d was used to print a size_t.
Bug fix
Problem:
Currently, when --enable_preduction --disable-debug modes are
selected during configure, the optimization level scheduled is
-O (-O2). Unfortunately, this level of optimization relaxes
too much the alignments necesary for the building, and testing
of the hdf5 library.
Solution:
Lower the level of optimiztion to -O1. This still maintains
required alignments.
Platform tested:
Irix6.5(fuga)
