----------------------
./MANIFEST
Added new files from testpar directory.
./config/commence.in
Removed -D_POSIX_SOURCE because hdf5 uses a couple of
non-Posix things when certain debugging flags are switched on.
to cleanup temporary files used by that particular module.
Added a cleanup() routine to testhdf5 that calls all the other
cleanup_xxx() routines to do cleanup.
Cleanup action can be disabled either by -c (no cleanup) option
or HDF5_NOCLEANUP environment variable.
----------------------
./MANIFEST
./configure.in
./examples/Makefile.in [NEW]
Added a Makefile.in for the examples directory. To generate
the Makefile run configure again.
changed to use system supported MPI library. The path to it is
changed slightly too. Updated all the paths variables.
Also updated to reflect the new program source files structure.
Platform tested: Intel Red
big testphdf5.c file. Broke it into modules of related routines.
testphdf5.c -- main routine and global variables initialization plus
some ulitility routines.
t_file.c -- tests of file operations using parallel I/O.
t_dset.c -- tests of datasets operations.
H5D.c:
If a dataset, using chunk storage, is created/opened and if its data
file is accessed via MPIO, allocate file space for the chunks immediately.
Also do the allocation when a dataset is extended. Must do it at these
points because H5Dcreate, H5Dopen and H5Dextend are collective calls
in MPIO access mode and are the only chance to get all MPI processes
to synchonize their meta-data.
Also disallow opening/creating datasets with compression option if
MPIO is used. Not easy to support concurrent compression writes in
an efficient way. But read-only access should be fine. Will implement
that later.
H5F.c:
Disable file cache if MPIO is used by forcing cache size and number
of elements to zero.
H5Fistore.c:
Added two new routines, H5F_istore_get_addr and H5F_istore_allocate.
H5F_istore_allocate supports allocating file space for the chunks
of a dataset. H5F_istore_get_addr returns the file space address
of a chunk that has been allocated.
Changed H5F_istore_read and H5F_istore_write to bypass the chunk
cache read/write mechanism. Instead, they do read/write directly
with the file for only the elements requested.
(Some debug statements, enclosed by #ifdef AKC, are still in the
file. I am still working on some code. They would be cleaned
out soon.)
----------------------
./src/H5Tbit.c
./src/H5Tpkg.h
Fixed a bug in H5T_bit_copy(). Added H5T_bit_get_d() and
H5T_bit_set_d() which treat portions of a bit vector as an
unsigned integer. Added H5T_bit_inc() that increments part of
a bit vector and returns an indication of overflow.
./src/H5Tconv.c
./src/H5Tpkg.h
./test/dtypes.c
Added a slow general floating point conversion which works so
far on Intel, MIPS, and DEC but the test is turned off because
a cast from double to float will cause a SIGFPE on some
systems if an overflow occurs.
Added fast hardware conversions between native floating point
types. This function is also subject to the SIGFPE problem.
./src/H5detect.c
Removed the exponent bias adjustment when the significand
isn't normalized. This is now handled in the conversion
functions instead.
./src/H5T.c
Register new conversion functions.
Plugged a memory leak in the library termination code.
./RELEASE
Added a list of major changes since the first alpha.
./src/H5.c
./src/H5private.h
./src/H5A.c
./src/H5AC.c
./src/H5D.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5G.c
./src/H5Gprivate.h
./src/H5HG.c
./src/H5O.c
./src/H5T.c
./src/H5Tbit.c
./src/H5Tconv.c
Fixed various compiler warnings on Irix64.
./src/H5MM.c
Added PABLO_MASK to this file.
./test/chunk.c
Removed a warning about memcpy().
./INSTALL_MAINT
./README
./RELEASE
Partially updated for second alpha, but haven't updated
version numbers yet.
./src/H5.c
./src/H5A.c
./src/H5AC.c
./src/H5B.c
./src/H5D.c
./src/H5F.c
./src/H5Fcore.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Fmpio.c
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5G.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5I.c
./src/H5MM.c
./src/H5MMprivate.h
./src/H5O.c
./src/H5Oattr.c
./src/H5Ocomp.c
./src/H5Ocont.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Olayout.c
./src/H5Oname.c
./src/H5Osdspace.c
./src/H5Oshared.c
./src/H5Ostab.c
./src/H5P.c
./src/H5S.c
./src/H5T.c
./src/H5Tconv.c
./src/H5detect.c
./test/hyperslab.c
./test/istore.c
Changed memory allocation functions so they fail instead of
dumping core. The `x' was removed from the name to remind us
of that: H5MM_xmalloc() -> H5MM_malloc(), etc.
H5MM_calloc() takes one argument like H5MM_malloc() instead of
two like calloc() because we almost always called it with `1'
for one of the arguments anyway. The only difference between
the two functions is that H5MM_calloc() returns memory which
is initialized to zero.
./src/H5Gent.c
./src/H5Gprivate.h
Removed H5G_ent_calloc() since it wasn't used.
./src/H5Fistore.c
Fixed a bug found by Albert. Thanks, Albert! This fix
combined with the changes to memory allocation prevent the
library from failing an assertion if the application uses an
unreasonable size for chunks (like Alberts 10000x10000x4).
./src/H5MF.c
./src/H5MFprivate.h
Changed H5MF_free() to H5MF_xfree() since calling it with an
undefined address is allowed.
----------------------
./html/Datatypes.html
./src/H5T.c
./src/H5Tconv.c
./src/H5Tpkg.h
./src/H5Tpublic.h
./src/H5detect.c
./test/cmpd_dset.c
./test/dsets.c
./test/dtypes.c
./test/external.c
./test/tattr.c
./test/th5s.c
Organized the predefined data types.
./html/H5.intro.html
NO MODIFICATIONS! This file is a mess (obviously generated
from something else) so I skipped updating it!
./src/h5ls.c
./test/dsets.c
./test/tattr.c
./test/th5s.c
Added extra argument to H5Sget_dims() calls per Albert's last
update.
Added maxdims agrument to H5Sget_dims. Also removed the check
of dims since H5Sget_dims can still return ndims even if both
*dims and *maxdims are NULLs.
----------------------
./src/H5T.c
Fixed a potential division by zero error.
./src/H5Fistore.c
Fixed an assertion failure that I accidently introduced with
the previous changes.
----------------------
./html/tracing.html NEW
This entire update is to make it possible for the library to
print the name, arguments, and return value of every API call
without requiring any extra work from developers or app
programmers. This file describes how this all works.
./configure.in
Added the `--enable-tracing' switch. If you use it then the
library will include code to print API function names,
argument names and values, and function return values.
However, you must then turn on the tracing by setting the
HDF5_TRACE environment variable to a file descriptor number.
The default is `--disable-tracing' since enabling it causes a
slight increase in library size and a slowdown resulting from
an extra function call for each API function call (I couldn't
even measure the slowdown :-)
./bin/trace NEW
A perl script that synchronizes the H5TRACE() macro calls in
the *.c files with the function return type and formal
argument names and types. If you use GNU make and gcc then
this will be done automatically, otherwise just invoke this
script with the names of one or more .c files. You could do
it by hand to, but encoding argument types is a little tricky
at first.
./config/commence.in
Added the $(TRACE) macro, which defaults to the no-op.
Added -D_POSIX_SOURCE to the compiler command line.
./src/Makefile.in
Override the default for $(TRACE).
./config/depend.in
Automatically calls $(TRACE) to synchronize the H5TRACE()
macros in any source file that changed. As with makefile
dependencies, one way to force synchronization of all files is
to remove the `.depend' file.
./MANIFEST
Added new files.
./src/H5Eprivate.h
Modified HRETURN_ERROR() and HRETURN() for tracing.
./src/H5.c
./src/H5private.h
This is where the real tracing work really happens, in
H5_trace().
./src/H5A.c
./src/H5D.c
./src/H5G.c
./src/H5P.c
./src/H5S.c
./src/H5Z.c
Added H5TRACE() calls to all API functions. You don't really
need these changes if you don't want to merge your stuff
because they can be generated automatically by going to the
hdf5/src directory and saying ../bin/trace *.c
./src/H5T.c
Added H5TRACE() calls. Other stuff below.
./src/H5E.c
./src/H5Epublic.h
Added H5TRACE() calls. Created a type H5E_auto_t for the
`func' argument of H5Eset_auto() and H5Eget_auto() to make
those arguments easier to parse for tracing. It should also
make it clearer for users that don't know how to read
complicated ANSI data types.
./src/H5F.c
Added H5TRACE() calls. Changed a couple `uintn' argument
types in API functions to `unsigned int' since `uintn' part of
the API. Changed a few "can't" and "cant" error messages to
"unable to".
./src/H5Ipublic.h
Removed H5_DIRECTORY from the H5I_group_t enum. It wasn't
used anywhere.
./src/H5Tconv.c
Removed an unused label.
./src/H5Fistore.c
./src/H5Oattr.c
./src/H5Odtype.c
./src/H5T.c
./test/dsets.c
./test/dtypes.c
Fixed a warning about a variable possibly used before it's
initialized. Added __unused__ to turn off some unused
argument warnings that pop up when debugging is turned off and
optimizations are turned on.
----------------------
./src/H5Tbit.c
./MANIFEST
./test/Makefile.in
./test/bittests.c NEW
Finished the bit vector operations and added test cases.
./src/H5Tconv.c
./test/dtypes.c
Finished integer->integer general conversion and added test
cases. Overflows and underflows are handled by substituting
the closest possible value. Examples:
(unsigned)0xffff -> (unsigned) 0xff
( signed)0xffff -> (unsigned)0x0000
(unsigned)0xffff -> ( signed)0x7fff
( signed)0x7fff -> ( signed) 0x7f
( signed)0xbfff -> ( signed) 0xbf
( signed)0x8000 -> ( signed) 0x80
./src/H5private.h
Added definitions for MIN and MAX that take 3 or 4 arguments:
MIN3(), MIN4(), MAX3(), MAX4(). Also added MIN2() and MAX2()
as aliases for MIN() and MAX().
./test/tattr.c
Removed some redundant `&' operators.
./configure.in
./src/H5config.h.in [regenerated]
./src/H5.c
Fixed warnings on DEC where long double is the same as
double.
----------------------
THIS CHECKIN IS FOR QUINCEY -- NOT EVERYTHING WORKS (but it compiles)
MOST OF THE CHANGES ARE FOR BETTER TYPE CONVERSION IN THE NEXT ALPHA
./MANIFEST
./src/H5Tbit.c NEW
./src/Makefile.in
Bit vector operations (not done yet)
./configure.in
Added -lm to the library list, needed by bit-vector operations
and conversion functions.
Removed vestiges of PARALLEL_SRC no longer used by the
makefiles. Albert came up with a better way (that actually
works :-)
./src/H5D.c
No code changes. Split a couple of long lines, refilled a
couple multi-line comments.
./src/H5T.c
./src/H5Tpublic.h
Fixed a bug reported by Jim Reus regarding conversion of
compound data types whose members require conversions which
are satisfied by as-yet unregistered soft conversion
functions.
Added H5T_IEEE architecture, but the funny-looking integer
types will be changed to H5T_BE_ and H5T_LE_ architectures
with the type names changed to match the H5T_NATIVE_ integers.
Added an H5Tconvert() but it hasn't been documented or tested
yet.
./src/H5Tconv.c
./src/H5Tpkg.h
Registered conversion functions integer->integer (a general
case) and integer->float (for a specific case). The
integer->integer conversion depends on the bitvector
operations which aren't finished yet and the int->float
conversion hasn't been retested since it was borrowed from
AIO. Don't look at them yet, they're ugly :-)
./src/H5detect.c
Fixed a typo which caused the msb_pad field of an atomic type
to not be initialized.
./test/dtypes.c
Added a test for number conversions but it's commented out
until the conversion stuff is truly working.
which can be set by passing to H5Fcreate and H5Fopen
a string of characters (one character for each flag to turn on)
as the value of MPI_Info with key H5F_MPIO_DEBUG_KEY.
./src/H5F.c
./src/HFcore.c
./src/H5Ffamily.c
./src/H5Flow.c
./src/H5Fmpio.c
./src/H5Fprivate.h
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5MF.c
./src/H5P.c
./src/H5Ppublic.h
Added H5Pset_alignment() so that it is now possible to align
file allocation requests on application-specified
boundaries. Any request >= the specified threshold will begin
on an address which is a multiple of the specified
alignment. Defaults are one for threshold and alignment. The
alignment is done on relative addresses, so the size of the
user block can affect the location of the data in the file.
./src/H5D.c
./src/dsets.c
Added a test for, and fixed the data space caching bug in
datasets. Extending a dataset through one handle will cause
all other handles to the same dataset to get the new dataset
size.
./src/H5S.c
./src/H5Sprivate.h
Removed an unused argument from H5S_read() which duplicated
information from the other argument.
./config/linux
Made `--enable-parallel' the default on my system. It used to
be that way before but then I accidently turned it off and
forgot about it.
./src/H5Fmpio.c
Qualified some function arguments with __unused__. Changed a
couple places where NULL was returned on error for herr_t
functions.
./src/H5P.c
Removed unused autos from H5Pset_mpi().
H5Fmpio.c was not part of the serial distribution. The auto-
dependence configuration runs for serail distribution only, thus
H5Fmpio.c has no dependence. It ended up not recompiled even
when header files it depends on are changed.
Solution:
Changed configuration to always include this file.
Enclosed the whole code in a big #ifdef HAVE_PARALLEL macro
so that they would be compiled for parallel distribution only.
Platform tested:
O2K
Parallel File open has a racing condition if file already exists.
Some processes see file exists, then truncate it (some system
by remove and create). Some processes does not see the file
exist, then attempt to just create it. Inconsistency.
Solution:
Instead of opening the file with COMM_SELF (which results in
racing condition in routine that calls it), open it with the
communicator in access_parms. (This assumes this access call
must be called collectively.)
Platform tested:
O2K.
mode to use for data transfer. Currently, INDEPENDENT transfer
is the default mode. Collective is supported for files opened
via the MPIO driver. Specific changes to individual files follow.
H5D.c:
Replaced previous kludge of passing xfer_mode via the file access_mode
with xfer_mode function parameter all the way to the low level I/O
drivers.
H5Dprivate.h:
The H5D_xfer_t.xfer_mode is now used in serial and parallel modes.
Removed the "HAVE_PARALLEL" macro condition to support it.
H5Dpublic.h:
Added H5D_XFER_DFLT as a new value for H5D_transfer_t and removed
the HAVE_PARALLEL condition macro to make the H5D_transfer_t
definition available to both serial and parallel version.
H5B.c:
H5F.c:
H5Farray.c:
H5Fcore.c:
H5Ffamily.c:
H5Fistore.c:
H5Flow.c:
H5Fmpio.c:
H5Fsec2.c:
H5Fsplit.c:
H5Fstdio.c:
H5Gnode.c:
H5HG.c:
H5HL.c:
H5O.c:
H5Sprivate.h:
H5Ssimp.c:
debug.c:
Added xfer_mode to the argument lists of the following routines to
specify data transfer mode. Then updated calls to these routines
with appropriate transfer mode. Whenever cases that collective
mode is inappropriate (e.g. object header read-write, flush),
H5D_XFER_DFLT is used. The H5D_XFER_DFLT mode is currently
acting the same as the independent mode.
H5F_block_read
H5F_block_write
H5F_arr_read
H5F_arr_write
H5F_core_read
H5F_core_write
H5F_fam_read
H5F_fam_write
H5F_mpio_read
H5F_mpio_write
H5F_low_write
H5F_low_read
H5F_sec2_read
H5F_sec2_write
H5F_split_read
H5F_split_write
H5F_stdio_read
H5F_stdio_write
H5S_simp_fgath
H5S_simp_fscat
H5S_simp_read
H5S_simp_write
H5Fprivate.h:
H5P.c:
Removed access_mode field from struct H5F_access_t type.
The access-mode was the initial design to control independent
or collective I/O mode. It is now supported by the data
transfer mode.
----------------------
./src/H5A.c
Named data types can have attributes.
Fixed bugs where the API functions didn't check the return
values of their internal counterparts and thus the automatic
error reporting didn't work.
Fixed some places where the error stack wasn't cleared after a
function returned failure.
Data types returned by H5Aget_type() are always read-only.
If the `attr_num' argument of H5Aiterate() is null then it
acts like H5Giterate() instead of failing -- it begins
processing attributes with the first one.
./src/H5D.c
We check for allocation overruns when scalar datasets are
stored in external files.
./src/H5O.c
H5O_modify() will fail if the message is >=16kB.
./src/H5Oattr.c
Split some long lines
./src/H5T.c
./src/H5Tprivate.h
Added H5T_entof() to support attributes on named types.
./src/h5ls.c
Prints the names of attributes and their sizes.
./test/cmpd_dset.c
./test/dsets.c
./test/dtypes.c
./test/extend.c
./test/external.c
./test/gheap.c
./test/istore.c
./test/links.c
./test/shtype.c
If the environment variable HDF5_NOCLEANUP is defined then the
temporary files are not removed. The testhdf5 program still
has the bug that it removes *.h5, clobbering test files from
other programs... oh well.
./test/dtypes.c
Added attribute tests.
----------------------
./html/Datatypes.html
Updated documentation for named data types and type sharing.
./src/H5Farray.c
Split a couple long lines.
./src/H5T.c
The H5T_copy() demotes immutable types to read-only types so
they're cleaned up properly and memory is not leaked.
./test/dtypes.c
Modified to be more consistent with other tests. Removed all
internal header files, constants, types, and functions calls
since the API is now complete enough to test
everything. Temporary files are now removed.
Added tests for transient and named types and their use in
datasets.
./test/shtype.c
Commented out all the tests since they no longer apply.
./test/chunk.c
Removed a couple int->double coercion warnings.
./src/H5D.c
./src/H5Tconv.c
./src/H5detect.c
Updated to work with new internal H5T functions. Fixed some
data type memory leaks during error recovery.
./src/H5Dprivate.h
Added H5D_typeof() similar to H5D_entof() that returns a
pointer directly to the dataset's type. This is used by
H5Tcopy() when invoked on a dataset (see below).
./src/H5Epublic.h
Fixed typos in H5E_BEGIN_TRY and H5E_END_TRY macros.
./src/H5F.c
Closing a file with objects still open reports the file name
in the warning message. Removed unnecessary invalidation of
shared data types.
./src/H5Gent.c
./src/H5Gpkg.h
./src/H5Gprivate.h
Added `const' to some arguments.
./src/H5O.c
./src/H5Oprivate.h
./src/H5Oshared.c
An object header message can now be a pointer to a message in
some other object header. The pointer is to the first message
of the appropriate type in the other object header and hard
link counts are maintained automatically to prevent dangling
pointers. The old global heap method of message sharing is
also supported although nothing actually uses it.
./src/H5Oattr.c
./src/H5Ocomp.c
./src/H5Ocont.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Olayout.c
./src/H5Oname.c
./src/H5Osdspace.c
./src/H5Oshare.c
./src/H5Ostab.c
Changed the data type for the shared message info struct to
H5O_shared_t and added an extra initializer to the class
methods struct for the set_share method.
./src/H5Odtype.c
Added the ability to share data type messages by pointing to
other object headers.
./src/H5T.c
./src/H5Tpkg.h
./src/H5Tprivate.h
./src/H5Tpublic.h
Added named data types and changed the functionality of some
API functions as follows:
* The term `read-only' means that a type handle cannot be
modified with functions like H5Tset_*() or H5Tinsert().
* The term `immutable' means the same as `read-only' with the
added restriction that H5Tclose() cannot be called for the
type. A transient type is made immutable by calling
H5Tlock().
* Handles to named types are always read-only.
* Handles to predefined types are immutable.
* A transient type can be converted to a named type by calling
H5Tcommit(). This function will fail if the type is already
named or is immutable.
* The API function H5Tcommitted() returns an indication of
whether a data type has been commited (or is named). If
H5Tcommitted() returns TRUE for a data type obtained by
calling H5Dget_type() on a dataset, then the dataset is
using a shared data type.
* H5Topen() returns a handle to a named type.
* H5Tcopy() always returns a handle to a modifiable transient
type even when invoked on a named type. Also, when invoked
on a dataset it returns a modifiable transient type which is
a copy of the dataset's data type.
* Using a named data type in the call to H5Dcreate() causes
the dataset object header to point to the named data type,
but using a transient type causes the type to be copied into
the dataset's object header.
* The data type returned from H5Dget_type() is a named data
type or a read-only transient data type depending on whether
the dataset points to a named data type. The old behavior,
to return a modifiable transient type, is accomplished by
overloading H5Tcopy() to operate on datasets (see above).
* H5Tshare(), H5Tunshare(), and H5Tis_shared() have been
removed from the API.
The following features were *not* implemented because they
need more discussion:
* A named data type can be opened by applying H5Topen() to a
dataset in which case the data type is the data type of the
dataset (or the data type to which the dataset points if the
dataset has a shared data type).
* A named data type can have attributes like groups or
datasets.
* The members of a compound data type can point to named data
types.
./src/h5ls.c
Reports `Data type' for named data type objects in the file.
----------------------
./html/Datasets.html
Fixed a couple of typos.
./src/H5.c
Added the `Z' modifier to HDfprintf() for `size_t' sizes. Use
it like this:
HDfprintf(stderr,"size is %Zd\n", (size_t)x);
./src/H5AC.c
./src/H5F.c
./src/H5Fprivate.h
The maximum number of meta data objects that can be cached can
be set from the application (but the library might not honor
it every time; it's a hint).
./src/H5D.c
Changed a warning message so it's not so alarming.
./src/H5Fistore.c
Chunks can be cached.
./src/H5O.c
./src/H5Oprivate.h
Added H5O_copy() and H5O_free() to copy and free messages.
./src/H5P.c
./src/H5Ppublic.h
Added H5Pset_cache() and H5Pget_cache() and changed lots of
"template" to "property list".
./src/H5Z.c
./src/H5Zpublic.h
Miscellaneous little things to clean up. Mostly just removed
H5Z_MAXVAL and added H5Z_USERDEF_MIN and H5Z_USERDEF_MAX.
./MANIFEST
./test/Makefile.in
./test/chunk.c [NEW]
Added a performance test for chunk caching. It looks at the
amount of I/O instead of timing because timing is partly
dependent on the chunk size and I wanted a measurement that
was a function of only the cache size. Run `chunk' with no
arguments and then say `gnuplot x-gnuplot' to see the plots
(press return between plots). Postscript files are created for
each plot.
./test/big.c
./test/cmpd_dset.c
./test/extend.c
./test/external.c
./test/gheap.c
Added H5F_ACC_DEBUG so we can see cache performance
statistics.
Change in Documentation for the INSTALL process on the ASCI Red
Problem:
As part of the parallel HDF5 install process, upon compiling the
library and sequential testers, it is necessary to copy the testers
onto local disk prior to testing. This is almost done by steps 9-11)
...
9) cp -r ../hdf5 ../scratch
10) cd /scratch/hdf5/test
11) make test >&! test.out
After the copying to /scratch, when the "make test" is run,
dependencies are checked and it is determined that the executables
need to be rebuilt. Unfortunately, the MPI-IO libraries in
sasn100:/usr/community/mpi-io are not accessible on ASCI Red, so
that step #11 above gives an error.
Solution:
Albert Cheng and Robb Matzke suggested to add the "-p" option in
step #9, if available, to maintain the original modifications.
Those install steps were changed to:
...
9) cp -rp ../hdf5 ../scratch
10) cd /scratch/hdf5/test
11) make test >&! test.out
Everything works fine!
Platform tested:
janus - ASCI Red
sasn100 - TFLOPS Application server
