API cleanup
Description:
Tweaked some parts of the generic properties API to reduce warnings.
Platforms tested:
FreeBSD 4.1.1 (hawkwind) & SGI O2K (modi4)
Code updates
Description:
Finished fleshing out code for generic properties, although they are not
used anywhere in the library yet.
Platforms tested:
FreeBSD 4.1.1 (hawkwind)
Maintainance & performance enhancements
Description:
Re-arranged header files to protect private symbols better.
Changed optimized regular hyperslab I/O to compute the offsets more
efficiently from previous method of using matrix operations.
Added sequential I/O operations at a more abstract level (at the same level
as H5F_arr_read/write), to support the optimized hyperslab I/O.
Platforms tested:
Solaris 2.6 (baldric) & FreeBSD 4.1.1 (hawkwind)
Implemented new feature
Description:
Added data sieve buffering code to raw I/O data path. This is enabled for
all the VFL drivers except the mpio & core drivers. Also added two new
API functions to control the sieve buffer size: H5Pset_sieve_buf_size() and
H5Pget_sieve_buf_size().
Platforms tested:
Solaris 2.6 (i.e. baldric)
Bug Fix
Description:
The prototype for the H5Pregister function has a variable named
`class'. This is a reserved word in C++ and causes the C++
compiler to freak.
Solution:
This variable's name was changed to cls_id in the .c file, so I
changed it in the header file to cls_id to match.
Platforms tested:
Linux
** src/H5Tconv.c
** src/H5Tpkg.h
** src/H5Tpublic.h
The H5T_conv_struct_opt() function had a design flaw -- it
didn't keep information about the stride to use to step
through the temporary/background-value buffer and thus nested
invocations would clobber each other's temp buffers. This was
fixed by splitting the `stride' argument into `buf_stride' and
`bkg_stride' arguments for all the conversion functions. THIS
IS AN API CHANGE, but users will get a compiler warning when
they pass their conversion function pointer to H5Tregister().
** src/H5T.c
** src/H5Tprivate.h
Added a bkg_stride argument to the H5T_convert() definition in
order to fix a bug related to the optimized compound datatype
conversion function.
** src/H5T.c
** src/H5A.c
** src/H5D.c
** src/H5Ofill.c
** src/H5P.c
Added bkg_stride=0 argument to the H5T_convert() calls.
** test/dtypes.c
Added a test for the H5T_conv_struct_opt() bug fixed above.
** src/H5FL.c
The H5FL_term() function should return non-zero even when it
couldn't free all the free lists do to their being used by
some other package. When that other package terminates it
will return non-zero, causing H5FL_term() to be called
again. This fixes some of the `infinite loop closing library'
messages.
** tools/pdb2hdf
Uses print_version() instead of doing that itself.
** src/H5Ppublic.h
Renamed H5Pget_gc_reference() declaration to make it match the
definition.
** src/H5FDlog.c
Added API tracing macros.
Removed `const' qualifier from a `char*' member of a struct
which was allocated on the heap.
** src/H5TB.c
Added curly braces to a couple deeply-nested `if' statements
to make them clearer and to shut up the increadibly stupid and
just plain incorrect gcc warning about ambiguous `else'.
** test/titerate.c
Removed incomplete initialization in favor of memset() for one
auto variable to stop compiler warnings.
** tools/Depencencies
Regenerated to remove references to h5dumputil.c
----------------------
This extensive change is the virtual file layer implementation. I've
ported and tested the sec2, family, and core drivers and only ported
the mpio driver (Albert will test it). So if you need MPIO I would
recommend sticking with the previous version for a while.
You will get a few compile warnings about split and stdio drivers not
being implemented and possibly tracing information not inserted in
some of the drivers. You can safely ignore them but I plan to fix
them.
I'm still working on the split driver because I just realized that it
needs a part of the VFL that isn't written yet.
Documentation is being updated also because there were some minor
changes (mostly just name changes). It should be available on my web
site later this week.
./MANIFEST
./src/Makefile.in
./src/hdf5.h
./src/H5Flow.c [REMOVED]
./src/H5Fstdio.c [REMOVED]
./src/H5Fsec2.c [REMOVED]
./src/H5Fsplit.c [REMOVED]
./src/H5Fmpio.c [REMOVED]
./src/H5Ffamily.c [REMOVED]
./src/H5Fcore.c [REMOVED]
./src/H5MFpublic.h [REMOVED]
./src/H5FD.c [NEW]
./src/H5FDcore.c [NEW]
./src/H5FDcore.h [NEW]
./src/H5FDfamily.c [NEW]
./src/H5FDfamily.h [NEW]
./src/H5FDmpio.c [NEW]
./src/H5FDmpio.h [NEW]
./src/H5FDprivate.h [NEW]
./src/H5FDpublic.h [NEW]
./src/H5FDsec2.c [NEW]
./src/H5FDsec2.h [NEW]
Removed/added files for virtual file layer.
./bin/trace
./src/H5.c
Removed unused public datatypes and added new VFL public
datatypes.
Changed an error message.
./config/BlankForm
./config/dec-flags
./config/gnu-flags
./config/hpux10.20
./config/hpux9.03
./config/irix5.x
./config/irix6.x
./config/solaris2.x
./config/unicosmk
Removed the H5F_OPT_SEEK and H5F_LOW_DFLT constants from the
configuration since they're no longer applicable. The default
file driver is always the sec2 driver and it always optimizes
calls to lseek() or lseek64().
./config/depend.in
C preprocessor errors generated during automatic dependency
building are sent to /dev/null to prevent them from appearing
twice in the make output.
./src/H5AC.c
./src/H5B.c
./src/H5D.c
./src/H5F.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5Oattr.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Oshared.c
./src/H5T.c
./src/H5detect.c
./test/ohdr.c
Changed H5F_ADDR_UNDEF to HADDR_UNDEF to be more consistent
with the `haddr_t' datatype which is now a public type.
./src/H5D.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Tconv.c
./test/cmpd_dset.c
./test/dsets.c
./test/overhead.c
./test/tselect.c
./test/tvltypes.c
The H5P_DATASET_XFER constant was changed to H5P_DATA_XFER
because the properties apply to all types of I/O operations,
not just datasets.
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5R.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Smpio.c
./src/H5Spoint.c
./src/H5Sprivate.h
./test/big.c
./test/h5test.c
./test/istore.c
./testpar/t_dset.c
./testpar/t_file.c
./tools/h5debug.c
./tools/h5ls.c
Modified to work with the virtual file layer by calling H5FD_*
functions instead of H5F_low_* functions and by passing file
access and data transfer properties by object ID instead of
pointer.
Changed H5D_transfer_t to H5FD_mpio_xfer_t since the
COLLECTIVE vs. INDEPENDENT transfer mode is specific to the
MPIO file driver.
Moved MPIO-specific stuff into the MPIO driver.
./src/H5B.c
./src/H5D.c
./src/H5Fprivate.h
The H5F_mpio_* private functions were renamed and placed in
the H5FDmpio driver except those which appeared in H5Smpio.c.
./src/H5E.c
./src/H5Epublic.h
Added major error number H5E_VFL for virtual file layer
related errors.
./src/H5F.c
./src/H5Fprivate.h
Changed the logic that controls whether the boot block is
written. Instead of assuming that the first call to write the
boot block is only to allocate space, I've added a function
argument which makes this explicit.
Changed the way files are compared so that a driver-defined
comparison function can be called. Files which belong to
different drivers are always considered different.
Removed H5F_driver_t since file drivers are now identified by
object ID instead of a special non-user-extendible datatype.
Removed all the hard-coded low-level file properties which
have been replaced by the various file drivers.
./src/H5I.c
./src/H5Iprivate.h
Added the H5I_inc_ref() which was removed a few months ago
since we finally have a use for it.
./src/H5Ipublic.h
Added the H5I_VFL object ID type to identify file drivers in
the virtual file layer.
./src/H5MF.c
./src/H5MFprivate.h
Moved all the allocation/deallocation code into the virtual
file layer which allows file drivers to override much of it.
./src/H5P.c
./src/H5Ppublic.h
Moved file driver-specific code into the various file driver
files.
The H5Pcopy() and H5Pclose() functions make calls into the
virtual file driver to manage the memory for driver-specific
file access and data transfer properties.
./src/H5private.h
./src/H5public.h
The `haddr_t' type is now public.
./test/tfile.c
Added a few more comments.
final parameter (so that VL datatypes have a way to pass in the custom memory
allocation routines).
Fixed a conversion bug when VL fields were part of a compound datatype that
was causing the no-op conversion routine to be used instead of the vlen routine.
Added the H5Pset_vlen_mem_manager and H5Pget_vlen_mem_manager routines to
allow users to provide their own custom memory allocation routines for VL data
read in from the file and reclaimed with H5Dvlen_reclaim.
Finished coding on H5Dvlen_reclaim so it works now.
----------------------
./src/H5.c
Fixed more dependency problems in H5_term_library(). There was
a bug in the previous version that could cause the wrong EOF
marker to be written to the boot block under certain
circumstances. Hopefully this fixes it although I don't ready
access to a test case (Mark Miller will test it).
./src/H5F.c
./src/H5Fprivate.h
Added an H5F_close_all() that is similar to
H5F_term_interface() but which doesn't close the
interface. Files that don't have open object headers are
closed, others are delayed until all object headers close. All
files are flushed.
./src/H5ACprivate.h
./src/H5Bprivate.h
./src/H5Dprivate.h
./src/H5Eprivate.h
./src/H5Fprivate.h
./src/H5Gpkg.h
./src/H5Gprivate.h
./src/H5Gpublic.h
./src/H5HGprivate.h
./src/H5HLprivate.h
./src/H5Iprivate.h
./src/H5MFprivate.h
./src/H5MMprivate.h
./src/H5Oprivate.h
./src/H5Pprivate.h
./src/H5Ppublic.h
./src/H5RAprivate.h
./src/H5Sprivate.h
./src/H5Spublic.h
./src/H5Tpkg.h
./src/H5Tprivate.h
./src/H5Tpublic.h
./src/H5Vprivate.h
./src/H5Zprivate.h
./src/H5private.h
./src/H5public.h
Reindented after __DLL__ was added.
----------------------
./tools/h5tools.c
Strings are not converted to null-padding before being
printed; they are printed with whatever byte values appear in
the file.
./tools/h5ls.c
Now able to display attribute data type and data.
Added a `-f' or `--full' switch which causes the full name of
each object to be displayed instead of just the base name.
Added a `-r' or `--recursive' switch that recursively prints
the contents of groups, avoiding cycles.
More bulletproofing for non-printable characters in things
like object names, attribute names, and comment strings. We
don't want listing a file to send termal escape sequences
because it's sometimes possible to execute commands that way.
Since h5ls doesn't usually use quotes around object names we
must sometimes escape space characters.
External files are listed in a table to make the output less
confusing.
./tools/h5tools.c
./tools/h5tools.h
Changed h5dump() to h5dump_dset() and added h5dump_mem(). Also
make h5dump_fixtype() public.
./test/dtypes.c
Wrote some data to an attribute to test h5ls attribute
printing.
./src/H5ACprivate.h
./src/H5Apublic.h
./src/H5Dprivate.h
./src/H5Dpublic.h
./src/H5Epublic.h
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gprivate.h
./src/H5HLprivate.h
./src/H5Oprivate.h
./src/H5Ppublic.h
./src/H5RApublic.h
./src/H5Rpublic.h
./src/H5Spublic.h
./src/H5Tpkg.h
./src/H5Tpublic.h
./src/H5Vprivate.h
./src/H5Zpublic.h
./src/H5private.h
./src/H5public.h
Reindented function prototypes after `HDF5API' was added. Also
rewrapped long lines.
./src/H5Flow.c
Added an `#ifdef WIN32' around an unused variable.
./src/H5api_adpt.h
Removed extra carriage returns inserted by "broken" operating
system.
./src/H5Dprivate.h
./src/H5Oprivate.h
./src/H5Vprivate.h
./src/H5private.h
Removed extraneous inclusion of H5api_adpt.h since it's
included in H5public.h which is included by everything.
./src/Makefile.in
Added H5api_adpt.h to the list of public header files to fix
broken `make install'.
----------------------
./doc/html/H5.format.html
./src/H5HG.c
Fixed a bug in the global heap that caused H5HG_read() to
write past the end of the buffer in certain cases.
./test/big.c
The test is skipped if hdf5 was configured with
`--disable-hsizet'.
./src/H5Ofill.c
Data type conversions are implemented for the fill value.
./src/H5.c
Tracing prints one of H5P_FILE_CREATE, H5P_FILE_ACCESS,
H5P_DATASET_CREATE, H5P_DATASET_XFER, or H5P_MOUNT instead of
the more cryptic H5I_TEMPLATE_* constants.
./src/H5D.c
Removed prototype for H5D_find_name().
./src/H5I.c
The GROUP_MASK and ID_MASK are both calculated from GROUP_BITS
instead of being set by hand.
We don't use the sign bit of hid_t; all valid hid_t values are
positive so we can say things like `if ((file=H5Fopen(...))<0)'.
Changed `(int)pow(2.0,x)' to `1<<x' so we don't have to worry
about rounding.
Fixed H5I_get_type() so it doesn't always fail an assertion.
./src/H5E.c
./src/H5Epublic.h
Added minor error H5E_MOUNT
./src/H5F.c
./src/H5Fprivate.h
Added H5Fmount() and H5Funmount(). Mounting and unmounting
works as documented but some of the other things aren't
implemented yet, the biggest being current working groups
always acting on the root of the mount tree, and H5Fclose()
closing the entire tree. The rest of the stuff will be added
shortly...
./src/H5P.c
./src/H5Ppublic.h
Added the H5P_MOUNT property list but haven't implemented any
particular properties for it yet.
./src/H5Gstab.c
Hard links across files return an error instead of failing an
assertion.
Changes since 19980930
----------------------
./MANIFEST
./doc/html/H5.format.html
./src/H5D.c
./src/H5Dprivate.h
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5O.c
./src/H5Ofill.c [NEW]
./src/H5Oprivate.h
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Sprivate.h
./src/Makefile.in
Added fill values as documented in previous e-mails. They only
work for chunked datasets so far.
./src/H5E.c
./src/H5Epublic.h
./src/H5P.c
Changed H5E_TEMPLATE to H5E_PLIST.
./src/H5Omtime.c
./src/H5V.c
Fixed a typo in a comment.
./src/H5Tconv.c
Fixed a bug in compound type conversions that caused an
assertion to fail.
----------------------
./MANIFEST
Changed freebsd2.2.1 to freebsd2.2.7 to match change in file name.
Added a few missing files. Are all of these supposed to be
distributed?
./tools/testfiles/h5dumptst.c
+ ./tools/testfiles/tall-1.ddl
+ ./tools/testfiles/tall-2.ddl
+ ./tools/testfiles/tall-3.ddl
./tools/testfiles/tall.ddl
./tools/testfiles/tall.h5
+ ./tools/testfiles/tattr-1.ddl
+ ./tools/testfiles/tattr-2.ddl
+ ./tools/testfiles/tattr-3.ddl
+ ./tools/testfiles/tattr-4.ddl
./tools/testfiles/tattr.ddl
./tools/testfiles/tattr.h5
+ ./tools/testfiles/tdset-1.ddl
+ ./tools/testfiles/tdset-2.ddl
+ ./tools/testfiles/tdset-3.ddl
+ ./tools/testfiles/tdset-4.ddl
./tools/testfiles/tdset.ddl
./tools/testfiles/tdset.h5
+ ./tools/testfiles/tgroup-1.ddl
+ ./tools/testfiles/tgroup-2.ddl
+ ./tools/testfiles/tgroup-3.ddl
./tools/testfiles/tgroup.ddl
./tools/testfiles/tgroup.h5
+ ./tools/testfiles/tslink-1.ddl
+ ./tools/testfiles/tslink-2.ddl
./tools/testfiles/tslink.ddl
./tools/testfiles/tslink.h5
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dprivate.h
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5Gstab.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Sprivate.h
./test/istore.c
Implemented split ratios as documented in previous
e-mails. Frank, the new API functions are H5Pget_btree_ratios()
and H5Pset_btree_ratios().
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Sprivate.h
Added more type checking for the data transfer property list
passed to these functions.
./src/H5D.c
Added a dataset transfer property list as an argument to
H5D_allocate() for the parallel version in order to pass split
ratios down to H5F_istore_lock(). Eventually we won't need
the split ratios for H5D_allocate() because we'll build a
completely full B-tree from the leaves up, but it might be
useful to have other transfer properties at that level
anyway. I always caled H5D_allocate() with &H5D_dflt_xfer.
./MANIFEST
./test/Makefile.in
./test/overhead.c [NEW]
Added an `overhead' test run by `make timings'.
----------------------
./src/H5F.c
./src/H5Fprivate.h
./src/H5P.c
./src/H5Ppublic.h
./test/chunk.c
./test/dsets.c
The number of slots in the raw data cache can be queried or
set with H5Pget/set_cache(), which now take an extra argument.
The default number of slots is 521 and the default maximum
size is 1MB.
./src/H5Fistore.c
./src/H5Fprivate.h
Finished optimizations. The cache is now a hash and a linked
list instead of an array. The cpu time on my machine for
H5F_istore_lock() has been cut by 60% and H5F_istore_unlock() by
35%.
----------------------
./MANIFEST
Replaced Compression.html with Filters.html
./doc/html/Filters.html
./src/H5Ocomp.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Z.c
./src/H5Zprivate.h
Added two extra arguments to H5Pget_filter() in order to
retrieve the filter name. The name is the name registered for
the filter with H5Zregister(), but if the dataset creation
property originally came from an existing file then the name
is that which is stored in the file.
./tools/h5ls.c
The `-v' option now prints the names of the filters.
./src/H5B.c
./src/H5Fistore.c
./src/H5O.c
./src/H5Oefl.c
./src/H5Oprivate.h
./src/H5P.c
Plugged a memory leak.
./src/H5MMprivate.h
H5MM_malloc(0) and H5MM_calloc(0) actually allocate a single
byte in order to be sure that we get a valid pointer.
./src/H5S.c
./src/H5Sselect.c
Fixed pointer->integer conversions in error return values in
three places.
----------------------
./bin/release
Added ./Makefile to the distribution again -- it got lost in
the changes last week although it isn't all that important a
file since it gets clobbered by configure anyway.
./bin/trace
./doc/html/Filters.html
./doc/html/H5.format.html
./doc/html/H5.user.html
./src/H5.c
./src/H5D.c
./src/H5Dprivate.h
./src/H5E.c
./src/H5Epublic.h
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5O.c
./src/H5Ocomp.c
./src/H5Oprivate.h
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Sprivate.h
./src/H5Ssimp.c
./src/H5Z.c
./src/H5Zprivate.h
./src/H5Zpublic.h
./src/hdf5.h
./test/dsets.c
./tools/h5ls.c
Added the data filter pipeline, a generalization of the
compression stuff which allows things like checksums,
encryption, compression, performance monitoring, etc. See
./doc/html/Filters.html for details -- it replaces the
Compression.html doc.
./src/H5T.c
Cleaned up debugging output.
./config/linux
Added checks for egcs and pgcc and changed optimization flags
for the compilers.
./src/H5G.c
./tools/h5dump.c
Fixed compiler warnings in these files and others.
./configure.in
./src/H5private.h
./test/mtime.c
Added a check for difftime() and defined HDdifftime() to do
something else on systems that don't have difftime().
----------------------
./src/H5.c
Handle hid_t of type H5_TEMPBUF, arguments usually called tbuf_id.
Added array tracing where the array rank is stored in a simple
data space. Just use the name of the data space argument when
declaring the array argument:
herr_t
H5Sselect_hyperslab (hid_t space_id, H5S_seloper_t op,
const hssize_t start[/*space_id*/],
const hsize_t _stride[/*space_id*/],
const hsize_t count[/*space_id*/],
const hsize_t _block[/*space_id*/])
and when the program runs you'll see array values printed:
H5Sselect_hyperslab(space=218103813, op=H5S_SELECT_SET,
start=0xbfffef4c {0}, _stride=NULL,
count=0xbfffef44 {64},
_block=NULL) = SUCCEED;
Added more symbolic data types to the tracing output.
./src/H5A.c
./src/H5Apublic.h
./src/H5D.c
./src/H5Dpublic.h
./src/H5F.c
./src/H5Fpublic.h
./src/H5G.c
./src/H5Gpublic.h
./src/H5P.c
./src/H5Ppublic.h
./src/H5S.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Spoint.c
./src/H5Spublic.h
./src/H5Sselect.c
./src/H5Ssimp.c
./src/H5TB.c
./src/H5V.c
Changed some API argument names to be more consistent with
other API functions and to produce better tracing output.
Reformatted some long lines. Indented printf statements.
./tools/h5ls.c
Fixed warnings about unsigned vs. signed comparisons.
./src/H5F.c
./src/HFcore.c
./src/H5Ffamily.c
./src/H5Flow.c
./src/H5Fmpio.c
./src/H5Fprivate.h
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5MF.c
./src/H5P.c
./src/H5Ppublic.h
Added H5Pset_alignment() so that it is now possible to align
file allocation requests on application-specified
boundaries. Any request >= the specified threshold will begin
on an address which is a multiple of the specified
alignment. Defaults are one for threshold and alignment. The
alignment is done on relative addresses, so the size of the
user block can affect the location of the data in the file.
./src/H5D.c
./src/dsets.c
Added a test for, and fixed the data space caching bug in
datasets. Extending a dataset through one handle will cause
all other handles to the same dataset to get the new dataset
size.
./src/H5S.c
./src/H5Sprivate.h
Removed an unused argument from H5S_read() which duplicated
information from the other argument.
./config/linux
Made `--enable-parallel' the default on my system. It used to
be that way before but then I accidently turned it off and
forgot about it.
./src/H5Fmpio.c
Qualified some function arguments with __unused__. Changed a
couple places where NULL was returned on error for herr_t
functions.
./src/H5P.c
Removed unused autos from H5Pset_mpi().
----------------------
./html/Datasets.html
Fixed a couple of typos.
./src/H5.c
Added the `Z' modifier to HDfprintf() for `size_t' sizes. Use
it like this:
HDfprintf(stderr,"size is %Zd\n", (size_t)x);
./src/H5AC.c
./src/H5F.c
./src/H5Fprivate.h
The maximum number of meta data objects that can be cached can
be set from the application (but the library might not honor
it every time; it's a hint).
./src/H5D.c
Changed a warning message so it's not so alarming.
./src/H5Fistore.c
Chunks can be cached.
./src/H5O.c
./src/H5Oprivate.h
Added H5O_copy() and H5O_free() to copy and free messages.
./src/H5P.c
./src/H5Ppublic.h
Added H5Pset_cache() and H5Pget_cache() and changed lots of
"template" to "property list".
./src/H5Z.c
./src/H5Zpublic.h
Miscellaneous little things to clean up. Mostly just removed
H5Z_MAXVAL and added H5Z_USERDEF_MIN and H5Z_USERDEF_MAX.
./MANIFEST
./test/Makefile.in
./test/chunk.c [NEW]
Added a performance test for chunk caching. It looks at the
amount of I/O instead of timing because timing is partly
dependent on the chunk size and I wanted a measurement that
was a function of only the cache size. Run `chunk' with no
arguments and then say `gnuplot x-gnuplot' to see the plots
(press return between plots). Postscript files are created for
each plot.
./test/big.c
./test/cmpd_dset.c
./test/extend.c
./test/external.c
./test/gheap.c
Added H5F_ACC_DEBUG so we can see cache performance
statistics.
----------------------
./src/H5Bprivate.h
./src/H5F.c
./src/H5Ffamily.c
./src/H5Fprivate.h
./src/H5Gpkg.h
./src/H5MF.c
./src/H5P.c
./src/H5Ppublic.h
./test/big.c
./html/Big.html
./html/Files.html
Family members can now be any size >1kB. Got rid of some
places where we were reading a property list after it was
closed.
./MANIFEST
./src/Makefile.in
./src/h5repart.c
A program to repartition file families. The source and/or
destination may be files or file families. Examples:
testhdf5 -c -o stab
h5repart -m 1m tstab2.h5 x%05d.h5
h5repart -m 100k x%05d.h5 y%05d.h5
h5repart y%05d.h5 z.h5
diff tstab2.h5 z.h5
./src/h5ls.c
Added a usage message, replaced assertions with error
messages.
./config/linux
Made a warning message more obvious. Added better
optimization flags for Pentium-Pro's.
----------------------
This checkin completes the compression stuff almost two weeks ahead of
schedule. It should now all be working and documented.
./Makefile.in
./config/commence.in
A distclean causes the distribution makefile to be copied to
Makefile again.
./src/H5D.c
Opening a compressed dataset now works properly.
./src/H5P.c
./src/H5Ppublic.h
The H5Pset_compression() and H5Pget_compression() now work as
documented.
./src/H5T.c
Fixed a bug that causes the library to crash sometimes if
debugging is turned on for the `t' package.
./src/H5Z.c
Fixed a bug where the number of bytes attributed to
compression overruns was too low in the debugging output.
./test/dsets.c
More compression tests: reading of uninitialized data, opening
an existing dataset and reading it, partial I/O which isn't
aligned on chunk boundaries, and use of application-defined
compression methods.
./MANIFEST
Removed ./test/dspace.c.
----------------------
./html/Compression.html [NEW]
./html/Datasets.html
./html/H5.format.html
./html/H5.user.html
Documented compression. A couple of the H5P functions aren't
quite implemented yet but they're coming soon...
./src/H5Dprivate.h
./src/H5E.c
./src/H5Epublic.h
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5MF.c
./src/H5MFprivate.h
./src/H5O.c
./src/H5Ocomp.c [NEW]
./src/H5Oprivate.h
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sprivate.h
./src/H5Ssimp.c
./src/H5Z.c [NEW]
./src/H5Zprivate.h [NEW]
./src/H5Zpublic.h [NEW]
./src/Makefile.in
./src/hdf5.h
./test/dsets.c
./test/istore.c
Compression is now mostly working. Don't try to open a
compressed dataset though because the compression message
won't be read.
./html/Datatypes.html
./html/H5.api.html
./src/H5.c
./src/H5private.h
./src/H5D.c
./src/H5T.c
./src/H5Tconv.c
./src/H5Tpkg.h
./src/H5Tprivate.h
./src/H5Tpublic.h
Added timing support. When compiled with H5T_DEBUG defined
the library will print conversion bandwidths when the library
closes. The H5Tregister functions take a string as the first
argument so the statistics output is meaningful.
./MANIFEST
Added new files.
./configure.in
./src/H5config.h.in
Check for getrusage(). Check for compress2() in libz.a and
the zlib.h header file. Added `z' to the debug list.
./src/H5B.c
./src/H5Bprivate.h
./src/H5Gnode.c
./src/debug.c
Cleaned up some indentation and added support to print istore
B-trees. From the debugger, give the B-tree address and the
dimensionality from the layout message of the object header.
./src/h5ls.c
The oid is printed as w❌y:z where w and x are the file ID
and y and z are the OID within the file. You can give z or
y*2^32+z as an argument to the debugger to print the object
header for the object.
./src/H5AC.c
Cleaned up statistics and made them match those reported by
H5T and H5Z.
./src/H5MM.c
./src/H5MMprivate.h
./src/H5Fistore.c
Finally got rid of a couple of long-standing const cast
warnings.
used to open an HDF5 file with MPIO access. The design has changed
to provide access-mode controll (independent or collective access)
as an argument in the H5D read/write calls.
Also removed the macros H5FACC_INDEPENDENT and H5FACC_COLLECTIVE.
The original thinking was to define how a file is independent or
collective access. They are replaced by the enum H5D_transfer_t
with values of H5D_XFER_INDEPENDENT and H5D_XFER_COLLECTIVE.
These values are used in the H5Pset_xfer to setup a data transfer
property list to be used during the H5D read/write calls.
The original access_mode field in the File access structure is
still there. It would be removed soon once the transfer control
can be passed down to the low level I/O routines.
H5D.c
Changed H5D_write and H5D_read to recognize collective transfer
requests. Current algorithm puts the xfer-mode in the file->mpio
structure. Not thread safe. Need to revise it to pass transfer
parameter as a function parameters to the lower levels.
H5Dprivate.h
H5Dpublic.h
Added xfer_mode to the transfer property list structure.
Added defination of transfer modes, H5D_XFER_INDEPENDENT and
H5D_XFER_COLLECTIVE.
H5Farray.c
Added code to handle collective access requests. Currently
works for one contiguous block at a time. Need to revise
it with the MPIO low level calls to combine all blocks as
one truely collective call.
H5P.c
H5Ppublic.h
Added H5Pset_xfer and H5Pget_xfer routines that can set and get
the transfer mode of a data transfer property list.
----------------------
./src/H5Osdspace.c
./html/H5.format.html
In the past we were allowed to have >2GB files on a 32-bit
machine as long as no dataset within the file was larger than
4GB (or whatever sizeof(size_t) is). That's been fixed now.
All dataset size calculations are done with `hsize_t' which is
normally defined as `unsigned long long'.
./src/H5F.c
./src/H5Ffamily.c
./src/H5Fprivate.h
./src/H5P.c
./src/H5Ppublic.h
The file family member size can now be set/queried. The
default is still 64MB, but it can be set to 1GB by saying:
H5Pset_family (plist, 30, H5P_DEFAULT);
When opening an existing file family the specified
bits-per-member is ignored and the first member of the family
determines the bits-per-member, which can be retrieved with
H5Pget_family().
./acconfig.h
./configure.in
./src/H5config.h
./src/H5public.h
Added `--disable-hsizet' so that those with old GCC compilers
(<2.8.1) can still compile the code.
./src/H5.c
./src/H5private.h
Added HDfprintf() which works just like fprintf() except you
can give `H' as a size modifier for the integer conversions
and supply an `hsize_t' or `hssize_t' argument without casting
it. For instance:
hsize_t npoints = H5Sget_npoints(space);
HDfprintf(stdout,"Dataset has %Hd (%#018Hx) points\n",
npoints, npoints);
You can now give `%a' as a format to print an address, but all
formating flags are ignored and it causes the return value of
HDfprintf() to not include the characters in the address (but
who uses the return value anyway :-). Example:
H5G_t *grp;
HDfprintf(stdout, "Group object header at %a\n",
&(grp->ent.header));
Added HDstrtoll() which works exactly like [HD]strtol() except
the result is an int64.
./src/debug.c
Large addresses can now be entered from the command-line. Use
either decimal, octal (leading `0') or hexadecimal (leading
`0x') when giving the address.
./src/h5ls.c
The printf format for dataset dimensions was changed to `%Hu'
to support large datasets.
./test/big.c [NEW]
A test for big datasets on 32-bit machines. This test is not
run by default. Don't try to run it on an nfs-mounted file
system or other file system that doesn't support holes because
it creates two 32GB datasets of all zero.
----------------------
./src/H5B.c
./src/H5D.c
./src/H5Dprivate.h
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fcore.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Flow.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Fsec2.c
./src/H5Fsplit.c
./src/H5Fstdio.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5MF.c
./src/H5MFprivate.h
./src/H5O.c
./src/H5Ocont.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Olayout.c
./src/H5Oname.c
./src/H5Oprivate.h
./src/H5Osdspace.c
./src/H5Oshared.c
./src/H5Ostab.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5S.c
./src/H5Sprivate.h
./src/H5Spublic.h
./src/H5Ssimp.c
./src/H5Tconv.c
./src/H5Tpkg.h
./src/H5V.c
./src/H5Vprivate.h
./src/H5private.h
./src/H5public.h
./src/h5ls.c
./test/cmpd_dset.c
./test/dsets.c
./test/extend.c
./test/external.c
./test/hyperslab.c
./test/iopipe.c
./test/istore.c
./test/shtype.c
./test/tfile.c
./test/th5s.c
Anything having to do with the size of a dataset now uses the
types `hsize_t' and `hssize_t' which must be the same size and
at least as large as `size_t'. This isn't fully tested yet,
so hsize_t and hssize_t are defined as size_t and ssize_t in
H5public.h. Setting them to larger values will trip up gcc
versions less than 2.8.1 on x86 platforms.
Documented unused function formals with `__unused__' before
the formal name. This also has the effect of supressing
warning messages for gcc since it's defined to be
`__attribute__((unused))' in the H5private.h file.
./src/debug.c
./src/h5ls.c
If the file name contains a `%' then the file is opened as a
file family with H5P_DEFAULT for the file member access
property list.
./src/h5ls.c
The group name is optional, defaulting to `/'.
./src/hdf5.h
Added some missing public header files.
----------------------
./MANIFEST
./src/H5HG.c [new]
./src/H5HGprivate.h [new]
./src/H5HGpublic.h [new]
./src/Makefile.in
Added global heaps, H5HG. This compiles but is untested.
./src/H5D.c
./src/H5G.c
./src/H5Gnode.c
./src/H5Gstab.c
./src/H5HL.c [renamed from H5H.c]
./src/H5HLprivate.h [renamed from H5Hprivate.h]
./src/H5HLpublic.h [renamed from H5Hpublic.h]
./src/H5Oefl.c
./src/Makefile.in
./src/hdf5.h
./test/theap.c
Renamed local heaps from H5H to H5HL.
./src/H5AC.c
./src/H5ACprivate.h
./src/debug.c
Added debugging support for global heap collections.
./html/H5.format.html
Ripped out the old small object heap documentation and added
documentation for the new global heap.
./html/H5.api.html
./src/H5P.c
./src/H5Ppublic.h
./src/H5F.c
./src/H5Fprivate.h
./src/H5private.h
The H5Pget_version() function no longer returns a small object
heap version number. Instead, each collection of the global
heap has its own version number instead of a single number in
the file boot block.
./bin/release
Omit files marked for removal by CVS even though the removal
hasn't been committed yet.
./test/iopipe.c
Removed warnings about casting unsigned int to double on Irix64.
./MANIFEST
Removed ./test/testpar/phdf5sup.c
----------------------
./src/H5D.c
./src/H5T.c
./src/H5Tprivate.h
Fixed a bug found by Kevin Powell regarding preservation of
existing data during a read or write when the source and
destination data types are the same. Thanks Kevin!
Fixed a couple warnings on Irix64.
./src/H5D.c
./src/H5Dprivate.h
./src/H5P.c
./src/H5Ppublic.c
./src/H5Tconv.c
./src/H5Tpublic.h
./test/cmpd_dset.c
The application can now turn on/off the part of the I/O
pipeline that deals with preservation of initialized data.
The default is off since this will be the usual case and
turning it on slows down the pipe. Use H5Pset_preserve() and
H5Pget_preserve().
./src/H5Fistore.c
Added an optimization for reading/writing a single chunk of
chunked storage.
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Olayout.c
./src/H5Oname.c
./src/H5Osdspace.c
./src/H5Ostab.c
Fixed warnings reported by marek@iiasa.ac.at. Thanks Marek.
./src/H5Odtype.c
Fixed a failing assert() wrt reading compound types (not
present in hdf5-1.0.0a).
----------------------
./configure.in
./configure
./test/iopipe.c
Added check for system(3)
./config/freebsd2.2.1
./config/linux
Added -DH5D_DEBUG to the debug flags.
./src/H5D.c
./src/H5Dprivate.h
./src/H5P.c
./src/H5Ppublic.h
./src/H5Sprivate.h
./src/H5Ssimp.c
./html/Datasets.html
Finally implemented strip mining in the I/O pipeline, placing
a user-defined upper bound on the amount of temporary memory
used by the pipeline. The default is 1MB type conversion and
background buffers allocated/freed on demand. However, the
size can be changed and/or application-allocated buffers
supplied with H5Pset_buffers() called on the data transfer
property list passed to H5Dread() and H5Dwrite().
Minor optimizations to H5Dread() and H5Dwrite(). More coming
later...
./test/dsets.c
Added calls to H5Pset_buffer() to test application-defined
temporary buffers in the I/O pipeline.
./test/Makefile.in
Removed `iopipe' from the list of confidence tests. Saying
`make timings' in the test directory runs timing tests. I did
this because (1) they don't really test anything new, and (2)
they can take a long time to run.
----------------------
./html/Dataspaces.html
./html/Errors.html
./html/Files.html
./html/H5.api.html
./html/review1.html
./src/H5private.h
./src/H5public.h
./test/dsets.c
./test/dtypes.c
Removed all the types like `int32' and `intn' into private
headers since they violate the naming scheme and pollute
application name space. Besides, our test files only use them
in a handful of places and it's probably useless to export
them to the app. The app is always written in terms of
standard numeric types or its own numeric types and probably
never in terms of HDF5 numeric types. If it were, then the
user would have to copy from their type to hdf5 type for
almost every hdf5 API function call! Same goes for return
values. I also removed SUCCEED/FAIL from the API since apps
should be checking against zero anyway.
if (FAIL==(space=H5Screate_simple(...))) /*wrong*/
if ((space=H5Fcreate_simple(...)<0)) /*right*/
./src/H5.c
Changed arguments of H5version() from `uintn' to `unsigned'.
./src/H5Tpublic.h
./src/H5T.c
Changed return type of H5Tget_nmembers() from `intn' to `int'
./src/H5A.c
./src/H5Aprivate.h
./src/H5Apublic.h
Changed `H5Asearch_func_t' to `H5A_search_func_t' and moved
its definition from the public to the private header file.
./html/H5.format.html
Documented changes made to the external file list (H5O_EFL)
message.
./src/H5D.c
./src/H5Dprivate.h
./src/H5E.c
./src/H5Epublic.h
./src/H5O.c
./src/H5Oefl.c
./src/H5Oprivate.h
./src/H5P.c
./src/H5Ppublic.h
Added partial support for external raw data files. HDF5 can
now describe external raw data files by listing the file
names, offsets, and size for a dataset. However, we will
restrict a dataset to be stored "contiguously" when the
external file list is viewed as a single address space. The
current implementation is unable to read/write to external
files--that will come later this week as will documentation.
For now, take a look at ./test/external.c, particularly the
calls to H5Pset_external().
./test/Makefile.in
./test/external.c [NEW]
./MANIFEST
Added tests for external storage. Note: the read test is
supposed to fail at this point since reading external datasets
is not implemented yet. There is no write test.
./src/H5S.c
./src/H5Sprivate.h
./src/H5Ssimp.c
Added H5S_get_npoints_max() to return the maximum possible
number of data points in a data space.
Added an extra argument to H5S_get_dims() which returns the
maximum dims.
./src/H5F.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5M.c [DEPRICATED]
./src/H5Mpublic.h [DEPRICATED]
Changed `template' to `property list' in lots of places.
./src/H5Osdspace.c
Removed an extra `\n' from a print statement.
./src/H5S_public.h
Changed H5S_UNLIMITED to the maximum size_t value.
./test/extend.c
"Extendable" is spelled "extendible".
./src/H5Farray.c
./src/H5V.c
./src/H5Vprivate.h
./test/hyperslab.c
Strides are now type ssize_t instead of int. These have
nothing to do with the sample granularity arguments for
hyperslabs, which are also called "strides" in the code.
./test/tstab.c
Changed assumptions about default address and length sizes.
----------------------
./html/Files.html
Added a few details for some of the new H5Pset/get functions.
./src/H5F.c
./src/H5Fpublic.h
Fixed automatic closing of files on exit().
Added public H5F_ACC_DEBUG. Using it to create or open a file
turns on debugging for that file, which currently just prints
cache statistics when the file is closed.
./src/H5G.c
An error is returned if one tries to set the current working
group to something other than a group.
./src/H5Gnode.c
Fixed a symbol table bug. Under certain circumstances it was
possible to enter a symbol in such a way that lookup of that
symbol failed. A bug report was sent to hdf5dev.
./src/H5P.c
./src/H5Ppublic.h
Added the H5Pget_...() functions for file drivers. The
H5Pget_mpi() is a no-op that always fails.
----------------------
./INSTALL
Added instructions for which C flags to set for debugging.
./src/H5C.c
./src/H5Cpublic.h
H5Cset_chunk() takes const pointer.
./src/H5D.c
./src/H5Dprivate.h
./src/H5Dpublic.h
Added H5Dextend() to extend the dimensions of a dataset.
./src/H5Osdspace.c
./src/H5P.c
./src/H5Pprivate.h
./src/H5Ppublic.h
./test/cmpd_dset.c
./test/dsets.c
./test/th5p.c
Added the optional `maxdims' argument to H5Pcreate_simple()
and defined constant H5P_UNLIMITED which can appear in the
maxdims. Added `const' to arguments.
Implemented H5Pcopy()
Removed the unused file argument from H5P_modify.
Added H5P_extend().
Removed the `flags' field from simple data types and we
determine if the `max' or `perm' arrays are valid by looking
at the pointer. Cleaned up the H5O_sdspace_debug output.
./src/H5T.c
Fixed a printf format.
./MANIFEST
./test/Makefile.in
./test/extend.c [NEW]
Added a test for multi-dimensional unlimited dimensions.
----------------------
./src/H5P.c
Removed H5Pcreate()
./src/H5Psimp.c
Copy int[] return value from H5P_get_hyperslab() to size_t[]
argument to pass to other hyperslab functions.
./test/dsets.c
Added a call to H5Eprint() to help track down Alberts O2k bug.
Changed interface to the H5P..hyperslab functions to 'int' instead of 'size_t'.
Cleaned up lots of warnings on IRIX 6.2 platform. Minor other tweaks to get
to a mostly clean build on the SGI. It still whines about 'long long' being
non-standard and some "pointless comparison of unsigned with 0" but those
aren't big problems.
----------------------
./src/H5D.c
./src/H5Dpublic.c
Added H5Dget_space().
./src/H5Gstab.c
./src/H5H.c
Fixed a comparison with size_t against <0.
./src/H5P.c
./src/H5Pprivate.h
./src/H5Ppublic.h
Changed `intn' to `int' for public function args and
returns. H5Pget_hyperslab() returns the dimensionality.
./test/testhdf5.h
Clears error stack more frequently.
./src/H5Psimp.c
./test/cmpd_dset.c
Impelementing data space conversion.
----------------------
./Makefile.in
./config/commence.in
With GNU make you can now use `-j' and `-l' options and things
get built correctly. I can do a `make -j -l6 test' from a
clean hdf5 source tree (after configure) in 45 seconds (8 to
build dependencies, 26 to compile everything, and 11 to run
the tests).
./src/H5Gnode.c
Removed a comment that no longer applies.
./src/H5P.c
./src/H5Pprivate.h
./src/H5Ppublic.h
Changed H5Pselect_hyperslab() to H5Pset_hyperslab() and added
H5Pget_hyperslab() and H5P_get_hyperslab().
Replaced a couple short memset() calls with a for loop.
Removed `if (foo!=NULL)' from around H5MM_xfree() calls.
Clear hslab_def when the hyperslab disappears.
./src/H5Tpublic.h
Removed trailing enum comma.
to the HDF4 SDwritedata routine: start, count & stride. The start indicates
the base location of the hyperslab, count indicates the number of elements in
the hyperslab and stride indicates the packing of the hyperslab. The hyperslab
is assumed to be the same rank as the dataspace it is located within. The
hyperslab is stored for future use, no I/O occurs in this routine.
