Take further advantage of the readers/writer locking in the metadata cache
and push the locking of the local heap up to the routines which call B-tree
routines, which should drastically lower the number of metadata cache protect/
unprotect calls.
Tested on:
Mac OS X/32 10.4.9 (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
Fully enable readers/writer locking in metadata cache and correct errors
in cache clients which were modifying a cache entry while only holding a
read lock on it.
Tested on:
Mac OS X/32 10.4.9 (amazon)
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
FreeBSD/32 6.2 (duty)
FreeBSD/64 6.2 (liberty)
Tested platform:
Kagiso only since it is only a comment block change. If it works in one
machine, it should work in all, I hope. Still need to check the parallel
build on copper.
Finish refactoring internal link deletion code, to make it possible to
wrap up the H5Ldelete_by_idx() coding.
Tested on:
Linxu/32 2.6 (chicago)
Linxu/64 2.6 (chicago2)
New feature
Description:
Check in baseline for compact group revisions, which radically revises the
source code for managing groups and object headers.
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
This initiates the "unstable" phase of the 1.7.x branch, leading up
to the 1.8.0 release. Please test this code, but do _NOT_ keep files created
with it - the format will change again before the release and you will not
be able to read your old files!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
Solution:
There's too many changes to really describe them all, but some of them
include:
- Stop abusing the H5G_entry_t structure and split it into two separate
structures for non-symbol table node use within the library: H5O_loc_t
for object locations in a file and H5G_name_t to store the path to
an opened object. H5G_entry_t is now only used for storing symbol
table entries on disk.
- Retire H5G_namei() in favor of a more general mechanism for traversing
group paths and issuing callbacks on objects located. This gets us out
of the business of hacking H5G_namei() for new features, generally.
- Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t
- Lots more...
Platforms tested:
h5committested and maybe another dozen configurations.... :-)
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
Code cleanup
Description:
Refactor metadata cache to merge "dirtied" flag in with other flags for
H5AC_unprotect and H5C_unprotect.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Interim checkin of code changes moving management of the is_dirty flag
into the cache code.
Description:
Prior to this checkin, management of the is_dirty flag was handled
above the level of the metadata cache. This can no longer be allowed,
as it introduces a race condition in the proposed fix for a cache
coherency bug in PHDF5.
Solution:
Move management fo the is_dirty flag to the cache code proper.
Entries are now marked as dirty via a flag on the unprotect call.
Platforms tested:
h5committested
Misc. update:
Update
Description:
A lot of modifications for the FPHDF5 stuff:
H5AC.c
H5ACprivate.h - Removed AC_find (it's replaced with AC_protect
and AC_unprotect). Added flushing if it's an FPHDF5 driver and
we're doing an AC_set or AC_unprotect with the dirty flag set.
H5B.c - Split up the B_flush function into different functions
since the one function was doing serialization which is better
left as a separate entity.
H5D.c - Removed some FPHDF5 code that was incorrect
H5F.c - Split up the F_flush function so that it no longer
allocates file space. Created new functions (F_init_superblock,
F_read_superblock, and F_write_superblock) for greater modularity
and so that the FPHDF5 non-captain processes can read the
superblock after the captain process writes it.
H5FD.c - Error message correction.
H5FDfphdf5.c - Removed MPI barrier call that wasn't needed.
H5FPclient.c
H5FPserver.c - Modified so that if a process requests data that
isn't exactly aligned, we can return it if we have the block that
contains the requested address.
H5G.c
H5Gent.c
H5Gnode.c
H5HL.c
H5HLpkg.h
H5HLprivate.h
H5Oefl.c - Removed the H5HL_peek function since it was doing a
(now unsafe) holding of the information in the cache. Replaced
with protect and unprotect calls.
H5TB.c - Error fix. The TB_dless function wasn't working
properly.
H5Gstab.c - Format change.
H5err.txt
H5Edefin.h
H5Einit.h
H5Epubgen.h
H5Eterm.h - Added new error code.
Platforms tested:
Modi4 (paralle, Fortran)
Sol (Fortran)
Linux (C++, Fortran)
Copper (Parallel, Fortran)
Misc. update:
Code cleanup
Description:
Break some of the "debugging" routines into their own module, so they
aren't pulled into every executable, which certainly isn't going to use them.
Platforms tested:
h5committested
Code cleanup
Description:
Limit the scope on more function prototypes/macros/typedefs.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not necessary.
Code cleanup, bug fix & new feature.
Description:
The local heaps were missed in the last changes to the metadata "dirty"
flag, making them operate incorrectly in the metadata cache. Fixed by changing
to use "cache_info.dirty" flag instead.
Track changes for internal metadata API, regarding marking metadata as clean
without flushing it to the file.
Added ability to delete a local heap in the file.
Added ability to reduce the size of a local heap if there is a free-space
block at the end of the heap and the free-space block is over half the size of
the local heap.
Some migration of package-specific information out of the private header
and into the code module as well.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
Purpose:
__DLL__ is a keyword in some platforms and __DLL__ is also defined as a macro for windows DLL applications.
That causes problems.
Description:
Solution:
Use H5_DLL*** to replace __DLL***__ at all header files.
Change the macro defination at H5api_adpt.h.
Platforms tested:
linux2.2.18smp, irix64, solaris 2.7 and windows 2000
Code cleanup (sorta)
Description:
When the first versions of the HDF5 library were designed, I remembered
vividly the difficulties of porting code from a 32-bit platform to a 16-bit
platform and asked that people use intn & uintn instead of int & unsigned
int, respectively. However, in hindsight, this was overkill and
unnecessary since we weren't going to be porting the HDF5 library to
16-bit architectures.
Currently, the extra uintn & intn typedefs are causing problems for users
who'd like to include both the HDF5 and HDF4 header files in one source
module (like Kent's h4toh5 library).
Solution:
Changed the uintn & intn's to unsigned and int's respectively.
Platforms tested:
FreeBSD 4.4 (hawkwind)
Update
Description:
Changed
#include <hdf_file.h>
construct to
#include "hdf_file.h"
so that the GNU compiler can more easily pick up the dependencies
which it places in the .depend and Dependencies files. Also
regenerated the Dependencies to go along with this.
Platforms tested:
Linux
----------------------
./src/H5.c
Fixed more dependency problems in H5_term_library(). There was
a bug in the previous version that could cause the wrong EOF
marker to be written to the boot block under certain
circumstances. Hopefully this fixes it although I don't ready
access to a test case (Mark Miller will test it).
./src/H5F.c
./src/H5Fprivate.h
Added an H5F_close_all() that is similar to
H5F_term_interface() but which doesn't close the
interface. Files that don't have open object headers are
closed, others are delayed until all object headers close. All
files are flushed.
./src/H5ACprivate.h
./src/H5Bprivate.h
./src/H5Dprivate.h
./src/H5Eprivate.h
./src/H5Fprivate.h
./src/H5Gpkg.h
./src/H5Gprivate.h
./src/H5Gpublic.h
./src/H5HGprivate.h
./src/H5HLprivate.h
./src/H5Iprivate.h
./src/H5MFprivate.h
./src/H5MMprivate.h
./src/H5Oprivate.h
./src/H5Pprivate.h
./src/H5Ppublic.h
./src/H5RAprivate.h
./src/H5Sprivate.h
./src/H5Spublic.h
./src/H5Tpkg.h
./src/H5Tprivate.h
./src/H5Tpublic.h
./src/H5Vprivate.h
./src/H5Zprivate.h
./src/H5private.h
./src/H5public.h
Reindented after __DLL__ was added.
----------------------
./tools/h5tools.c
Strings are not converted to null-padding before being
printed; they are printed with whatever byte values appear in
the file.
./tools/h5ls.c
Now able to display attribute data type and data.
Added a `-f' or `--full' switch which causes the full name of
each object to be displayed instead of just the base name.
Added a `-r' or `--recursive' switch that recursively prints
the contents of groups, avoiding cycles.
More bulletproofing for non-printable characters in things
like object names, attribute names, and comment strings. We
don't want listing a file to send termal escape sequences
because it's sometimes possible to execute commands that way.
Since h5ls doesn't usually use quotes around object names we
must sometimes escape space characters.
External files are listed in a table to make the output less
confusing.
./tools/h5tools.c
./tools/h5tools.h
Changed h5dump() to h5dump_dset() and added h5dump_mem(). Also
make h5dump_fixtype() public.
./test/dtypes.c
Wrote some data to an attribute to test h5ls attribute
printing.
./src/H5ACprivate.h
./src/H5Apublic.h
./src/H5Dprivate.h
./src/H5Dpublic.h
./src/H5Epublic.h
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gprivate.h
./src/H5HLprivate.h
./src/H5Oprivate.h
./src/H5Ppublic.h
./src/H5RApublic.h
./src/H5Rpublic.h
./src/H5Spublic.h
./src/H5Tpkg.h
./src/H5Tpublic.h
./src/H5Vprivate.h
./src/H5Zpublic.h
./src/H5private.h
./src/H5public.h
Reindented function prototypes after `HDF5API' was added. Also
rewrapped long lines.
./src/H5Flow.c
Added an `#ifdef WIN32' around an unused variable.
./src/H5api_adpt.h
Removed extra carriage returns inserted by "broken" operating
system.
./src/H5Dprivate.h
./src/H5Oprivate.h
./src/H5Vprivate.h
./src/H5private.h
Removed extraneous inclusion of H5api_adpt.h since it's
included in H5public.h which is included by everything.
./src/Makefile.in
Added H5api_adpt.h to the list of public header files to fix
broken `make install'.
----------------------
./configure.in
Moved setting of compiler warning switches earlier in the file.
Turned on more warning switches to gcc.
./config/linux
Prints a warning if the gcc version is less than 2.8.1 since
that version has problems with register allocation for `long
long'.
./html/Datatypes.html
Documented sharing of data types between datasets.
./src/H5G.c
./src/H5Gpublic.h
Implemented H5Gmove(), H5Glink() and H5Gunlink() for hard
links. Still have soft links to do.
./src/H5AC.c
./src/H5ACprivate.h
./src/H5D.c
./src/H5E.c
./src/H5Eprivate.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fcore.c
./src/H5Ffamily.c
./src/H5Fistore.c
./src/H5Flow.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Fsec2.c
./src/H5Fstdio.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5Gprivate.h
./src/H5HG.c
./src/H5HL.c
./src/H5HLprivate.h
./src/H5I.c
./src/H5Iprivate.h
./src/H5MM.c
./src/H5MMprivate.h
./src/H5O.c
./src/H5Oefl.c
./src/H5Oprivate.h
./src/H5Osdspace.c
./src/H5Oshared.c
./src/H5Ostab.c
./src/H5P.c
./src/H5S.c
./src/H5Ssimp.c
./src/H5T.c
./src/H5Tconv.c
./src/H5Tprivate.h
./src/H5Tpublic.h
./src/H5V.c
./src/H5Vprivate.h
./src/H5detect.c
./src/h5ls.c
./test/cmpd_dset.c
./test/dsets.c
./test/external.c
./test/hyperslab.c
./test/iopipe.c
./test/istore.c
./test/shtype.c
./test/tstab.c
Fixed comparisons between signed and unsigned values. Fixed
warnings about unused function arguments.
----------------------
./MANIFEST
./src/Makefile.in
./test/Makefile.in
Added new files.
./config/linux
./src/H5HL.c
./src/H5HLprivate.h
./src/H5MF.c
./src/H5MFprivate.h
Added `-DH5HL_DEBUG -DH5MF_DEBUG' to the debug list.
./html/H5.format.html
Updated shared object message information.
./src/H5D.c
Datasets can now share data types.
./src/H5F.c
Updated a comment that referred to H5ACC_WRITE.
./src/H5HG.c
./src/H5HGprivate.h
Moved a few things around. Made debugging better so you can
now give a collection address to ./src/debug and it shows some
useful stuff.
./src/H5O.c
./src/H5Ocont.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Olayout.c
./src/H5Oname.c
./src/H5Onull.c
./src/H5Oprivate.h
./src/H5Osdspace.c
./src/H5Oshared.c [NEW]
./src/H5Ostab.c
Supports shared messages.
./src/H5T.c
./src/H5Tpkg.h
./src/H5Tprivate.h
./src/H5Tpublic.h
The H5Tshare() function allows the user to give the library
hints about how a data type will be used.
./test/shtype.c
Tests the H5Tshare() function.
./test/gheap.c
Tests the global heap.
./configure.in
./config/BlankForm [NEW]
./config/alpha-dec
./config/freebsd2.2.1
./config/hpux10.20
./config/irix6.2
./config/irix64
./config/linux
./config/powerpc-ibm-aix4.2.1.0
./config/rs6000-ibm-aix4.1.4.0
./config/solaris2.5
Cleaned up lots of configuration stuff and made the site
configuration files lots easier and more uniform. To make a
new file grab the BlankForm and modify it.
By default, debugging is turned on for most packages. Within
a package one can use `#ifdef H5AC_DEBUG' to wrap debugging
code. Other options are:
--enable-debug
--enable-debug=yes
The default, most but not all packages.
--disable-debug
--enable-debug=no
--enable-debug=none
The symbol NDEBUG is defined and none of the package
debug symbols.
--enable-debug=all
Debugging is turned on for all packages. This might
produce lots of output.
--enable-debug=g,d
Debugging is turned on for H5G and H5D.
A compile mode is also now supported
--enable-production
--enable-production=yes
The library is compiled with optimizations turned on.
The compiler flags are augmented by adding PROD_CFLAGS
and PROD_CPPFLAGS which are defined in the site config
file.
--disable-production
--enable-production=no
The default. The library is compiled for development
by including DEBUG_CFLAGS and DEBUG_CPPFLAGS defined in
the site config file. This is usually just `-g'.
--enable-production=profile
--enable-production=pg
Builds a library for profiling by including the flags
from PROFILE_CFLAGS and PROFILE_CPPFLAGS defined in the
site config file. This is usullay just `-pg' but it
could include optimization flags as well depending on
the type of profile one wants.
In summary, configure by saying `./configure' and you'll get a
development version of the library. Configure by saying
`./configure --enable-production --disable-debug' and you'll
get a production version with no debugging code.
----------------------
./MANIFEST
./src/H5HG.c [new]
./src/H5HGprivate.h [new]
./src/H5HGpublic.h [new]
./src/Makefile.in
Added global heaps, H5HG. This compiles but is untested.
./src/H5D.c
./src/H5G.c
./src/H5Gnode.c
./src/H5Gstab.c
./src/H5HL.c [renamed from H5H.c]
./src/H5HLprivate.h [renamed from H5Hprivate.h]
./src/H5HLpublic.h [renamed from H5Hpublic.h]
./src/H5Oefl.c
./src/Makefile.in
./src/hdf5.h
./test/theap.c
Renamed local heaps from H5H to H5HL.
./src/H5AC.c
./src/H5ACprivate.h
./src/debug.c
Added debugging support for global heap collections.
./html/H5.format.html
Ripped out the old small object heap documentation and added
documentation for the new global heap.
./html/H5.api.html
./src/H5P.c
./src/H5Ppublic.h
./src/H5F.c
./src/H5Fprivate.h
./src/H5private.h
The H5Pget_version() function no longer returns a small object
heap version number. Instead, each collection of the global
heap has its own version number instead of a single number in
the file boot block.
./bin/release
Omit files marked for removal by CVS even though the removal
hasn't been committed yet.
./test/iopipe.c
Removed warnings about casting unsigned int to double on Irix64.
./MANIFEST
Removed ./test/testpar/phdf5sup.c
