mirror/hdf5
2
0
Fork 0
mirror of https://github.com/HDFGroup/hdf5.git synced 2025-03-19 16:50:46 +08:00
Code Issues Packages Projects Releases Wiki Activity

[svn-r10847] Purpose: Bug fix

Browse Source 
Description: See details from Bug #213.  Family member file size wasn't saved
anywhere in file.  When family file is opened, the first member size determine
the member size.

Solution: This is the fourth step of checkin.  A test suit is added for h5repart,
including a program to generate the test files, a script file to run h5repart,
and a program to verify repartitioned files can be opened by the library.

There's a change from the first step of checkin.  Family name template is no
longer saved in the superblock because different pathname can make the name
different.

In the third step of checkin, h5repart has been modified.  If h5repart is used
to change the size of family member file, the new size(actual member size) is saved
in the superblock.

In the second step of checkin, multi driver is checked against the driver
name saved in superblock.  Wrong driver will result in a failure with an error message indicating
multi driver should be used.  This change includes split driver because it's a special case for multi
driver.

In the first step of checkin.  Family member size and name template(unused at this stage) are saved
in file superblock.  When file is reopened,the size passed in thrin superblock.  A different size
will trigger a failure with an error message indicating the right size.  Wrong driver to open family
file will cause a failure, too.


Platforms tested: h5committest and fuss.

Misc. update:  MANIFEST
This commit is contained in:
Raymond Lu 2005-06-02 16:00:23 -05:00
parent c02d9c7c0d
commit 7696942940
34 changed files with 496 additions and 64 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
MANIFEST
View File

@ -1248,8 +1248,29 @@
./tools/misc/h5import.c
./tools/misc/h5redeploy.in
./tools/misc/h5repart.c
./tools/misc/h5repart_gentest.c
./tools/misc/pdb2hdf.c
./tools/misc/repart_test.c
./tools/misc/testh5repart.sh.in
./tools/testfiles/family_file00000.h5
./tools/testfiles/family_file00001.h5
./tools/testfiles/family_file00002.h5
./tools/testfiles/family_file00003.h5
./tools/testfiles/family_file00004.h5
./tools/testfiles/family_file00005.h5
./tools/testfiles/family_file00006.h5
./tools/testfiles/family_file00007.h5
./tools/testfiles/family_file00008.h5
./tools/testfiles/family_file00009.h5
./tools/testfiles/family_file00010.h5
./tools/testfiles/family_file00011.h5
./tools/testfiles/family_file00012.h5
./tools/testfiles/family_file00013.h5
./tools/testfiles/family_file00014.h5
./tools/testfiles/family_file00015.h5
./tools/testfiles/family_file00016.h5
./tools/testfiles/family_file00017.h5
./tools/testfiles/tall-1.ddl
./tools/testfiles/tall-2.ddl
./tools/testfiles/tall-3.ddl

3
configure vendored
View File

@ -50594,7 +50594,7 @@ if test -n "$TESTPARALLEL"; then
fi
fi
ac_config_files="$ac_config_files src/libhdf5.settings Makefile src/Makefile test/Makefile testpar/Makefile testpar/testph5.sh perform/Makefile tools/Makefile tools/h5dump/Makefile tools/h5dump/testh5dump.sh tools/h5import/Makefile tools/h5diff/Makefile tools/h5jam/Makefile tools/h5jam/testh5jam.sh tools/h5repack/Makefile tools/h5repack/h5repack.sh tools/h5ls/Makefile tools/lib/Makefile tools/misc/Makefile tools/misc/h5cc tools/gifconv/Makefile examples/Makefile doc/Makefile doc/html/Makefile doc/html/ed_libs/Makefile doc/html/ed_styles/Makefile doc/html/ADGuide/Makefile doc/html/Graphics/Makefile doc/html/Intro/Makefile doc/html/PSandPDF/Makefile doc/html/TechNotes/Makefile doc/html/Tutor/Makefile doc/html/Tutor/Graphics/Makefile doc/html/Tutor/examples/Makefile doc/html/cpplus/Makefile doc/html/fortran/Makefile c++/Makefile c++/src/Makefile c++/src/h5c++ c++/test/Makefile c++/examples/Makefile fortran/Makefile fortran/src/h5fc fortran/src/libhdf5_fortran.settings fortran/src/Makefile fortran/test/Makefile fortran/testpar/Makefile fortran/examples/Makefile hl/Makefile hl/src/Makefile hl/test/Makefile hl/c++/Makefile hl/c++/src/Makefile hl/c++/test/Makefile hl/fortran/Makefile hl/fortran/src/Makefile hl/fortran/test/Makefile"
ac_config_files="$ac_config_files src/libhdf5.settings Makefile src/Makefile test/Makefile testpar/Makefile testpar/testph5.sh perform/Makefile tools/Makefile tools/h5dump/Makefile tools/h5dump/testh5dump.sh tools/h5import/Makefile tools/h5diff/Makefile tools/h5jam/Makefile tools/h5jam/testh5jam.sh tools/h5repack/Makefile tools/h5repack/h5repack.sh tools/h5ls/Makefile tools/lib/Makefile tools/misc/Makefile tools/misc/h5cc tools/misc/testh5repart.sh tools/gifconv/Makefile examples/Makefile doc/Makefile doc/html/Makefile doc/html/ed_libs/Makefile doc/html/ed_styles/Makefile doc/html/ADGuide/Makefile doc/html/Graphics/Makefile doc/html/Intro/Makefile doc/html/PSandPDF/Makefile doc/html/TechNotes/Makefile doc/html/Tutor/Makefile doc/html/Tutor/Graphics/Makefile doc/html/Tutor/examples/Makefile doc/html/cpplus/Makefile doc/html/fortran/Makefile c++/Makefile c++/src/Makefile c++/src/h5c++ c++/test/Makefile c++/examples/Makefile fortran/Makefile fortran/src/h5fc fortran/src/libhdf5_fortran.settings fortran/src/Makefile fortran/test/Makefile fortran/testpar/Makefile fortran/examples/Makefile hl/Makefile hl/src/Makefile hl/test/Makefile hl/c++/Makefile hl/c++/src/Makefile hl/c++/test/Makefile hl/fortran/Makefile hl/fortran/src/Makefile hl/fortran/test/Makefile"
cat >confcache <<\_ACEOF
@ -51223,6 +51223,7 @@ do
"tools/lib/Makefile" ) CONFIG_FILES="$CONFIG_FILES tools/lib/Makefile" ;;
"tools/misc/Makefile" ) CONFIG_FILES="$CONFIG_FILES tools/misc/Makefile" ;;
"tools/misc/h5cc" ) CONFIG_FILES="$CONFIG_FILES tools/misc/h5cc" ;;
"tools/misc/testh5repart.sh" ) CONFIG_FILES="$CONFIG_FILES tools/misc/testh5repart.sh" ;;
"tools/gifconv/Makefile" ) CONFIG_FILES="$CONFIG_FILES tools/gifconv/Makefile" ;;
"examples/Makefile" ) CONFIG_FILES="$CONFIG_FILES examples/Makefile" ;;
"doc/Makefile" ) CONFIG_FILES="$CONFIG_FILES doc/Makefile" ;;

1
configure.in
View File

@ -2973,6 +2973,7 @@ AC_CONFIG_FILES([src/libhdf5.settings
tools/lib/Makefile
tools/misc/Makefile
tools/misc/h5cc
tools/misc/testh5repart.sh
tools/gifconv/Makefile
examples/Makefile
doc/Makefile

1
src/H5E.c
View File

@ -2112,7 +2112,6 @@ H5E_walk_stack(const H5E_t *estack, H5E_direction_t direction, H5E_walk_t func,
/* Walk the stack if a callback function was given */
if(func) {
status=SUCCEED;
if (H5E_WALK_UPWARD==direction) {
for (i=0; i<(int)estack->nused && status>=0; i++)
status = (func)((unsigned)i, estack->slot+i, client_data);

8
src/H5F.c
View File

@ -212,6 +212,7 @@ H5F_init_interface(void)
void *driver_info = H5F_ACS_FILE_DRV_INFO_DEF;
H5F_close_degree_t close_degree = H5F_CLOSE_DEGREE_DEF;
hsize_t family_offset = H5F_ACS_FAMILY_OFFSET_DEF;
hsize_t family_newsize = H5F_ACS_FAMILY_NEWSIZE_DEF;
H5FD_mem_t mem_type = H5F_ACS_MULTI_TYPE_DEF;
/* File mount property class variable.
@ -361,7 +362,11 @@ H5F_init_interface(void)
/* Register the offset of family driver info */
if(H5P_register(acs_pclass,H5F_ACS_FAMILY_OFFSET_NAME,H5F_ACS_FAMILY_OFFSET_SIZE, &family_offset,NULL,NULL,NULL,NULL,NULL,NULL,NULL)<0)
HGOTO_ERROR(H5E_PLIST, H5E_CANTINSERT, FAIL, "can't insert property into class")
/* Register the private property of new family file size. It's used by h5repart only. */
if(H5P_register(acs_pclass,H5F_ACS_FAMILY_NEWSIZE_NAME,H5F_ACS_FAMILY_NEWSIZE_SIZE, &family_newsize,NULL,NULL,NULL,NULL,NULL,NULL,NULL)<0)
HGOTO_ERROR(H5E_PLIST, H5E_CANTINSERT, FAIL, "can't insert property into class")
/* Register the data type of multi driver info */
if(H5P_register(acs_pclass,H5F_ACS_MULTI_TYPE_NAME,H5F_ACS_MULTI_TYPE_SIZE, &mem_type,NULL,NULL,NULL,NULL,NULL,NULL,NULL)<0)
HGOTO_ERROR(H5E_PLIST, H5E_CANTINSERT, FAIL, "can't insert property into class")
@ -2630,7 +2635,6 @@ H5F_read_superblock(H5F_t *f, hid_t dxpl_id, H5G_entry_t *root_ent, haddr_t addr
if (H5FD_sb_decode(lf, driver_name, p) < 0)
HGOTO_ERROR(H5E_FILE, H5E_CANTOPENFILE, FAIL, "unable to decode driver information")
p += driver_size; /* advance past driver information section */
/* Compute driver info block checksum */
assert(sizeof(chksum) == sizeof(shared->drvr_chksum));

45
src/H5FDfamily.c
View File

@ -60,6 +60,8 @@ typedef struct H5FD_family_t {
hid_t memb_fapl_id; /*file access property list for members */
hsize_t memb_size; /*actual size of each member file */
hsize_t pmem_size; /*member size passed in from property */
hsize_t mem_newsize; /*new member size passed in as private property.
*It's used only by h5repart */
unsigned nmembs; /*number of family members */
unsigned amembs; /*number of member slots allocated */
H5FD_t **memb; /*dynamic array of member pointers */
@ -584,9 +586,6 @@ H5FD_family_sb_size(H5FD_t *_file)
* enough for now. */
ret_value += 8;
/* name template, NULL termination included */
ret_value += HDstrlen(file->name) + 1;
done:
FUNC_LEAVE_NOAPI(ret_value)
}
@ -627,12 +626,8 @@ H5FD_family_sb_encode(H5FD_t *_file, char *name/*out*/,
name[8] = '\0';
/* copy member file size */
UINT64ENCODE(buf, file->memb_size);
p += 8;
UINT64ENCODE(p, file->memb_size);
/* copy name template */
HDmemcpy(p, file->name, HDstrlen(file->name) + 1);
done:
FUNC_LEAVE_NOAPI(ret_value)
}
@ -641,12 +636,11 @@ done:
/*-------------------------------------------------------------------------
* Function: H5FD_family_sb_decode
*
* Purpose: Decodes the superblock information for this driver. The NAME
* argument is the eight-character (plus null termination) name
* stored in the file.
*
* The FILE argument is updated according to the information in
* the superblock.
* Purpose: This function has 2 seperate purpose. One is to decodes the
* superblock information for this driver. The NAME argument is
* the eight-character (plus null termination) name stored in i
* the file. The FILE argument is updated according to the
* information in the superblock.
*
* Return: Success: 0
*
@ -663,7 +657,6 @@ static herr_t
H5FD_family_sb_decode(H5FD_t *_file, const char *name, const unsigned char *buf)
{
H5FD_family_t *file = (H5FD_family_t*)_file;
unsigned char *p = buf;
uint64_t msize;
char err_msg[128];
herr_t ret_value=SUCCEED; /* Return value */
@ -671,13 +664,14 @@ H5FD_family_sb_decode(H5FD_t *_file, const char *name, const unsigned char *buf)
FUNC_ENTER_NOAPI(H5FD_family_sb_decode, FAIL)
/* Read member file size. Skip name template for now although it's saved. */
UINT64DECODE(p, msize);
UINT64DECODE(buf, msize);
/* For h5repart only. Member size 1 is used to signal h5repart is being used to
* change member file size. Encode the new size. */
if(file->pmem_size == 1) {
msize = file->memb_size;
UINT64ENCODE(p, msize);
/* For h5repart only. Private property of new member size is used to signal
* h5repart is being used to change member file size. h5repart will open
* files for read and write. When the files are closed, metadata will be
* flushed to the files and updated this new size */
if(file->mem_newsize) {
file->memb_size = file->mem_newsize;
HGOTO_DONE(ret_value)
}
@ -742,6 +736,7 @@ H5FD_family_open(const char *name, unsigned flags, hid_t fapl_id,
H5FD_t *ret_value=NULL;
char memb_name[4096], temp[4096];
hsize_t eof=HADDR_UNDEF;
hsize_t fam_newsize = 0;
unsigned t_flags = flags & ~H5F_ACC_CREAT;
H5P_genplist_t *plist; /* Property list pointer */
@ -762,12 +757,19 @@ H5FD_family_open(const char *name, unsigned flags, hid_t fapl_id,
HGOTO_ERROR(H5E_VFL, H5E_CANTINC, NULL, "unable to increment ref count on VFL driver")
file->memb_size = 1024*1024*1024; /*1GB. Actual member size to be updated later */
file->pmem_size = 1024*1024*1024; /*1GB. Member size passed in through property */
file->mem_newsize = 0; /*New member size used by h5repart only */
} else {
H5FD_family_fapl_t *fa;
if(NULL == (plist = H5I_object(fapl_id)))
HGOTO_ERROR(H5E_ARGS, H5E_BADTYPE, NULL, "not a file access property list")
fa = H5P_get_driver_info(plist);
/* New family file size. It's used by h5repart only. */
if(H5P_exist_plist(plist, H5F_ACS_FAMILY_NEWSIZE_NAME) > 0)
if(H5P_get(plist, H5F_ACS_FAMILY_NEWSIZE_NAME, &fam_newsize) < 0)
HGOTO_ERROR(H5E_PLIST, H5E_CANTGET, NULL, "can't get new family member size")
if(fa->memb_fapl_id==H5P_FILE_ACCESS_DEFAULT) {
if(H5I_inc_ref(fa->memb_fapl_id)<0)
HGOTO_ERROR(H5E_VFL, H5E_CANTINC, NULL, "unable to increment ref count on VFL driver")
@ -780,6 +782,7 @@ H5FD_family_open(const char *name, unsigned flags, hid_t fapl_id,
} /* end else */
file->memb_size = fa->memb_size; /* Actual member size to be updated later */
file->pmem_size = fa->memb_size; /* Member size passed in through property */
file->mem_newsize = fam_newsize; /* New member size passed in through property */
}
file->name = H5MM_strdup(name);
file->flags = flags;

2
src/H5FLprivate.h
View File

@ -35,7 +35,7 @@
/* Private headers needed by this file */
/* Macros for turning off free lists in the library */
/* #define H5_NO_FREE_LISTS */
#define H5_NO_FREE_LISTS
#if defined H5_NO_FREE_LISTS || defined H5_USING_PURIFY
#define H5_NO_REG_FREE_LISTS
#define H5_NO_ARR_FREE_LISTS

6
src/H5Fprivate.h
View File

@ -392,6 +392,12 @@ typedef struct H5F_t H5F_t;
#define H5F_ACS_FAMILY_OFFSET_SIZE sizeof(hsize_t)
#define H5F_ACS_FAMILY_OFFSET_DEF 0
/* Definition for new member size of family driver. It's private
* property only used by h5repart */
#define H5F_ACS_FAMILY_NEWSIZE_NAME "family_newsize"
#define H5F_ACS_FAMILY_NEWSIZE_SIZE sizeof(hsize_t)
#define H5F_ACS_FAMILY_NEWSIZE_DEF 0
/* Definition for data type in multi file driver */
#define H5F_ACS_MULTI_TYPE_NAME "multi_type"
#define H5F_ACS_MULTI_TYPE_SIZE sizeof(H5FD_mem_t)

2
src/H5Pfapl.c
View File

@ -620,7 +620,7 @@ H5P_get_family_offset(H5P_genplist_t *plist, hsize_t *offset)
done:
FUNC_LEAVE_NOAPI(ret_value);
}
}
/*-------------------------------------------------------------------------

19
test/file_handle.c
View File

@ -32,6 +32,8 @@ const char *FILENAME[] = {
"core_file",
"family_file",
"multi_file",
"fst_family",
"scd_family",
NULL
};
@ -231,7 +233,7 @@ test_family_opens(char *fname, hid_t fa_pl)
} H5E_END_TRY;
/* Case 3: reopen file with wrong member size */
if(H5Pset_fapl_family(fa_pl, 128, H5P_DEFAULT)<0)
if(H5Pset_fapl_family(fa_pl, (hsize_t)128, H5P_DEFAULT)<0)
goto error;
H5E_BEGIN_TRY {
@ -247,7 +249,7 @@ test_family_opens(char *fname, hid_t fa_pl)
}
}
if(H5Pset_fapl_family(fa_pl, FAMILY_SIZE, H5P_DEFAULT)<0)
if(H5Pset_fapl_family(fa_pl, (hsize_t)FAMILY_SIZE, H5P_DEFAULT)<0)
goto error;
H5E_BEGIN_TRY {
@ -278,6 +280,11 @@ error:
* Wednesday, June 23, 2004
* Added test for H5Fget_filesize.
*
* Raymond Lu
* June 2, 2005
* Added a function test_family_opens() to test different
* wrong way to reopen family files.
*
*-------------------------------------------------------------------------
*/
static herr_t
@ -297,6 +304,7 @@ test_family(void)
/* Set property list and file name for FAMILY driver */
fapl = h5_fileaccess();
if(H5Pset_fapl_family(fapl, (hsize_t)FAMILY_SIZE, H5P_DEFAULT)<0)
goto error;
h5_fixname(FILENAME[2], fapl, filename, sizeof filename);
@ -313,7 +321,7 @@ test_family(void)
goto error;
/* Reopen the file with default member file size */
if(H5Pset_fapl_family(fapl, H5F_FAMILY_DEFAULT, H5P_DEFAULT)<0)
if(H5Pset_fapl_family(fapl, (hsize_t)H5F_FAMILY_DEFAULT, H5P_DEFAULT)<0)
goto error;
if((file=H5Fopen(filename, H5F_ACC_RDWR, fapl))<0)
@ -395,7 +403,7 @@ test_family(void)
goto error;
/* Reopen the file with correct member file size. */
if(H5Pset_fapl_family(fapl, FAMILY_SIZE, H5P_DEFAULT)<0)
if(H5Pset_fapl_family(fapl, (hsize_t)FAMILY_SIZE, H5P_DEFAULT)<0)
goto error;
if((file=H5Fopen(filename, H5F_ACC_RDWR, fapl))<0)
@ -413,7 +421,6 @@ error:
H5Sclose(space);
H5Dclose(dset);
H5Pclose (fapl2);
H5Pclose (fapl2);
H5Fclose(file);
} H5E_END_TRY;
return -1;
@ -452,8 +459,6 @@ test_multi_opens(char *fname, hid_t fa_pl)
} H5E_END_TRY;
return 0;
error:
return -1;
}

18
tools/misc/Makefile.am
View File

@ -22,20 +22,32 @@ include $(top_srcdir)/config/commence.am
# Include src directory
INCLUDES=-I$(top_srcdir)/src
#test script and program
TEST_PROG=h5repart_gentest
TEST_SCRIPT=testh5repart.sh
check_PROGRAMS=$(TEST_PROG)
check_SCRIPTS=$(TEST_SCRIPT)
# Build pdb2hdf5 if LLNL's PDB is present (checked in configure)
if BUILD_PDB2HDF_CONDITIONAL
PDB2HDF=pdb2hdf5
endif
# These are our main targets, the tools
bin_PROGRAMS=h5debug h5repart $(PDB2HDF)
bin_PROGRAMS=h5debug h5repart $(PDB2HDF) repart_test
bin_SCRIPTS=h5redeploy
# Tell automake to clean h5redeploy script
CLEANFILES=h5redeploy
# h5cc is generated by configure. Remove it only when distclean.
DISTCLEANFILES=h5cc
# Temporary files. *.h5 are generated by h5repart_gentest. They should
# copied to the testfiles/ directory if update is required. fst_family*.h5
# and scd_family*.h5 were created by setting the HDF5_NOCLEANUP variable.
MOSTLYCLEANFILES=*.h5 ../testfiles/fst_family*.h5 ../testfiles/scd_family*.h5
# These were generated by configure. Remove them only when distclean.
DISTCLEANFILES=h5cc testh5repart.sh
# All programs rely on hdf5 library and h5tools library
LDADD=$(LIBHDF5) $(LIBH5TOOLS)

88
tools/misc/Makefile.in
View File

@ -31,7 +31,7 @@
#
SOURCES = h5debug.c h5repart.c pdb2hdf5.c
SOURCES = h5debug.c h5repart.c h5repart_gentest.c pdb2hdf5.c repart_test.c
srcdir = @srcdir@
top_srcdir = @top_srcdir@
@ -56,9 +56,12 @@ POST_UNINSTALL = :
build_triplet = @build@
host_triplet = @host@
DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in \
$(srcdir)/h5cc.in $(top_srcdir)/config/commence.am \
$(srcdir)/h5cc.in $(srcdir)/testh5repart.sh.in \
$(top_srcdir)/config/commence.am \
$(top_srcdir)/config/conclude.am
bin_PROGRAMS = h5debug$(EXEEXT) h5repart$(EXEEXT) $(am__EXEEXT_1)
check_PROGRAMS = $(am__EXEEXT_2)
bin_PROGRAMS = h5debug$(EXEEXT) h5repart$(EXEEXT) $(am__EXEEXT_1) \
repart_test$(EXEEXT)
subdir = tools/misc
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
am__aclocal_m4_deps = $(top_srcdir)/configure.in
@ -66,10 +69,11 @@ am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \
$(ACLOCAL_M4)
mkinstalldirs = $(SHELL) $(top_srcdir)/bin/mkinstalldirs
CONFIG_HEADER = $(top_builddir)/src/H5config.h
CONFIG_CLEAN_FILES = h5cc
CONFIG_CLEAN_FILES = h5cc testh5repart.sh
@BUILD_PDB2HDF_CONDITIONAL_TRUE@am__EXEEXT_1 = pdb2hdf5$(EXEEXT)
am__installdirs = "$(DESTDIR)$(bindir)" "$(DESTDIR)$(bindir)"
binPROGRAMS_INSTALL = $(INSTALL_PROGRAM)
am__EXEEXT_2 = h5repart_gentest$(EXEEXT)
PROGRAMS = $(bin_PROGRAMS)
h5debug_SOURCES = h5debug.c
h5debug_OBJECTS = h5debug.$(OBJEXT)
@ -81,10 +85,19 @@ h5repart_SOURCES = h5repart.c
h5repart_OBJECTS = h5repart.$(OBJEXT)
h5repart_LDADD = $(LDADD)
h5repart_DEPENDENCIES = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_2)
h5repart_gentest_SOURCES = h5repart_gentest.c
h5repart_gentest_OBJECTS = h5repart_gentest.$(OBJEXT)
h5repart_gentest_LDADD = $(LDADD)
h5repart_gentest_DEPENDENCIES = $(am__DEPENDENCIES_1) \
$(am__DEPENDENCIES_2)
pdb2hdf5_SOURCES = pdb2hdf5.c
pdb2hdf5_OBJECTS = pdb2hdf5.$(OBJEXT)
pdb2hdf5_LDADD = $(LDADD)
pdb2hdf5_DEPENDENCIES = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_2)
repart_test_SOURCES = repart_test.c
repart_test_OBJECTS = repart_test.$(OBJEXT)
repart_test_LDADD = $(LDADD)
repart_test_DEPENDENCIES = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_2)
binSCRIPT_INSTALL = $(INSTALL_SCRIPT)
SCRIPTS = $(bin_SCRIPTS)
DEFAULT_INCLUDES = -I. -I$(srcdir) -I$(top_builddir)/src
@ -98,8 +111,10 @@ LTCOMPILE = $(LIBTOOL) --tag=CC --mode=compile $(CC) $(DEFS) \
CCLD = $(CC)
LINK = $(LIBTOOL) --tag=CC --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \
$(AM_LDFLAGS) $(LDFLAGS) -o $@
SOURCES = h5debug.c h5repart.c pdb2hdf5.c
DIST_SOURCES = h5debug.c h5repart.c pdb2hdf5.c
SOURCES = h5debug.c h5repart.c h5repart_gentest.c pdb2hdf5.c \
repart_test.c
DIST_SOURCES = h5debug.c h5repart.c h5repart_gentest.c pdb2hdf5.c \
repart_test.c
ETAGS = etags
CTAGS = ctags
DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST)
@ -313,6 +328,11 @@ H5FC_PP = $(bindir)/h5pfc
# Include src directory
INCLUDES = -I$(top_srcdir)/src
#test script and program
TEST_PROG = h5repart_gentest
TEST_SCRIPT = testh5repart.sh
check_SCRIPTS = $(TEST_SCRIPT)
# Build pdb2hdf5 if LLNL's PDB is present (checked in configure)
@BUILD_PDB2HDF_CONDITIONAL_TRUE@PDB2HDF = pdb2hdf5
bin_SCRIPTS = h5redeploy
@ -320,8 +340,13 @@ bin_SCRIPTS = h5redeploy
# Tell automake to clean h5redeploy script
CLEANFILES = h5redeploy
# h5cc is generated by configure. Remove it only when distclean.
DISTCLEANFILES = h5cc
# Temporary files. *.h5 are generated by h5repart_gentest. They should
# copied to the testfiles/ directory if update is required. fst_family*.h5
# and scd_family*.h5 were created by setting the HDF5_NOCLEANUP variable.
MOSTLYCLEANFILES = *.h5 ../testfiles/fst_family*.h5 ../testfiles/scd_family*.h5
# These were generated by configure. Remove them only when distclean.
DISTCLEANFILES = h5cc testh5repart.sh
# All programs rely on hdf5 library and h5tools library
LDADD = $(LIBHDF5) $(LIBH5TOOLS)
@ -378,6 +403,8 @@ $(ACLOCAL_M4): @MAINTAINER_MODE_TRUE@ $(am__aclocal_m4_deps)
cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh
h5cc: $(top_builddir)/config.status $(srcdir)/h5cc.in
cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@
testh5repart.sh: $(top_builddir)/config.status $(srcdir)/testh5repart.sh.in
cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@
install-binPROGRAMS: $(bin_PROGRAMS)
@$(NORMAL_INSTALL)
test -z "$(bindir)" || $(mkdir_p) "$(DESTDIR)$(bindir)"
@ -406,15 +433,28 @@ clean-binPROGRAMS:
echo " rm -f $$p $$f"; \
rm -f $$p $$f ; \
done
clean-checkPROGRAMS:
@list='$(check_PROGRAMS)'; for p in $$list; do \
f=`echo $$p|sed 's/$(EXEEXT)$$//'`; \
echo " rm -f $$p $$f"; \
rm -f $$p $$f ; \
done
h5debug$(EXEEXT): $(h5debug_OBJECTS) $(h5debug_DEPENDENCIES)
@rm -f h5debug$(EXEEXT)
$(LINK) $(h5debug_LDFLAGS) $(h5debug_OBJECTS) $(h5debug_LDADD) $(LIBS)
h5repart$(EXEEXT): $(h5repart_OBJECTS) $(h5repart_DEPENDENCIES)
@rm -f h5repart$(EXEEXT)
$(LINK) $(h5repart_LDFLAGS) $(h5repart_OBJECTS) $(h5repart_LDADD) $(LIBS)
h5repart_gentest$(EXEEXT): $(h5repart_gentest_OBJECTS) $(h5repart_gentest_DEPENDENCIES)
@rm -f h5repart_gentest$(EXEEXT)
$(LINK) $(h5repart_gentest_LDFLAGS) $(h5repart_gentest_OBJECTS) $(h5repart_gentest_LDADD) $(LIBS)
pdb2hdf5$(EXEEXT): $(pdb2hdf5_OBJECTS) $(pdb2hdf5_DEPENDENCIES)
@rm -f pdb2hdf5$(EXEEXT)
$(LINK) $(pdb2hdf5_LDFLAGS) $(pdb2hdf5_OBJECTS) $(pdb2hdf5_LDADD) $(LIBS)
repart_test$(EXEEXT): $(repart_test_OBJECTS) $(repart_test_DEPENDENCIES)
@rm -f repart_test$(EXEEXT)
$(LINK) $(repart_test_LDFLAGS) $(repart_test_OBJECTS) $(repart_test_LDADD) $(LIBS)
install-binSCRIPTS: $(bin_SCRIPTS)
@$(NORMAL_INSTALL)
test -z "$(bindir)" || $(mkdir_p) "$(DESTDIR)$(bindir)"
@ -443,7 +483,9 @@ distclean-compile:
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/h5debug.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/h5repart.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/h5repart_gentest.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pdb2hdf5.Po@am__quote@
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/repart_test.Po@am__quote@
.c.o:
@am__fastdepCC_TRUE@ if $(COMPILE) -MT $@ -MD -MP -MF "$(DEPDIR)/$*.Tpo" -c -o $@ $<; \
@ -553,6 +595,7 @@ distdir: $(DISTFILES)
fi; \
done
check-am: all-am
$(MAKE) $(AM_MAKEFLAGS) $(check_PROGRAMS) $(check_SCRIPTS)
$(MAKE) $(AM_MAKEFLAGS) check-TESTS
check: check-am
all-am: Makefile $(PROGRAMS) $(SCRIPTS) all-local
@ -575,6 +618,7 @@ install-strip:
`test -z '$(STRIP)' || \
echo "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'"` install
mostlyclean-generic:
-test -z "$(MOSTLYCLEANFILES)" || rm -f $(MOSTLYCLEANFILES)
clean-generic:
-test -z "$(CLEANFILES)" || rm -f $(CLEANFILES)
@ -588,7 +632,8 @@ maintainer-clean-generic:
@echo "it deletes files that may require special tools to rebuild."
clean: clean-am
clean-am: clean-binPROGRAMS clean-generic clean-libtool mostlyclean-am
clean-am: clean-binPROGRAMS clean-checkPROGRAMS clean-generic \
clean-libtool mostlyclean-am
distclean: distclean-am
-rm -rf ./$(DEPDIR)
@ -639,18 +684,19 @@ uninstall-am: uninstall-binPROGRAMS uninstall-binSCRIPTS \
uninstall-info-am uninstall-local
.PHONY: CTAGS GTAGS all all-am all-local check check-TESTS check-am \
clean clean-binPROGRAMS clean-generic clean-libtool ctags \
distclean distclean-compile distclean-generic \
distclean-libtool distclean-tags distdir dvi dvi-am html \
html-am info info-am install install-am install-binPROGRAMS \
install-binSCRIPTS install-data install-data-am install-exec \
install-exec-am install-exec-local install-info \
install-info-am install-man install-strip installcheck \
installcheck-am installdirs maintainer-clean \
maintainer-clean-generic mostlyclean mostlyclean-compile \
mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \
tags uninstall uninstall-am uninstall-binPROGRAMS \
uninstall-binSCRIPTS uninstall-info-am uninstall-local
clean clean-binPROGRAMS clean-checkPROGRAMS clean-generic \
clean-libtool ctags distclean distclean-compile \
distclean-generic distclean-libtool distclean-tags distdir dvi \
dvi-am html html-am info info-am install install-am \
install-binPROGRAMS install-binSCRIPTS install-data \
install-data-am install-exec install-exec-am \
install-exec-local install-info install-info-am install-man \
install-strip installcheck installcheck-am installdirs \
maintainer-clean maintainer-clean-generic mostlyclean \
mostlyclean-compile mostlyclean-generic mostlyclean-libtool \
pdf pdf-am ps ps-am tags uninstall uninstall-am \
uninstall-binPROGRAMS uninstall-binSCRIPTS uninstall-info-am \
uninstall-local
install-exec-local:

23
tools/misc/h5repart.c
View File

@ -65,6 +65,10 @@
# define MIN3(X,Y,Z) MIN(MIN(X,Y),Z)
#endif
/*Make this private property(defined in H5Fprivate.h) available to h5repart,
*to update the member file size in the superblock.*/
#define H5F_ACS_FAMILY_NEWSIZE_NAME "family_newsize"
/*-------------------------------------------------------------------------
* Function: usage
@ -223,7 +227,7 @@ main (int argc, char *argv[])
off_t src_act_size; /*source actual member size */
off_t dst_size=1 GB; /*destination logical memb size */
#endif
hid_t fapl;
hid_t fapl; /*file access property list */
hid_t file;
/*
@ -444,19 +448,26 @@ main (int argc, char *argv[])
}
close (dst);
/* modify family size saved in superblock. Member size 1 signals library to
* save the new size(actual member file size) in superblock. It's for this
* tool only. */
/* Modify family size saved in superblock through private property. It signals
* library to save the new member size(specified in command line) in superblock.
* This private property is for this tool only. */
if ((fapl=H5Pcreate(H5P_FILE_ACCESS))<0) {
perror ("H5Pcreate");
exit (1);
}
if(H5Pset_fapl_family(fapl, 1, H5P_DEFAULT)<0) {
if(H5Pset_fapl_family(fapl, H5F_FAMILY_DEFAULT, H5P_DEFAULT) < 0) {
perror ("H5Pset_fapl_family");
exit (1);
}
if(H5Pset(fapl, H5F_ACS_FAMILY_NEWSIZE_NAME, &dst_size) < 0) {
perror ("H5Pset_family_newsize");
exit (1);
}
/* Open file for "read and write" to flush metadata. Flushing metadata
* will update the superblock to the new member size. */
if((file=H5Fopen(dst_gen_name, H5F_ACC_RDWR, fapl))<0) {
perror ("H5Fopen");
exit (1);

99
tools/misc/h5repart_gentest.c Normal file
View File

@ -0,0 +1,99 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the files COPYING and Copyright.html. COPYING can be found at the root *
* of the source code distribution tree; Copyright.html can be found at the *
* root level of an installed copy of the electronic HDF5 document set and *
* is linked from the top-level documents page. It can also be found at *
* http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html. If you do not have *
* access to either file, you may request a copy from hdfhelp@ncsa.uiuc.edu. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
/*
* Programmer: Raymond Lu<slu@ncsa.uiuc.edu>
* June 1, 2005
*
* Purpose: Generate a family file of 1024 bytes for each member
* for h5repart test.
*/
#include "hdf5.h"
#define KB 1024
#define FAMILY_NUMBER 4
#define FAMILY_SIZE 1024
#define FILENAME "family_file%05d.h5"
int main(void)
{
hid_t file=(-1), fapl, fapl2=(-1), space=(-1), dset=(-1);
char dname[]="dataset";
int i, j;
int buf[FAMILY_NUMBER][FAMILY_SIZE];
hsize_t dims[2]={FAMILY_NUMBER, FAMILY_SIZE};
/* Set property list and file name for FAMILY driver */
if ((fapl=H5Pcreate(H5P_FILE_ACCESS))<0) {
perror ("H5Pcreate");
exit (1);
}
if(H5Pset_fapl_family(fapl, (hsize_t)FAMILY_SIZE, H5P_DEFAULT)<0) {
perror ("H5Pset_fapl_family");
exit (1);
}
if((file=H5Fcreate(FILENAME, H5F_ACC_TRUNC, H5P_DEFAULT, fapl))<0) {
perror ("H5Fcreate");
exit (1);
}
/* Create and write dataset */
if((space=H5Screate_simple(2, dims, NULL))<0) {
perror ("H5Screate_simple");
exit (1);
}
if((dset=H5Dcreate(file, dname, H5T_NATIVE_INT, space, H5P_DEFAULT))<0) {
perror ("H5Dcreate");
exit (1);
}
for(i=0; i<FAMILY_NUMBER; i++)
for(j=0; j<FAMILY_SIZE; j++)
buf[i][j] = i*10000+j;
if(H5Dwrite(dset, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, buf)<0) {
perror ("H5Dwrite");
exit (1);
}
if(H5Sclose(space)<0) {
perror ("H5Sclose");
exit (1);
}
if(H5Dclose(dset)<0) {
perror ("H5Dclose");
exit (1);
}
if(H5Pclose(fapl)<0) {
perror ("H5Pclose");
exit (1);
}
if(H5Fclose(file)<0) {
perror ("H5Fclose");
exit (1);
}
puts(" PASSED"); fflush(stdout);
return 0;
}

123
tools/misc/repart_test.c Normal file
View File

@ -0,0 +1,123 @@
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the files COPYING and Copyright.html. COPYING can be found at the root *
* of the source code distribution tree; Copyright.html can be found at the *
* root level of an installed copy of the electronic HDF5 document set and *
* is linked from the top-level documents page. It can also be found at *
* http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html. If you do not have *
* access to either file, you may request a copy from hdfhelp@ncsa.uiuc.edu. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
/*
* Programmer: Raymond Lu <slu@ncsa.uiuc.edu>
* June 1, 2005
*
* Purpose: This program tests family files after being repartitioned
* by h5repart. It simply tries to reopen the files with
* correct family driver and member size.
*/
#include "hdf5.h"
#define KB 1024
#define FAMILY_H5REPART_SIZE1 20000
#define FAMILY_H5REPART_SIZE2 (5*KB)
#define MAX(a,b) (a>b ? a:b)
const char *FILENAME[] = {
"fst_family%05d.h5",
"scd_family%05d.h5",
NULL
};
/*-------------------------------------------------------------------------
* Function: test_family_h5repart_opens
*
* Purpose: Tries to reopen family files.
*
* Return: Success: exit(0)
*
* Failure: exit(1)
*
* Programmer: Raymond Lu
* June 1, 2005
*
* Modifications:
*-------------------------------------------------------------------------
*/
static herr_t
test_family_h5repart_opens(void)
{
hid_t file=(-1), fapl=(-1);
char filename[1024];
/* open 1st file(single member file) with correct family size(20000 byte) */
if ((fapl=H5Pcreate(H5P_FILE_ACCESS))<0)
goto error;
if(H5Pset_fapl_family(fapl, (hsize_t)FAMILY_H5REPART_SIZE1, H5P_DEFAULT)<0)
goto error;
if((file=H5Fopen(FILENAME[0], H5F_ACC_RDWR, fapl))<0)
goto error;
if(H5Fclose(file)<0)
goto error;
/* open 2nd file(multiple member files) with correct family size(5KB) */
if(H5Pset_fapl_family(fapl, (hsize_t)FAMILY_H5REPART_SIZE2, H5P_DEFAULT)<0)
goto error;
if((file=H5Fopen(FILENAME[1], H5F_ACC_RDWR, fapl))<0)
goto error;
if(H5Fclose(file)<0)
goto error;
return 0;
error:
H5E_BEGIN_TRY {
H5Fclose(file);
} H5E_END_TRY;
return -1;
}
/*-------------------------------------------------------------------------
* Function: main
*
* Purpose: Tests h5repart-ed family files
*
* Return: Success: exit(0)
*
* Failure: exit(1)
*
* Programmer: Raymond Lu
* June 1, 2005
*
* Modifications:
*
*-------------------------------------------------------------------------
*/
int
main(void)
{
int nerrors=0;
nerrors += test_family_h5repart_opens()<0 ?1:0;
if (nerrors) goto error;
return 0;
error:
nerrors = MAX(1, nerrors);
printf("***** %d FAMILY FILE TEST%s FAILED! *****\n",
nerrors, 1 == nerrors ? "" : "S");
return 1;
}

101
tools/misc/testh5repart.sh.in Normal file
View File

@ -0,0 +1,101 @@
#! /bin/sh
#
# Copyright by the Board of Trustees of the University of Illinois.
# All rights reserved.
#
# This file is part of HDF5. The full HDF5 copyright notice, including
# terms governing use, modification, and redistribution, is contained in
# the files COPYING and Copyright.html. COPYING can be found at the root
# of the source code distribution tree; Copyright.html can be found at the
# root level of an installed copy of the electronic HDF5 document set and
# is linked from the top-level documents page. It can also be found at
# http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html. If you do not have
# access to either file, you may request a copy from hdfhelp@ncsa.uiuc.edu.
#
# Tests for the h5repart tool
REPART=h5repart # The tool name
REPART_BIN=`pwd`/$REPART # The path of the tool binary
REPARTED_FAM=repart_test # The test name
REPARTED_FAM_BIN=`pwd`/$REPARTED_FAM # The path of the test binary
nerrors=0
verbose=yes
# The build (current) directory might be different than the source directory.
if test -z "$srcdir"; then
srcdir=.
fi
test -d ../testfiles || mkdir ../testfiles
actual_dir=`pwd`/../testfiles
# Print a line-line message left justified in a field of 70 characters
# beginning with the word "Testing".
#
TESTING() {
SPACES=" "
echo "Testing $* $SPACES" | cut -c1-70 | tr -d '\012'
}
# Run a test and print PASS or *FAIL*. If a test fails then increment
# the `nerrors' global variable and (if $verbose is set) display the
# difference between the actual output and the expected output. The
# expected output is given as the first argument to this function and
# the actual output file is calculated by replacing the `.ddl' with
# `.out'. The actual output is not removed if $HDF5_NOCLEANUP has a
# non-zero value.
#
TOOLTEST() {
# Run tool test.
TESTING $REPART $@
(
echo
cd $srcdir/../testfiles
$RUNSERIAL $REPART_BIN $@
)
}
OUTPUTTEST() {
# Run test program.
TESTING $REPARTED_FAM $@
(
cd $actual_dir
$RUNSERIAL $REPARTED_FAM_BIN $@
)
}
# Print a "SKIP" message
SKIP() {
TESTING $REPART $@
echo " -SKIP-"
}
##############################################################################
##############################################################################
### T H E T E S T S ###
##############################################################################
##############################################################################
# repartition family member size to 20,000 bytes.
TOOLTEST -m 20000 family_file%05d.h5 $actual_dir/fst_family%05d.h5
# repartition family member size to 5 KB.
TOOLTEST -m 5k family_file%05d.h5 $actual_dir/scd_family%05d.h5
# test the output files repartitioned above.
OUTPUTTEST
echo
if test $nerrors -eq 0 ; then
echo "All $REPART tests passed."
fi
# Clean up output file
if test -z "$HDF5_NOCLEANUP"; then
cd $actual_dir
rm -f fst_family*.h5 scd_family*.h5
fi
exit $nerrors

BIN
tools/testfiles/family_file00000.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00001.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00002.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00003.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00004.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00005.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00006.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00007.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00008.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00009.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00010.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00011.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00012.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00013.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00014.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00015.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00016.h5 Normal file
View File

Binary file not shown.

BIN
tools/testfiles/family_file00017.h5 Normal file
View File

Binary file not shown.