mirror/hdf5
2
0
Fork 0
mirror of https://github.com/HDFGroup/hdf5.git synced 2025-01-18 15:15:56 +08:00
Code Issues Packages Projects Releases Wiki Activity
0f125bf0fc
hdf5/test/trefstr.c
Quincey Koziol 9a433b99a5 [svn-r6252] Purpose: 
Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial
2003-01-09 12:20:03 -05:00

379 lines
12 KiB
C
Raw Blame History

/****************************************************************************
* NCSA HDF *
* Software Development Group *
* National Center for Supercomputing Applications *
* University of Illinois at Urbana-Champaign *
* 605 E. Springfield, Champaign IL 61820 *
* *
* For conditions of distribution and use, see the accompanying *
* hdf/COPYING file. *
* *
****************************************************************************/
/* $Id$ */
/*
FILE
refstr.c
Test HDF reference counted string routines.
REMARKS
DESIGN
BUGS/LIMITATIONS
EXPORTED ROUTINES
AUTHOR
Quincey Koziol
MODIFICATION HISTORY
12/17/02 - Started coding
*/
#include "testhdf5.h"
#include "H5FLprivate.h"
#include "H5RSprivate.h"
/* Declare extern the PQ free list for the wrapped strings */
H5FL_BLK_EXTERN(str_buf);
/****************************************************************
**
** test_refstr_init(): Test basic H5RS (ref-counted strings) code.
** Initialize data for RS testing
**
****************************************************************/
static void
test_refstr_init(void)
{
} /* end test_refstr_init() */
/****************************************************************
**
** test_refstr_create(): Test basic H5RS (ref-counted strings) code.
** Tests creating and closing ref-counted strings.
**
****************************************************************/
static void
test_refstr_create(void)
{
H5RS_str_t *rs; /* Ref-counted string created */
unsigned count; /* Reference count on string */
herr_t ret; /* Generic return value */
/* Output message about test being performed */
MESSAGE(5, ("Testing Creating & Closing Ref-Counted Strings\n"));
/* Try creating a ref-counted string */
rs=H5RS_create("foo");
CHECK(rs, NULL, "H5RS_create");
/* Get the reference count on the string */
count=H5RS_GET_COUNT(rs);
VERIFY(count, 1, "H5RS_GET_COUNT");
/* Try closing a real ref-counted string */
ret=H5RS_decr(rs);
CHECK(ret, FAIL, "H5RS_decr");
} /* end test_refstr_create() */
/****************************************************************
**
** test_refstr_count(): Test basic H5RS (ref-counted strings) code.
** Tests reference counting on ref-counted strings.
**
****************************************************************/
static void
test_refstr_count(void)
{
H5RS_str_t *rs; /* Ref-counted string created */
unsigned count; /* Reference count on string */
herr_t ret; /* Generic return value */
/* Output message about test being performed */
MESSAGE(5, ("Testing Incrementing & Decrementing Ref-Counted Strings\n"));
/* Try creating a ref-counted string */
rs=H5RS_create("foo");
CHECK(rs, NULL, "H5RS_create");
/* Get the reference count on the string */
count=H5RS_GET_COUNT(rs);
VERIFY(count, 1, "H5RS_GET_COUNT");
/* Increment reference count */
ret=H5RS_incr(rs);
CHECK(ret, FAIL, "H5RS_incr");
/* Get the reference count on the string */
count=H5RS_GET_COUNT(rs);
VERIFY(count, 2, "H5RS_GET_COUNT");
/* Decrement reference count for string */
ret=H5RS_decr(rs);
CHECK(ret, FAIL, "H5RS_decr");
/* Get the reference count on the string */
count=H5RS_GET_COUNT(rs);
VERIFY(count, 1, "H5RS_GET_COUNT");
/* Decrement reference count for string */
ret=H5RS_decr(rs);
CHECK(ret, FAIL, "H5RS_decr");
} /* end test_refstr_count() */
/****************************************************************
**
** test_refstr_dup(): Test basic H5RS (ref-counted strings) code.
** Tests duplicating ref-counted strings.
**
****************************************************************/
static void
test_refstr_dup(void)
{
H5RS_str_t *rs1; /* Ref-counted string created */
H5RS_str_t *rs2; /* Ref-counted string created */
unsigned count; /* Reference count on string */
herr_t ret; /* Generic return value */
/* Output message about test being performed */
MESSAGE(5, ("Testing Duplicating Ref-Counted Strings\n"));
/* Try creating a ref-counted string */
rs1=H5RS_create("foo");
CHECK(rs1, NULL, "H5RS_create");
/* Get the reference count on the string */
count=H5RS_GET_COUNT(rs1);
VERIFY(count, 1, "H5RS_GET_COUNT");
/* Duplicate r-string */
rs2=H5RS_dup(rs1);
CHECK(rs2, NULL, "H5RS_dup");
/* Get the reference count on the strings */
count=H5RS_GET_COUNT(rs1);
VERIFY(count, 2, "H5RS_GET_COUNT");
count=H5RS_GET_COUNT(rs2);
VERIFY(count, 2, "H5RS_GET_COUNT");
/* Decrement reference count for string */
ret=H5RS_decr(rs2);
CHECK(ret, FAIL, "H5RS_decr");
/* Get the reference count on the string */
count=H5RS_GET_COUNT(rs1);
VERIFY(count, 1, "H5RS_GET_COUNT");
/* Decrement reference count for string */
ret=H5RS_decr(rs1);
CHECK(ret, FAIL, "H5RS_decr");
} /* end test_refstr_dup() */
/****************************************************************
**
** test_refstr_cmp(): Test basic H5RS (ref-counted strings) code.
** Tests comparing ref-counted strings.
**
****************************************************************/
static void
test_refstr_cmp(void)
{
H5RS_str_t *rs1; /* Ref-counted string created */
H5RS_str_t *rs2; /* Ref-counted string created */
int cmp; /* Comparison value */
ssize_t len; /* Length of string */
herr_t ret; /* Generic return value */
/* Output message about test being performed */
MESSAGE(5, ("Testing Comparing Ref-Counted Strings\n"));
/* Create first reference counted string */
rs1=H5RS_create("foo");
CHECK(rs1, NULL, "H5RS_create");
/* Create second reference counted string */
rs2=H5RS_create("foo2");
CHECK(rs2, NULL, "H5RS_create");
/* Compare the strings in various ways */
cmp=H5RS_cmp(rs1,rs1);
VERIFY(cmp, 0, "H5RS_cmp");
cmp=H5RS_cmp(rs2,rs2);
VERIFY(cmp, 0, "H5RS_cmp");
cmp=H5RS_cmp(rs1,rs2);
if(cmp>=0) {
num_errs++;
printf("%d: string comparison incorrect!\n",__LINE__);
} /* end if */
/* Check the lengths of the strings also */
len=H5RS_len(rs1);
VERIFY(len, 3, "H5RS_len");
len=H5RS_len(rs2);
VERIFY(len, 4, "H5RS_len");
/* Decrement reference count for strings */
ret=H5RS_decr(rs2);
CHECK(ret, FAIL, "H5RS_decr");
ret=H5RS_decr(rs1);
CHECK(ret, FAIL, "H5RS_decr");
} /* end test_refstr_cmp() */
/****************************************************************
**
** test_refstr_wrap(): Test basic H5RS (ref-counted strings) code.
** Tests wrapping ref-counted strings around existing strings.
**
****************************************************************/
static void
test_refstr_wrap(void)
{
H5RS_str_t *rs; /* Ref-counted string created */
const char *s; /* Pointer to raw string in ref-counted string */
char buf[16]; /* Buffer to wrap */
int cmp; /* Comparison value */
herr_t ret; /* Generic return value */
/* Output message about test being performed */
MESSAGE(5, ("Testing Wrapping Ref-Counted Strings\n"));
/* Initialize buffer */
HDstrcpy(buf,"foo");
/* Wrap ref-counted string around existing buffer */
rs=H5RS_wrap(buf);
CHECK(rs, NULL, "H5RS_wrap");
/* Get pointer to raw string in ref-counted string */
s=H5RS_GET_STR(rs);
CHECK(s, NULL, "H5RS_GET_STR");
VERIFY(s, buf, "wrapping");
cmp=HDstrcmp(s,buf);
VERIFY(cmp, 0, "HDstrcmp");
/* Increment reference count (should duplicate string) */
ret=H5RS_incr(rs);
CHECK(ret, FAIL, "H5RS_incr");
/* Change the buffer initially wrapped */
buf[0]='F';
/* Get pointer to raw string in ref-counted string */
s=H5RS_GET_STR(rs);
CHECK(s, NULL, "H5RS_GET_STR");
CHECK(s, buf, "wrapping");
cmp=HDstrcmp(s,buf);
if(cmp<=0) {
num_errs++;
printf("%d: string comparison incorrect!\n",__LINE__);
} /* end if */
/* Decrement reference count for string */
ret=H5RS_decr(rs);
CHECK(ret, FAIL, "H5RS_decr");
ret=H5RS_decr(rs);
CHECK(ret, FAIL, "H5RS_decr");
} /* end test_refstr_wrap() */
/****************************************************************
**
** test_refstr_own(): Test basic H5RS (ref-counted strings) code.
** Tests transferring ownership of dynamically allocated strings
** to ref-counted strings.
**
****************************************************************/
static void
test_refstr_own(void)
{
H5RS_str_t *rs; /* Ref-counted string created */
char *s; /* Pointer to string to transfer */
const char *t; /* Temporary pointers to string */
int cmp; /* Comparison value */
herr_t ret; /* Generic return value */
/* Output message about test being performed */
MESSAGE(5, ("Testing Transferring Ref-Counted Strings\n"));
/* Initialize buffer */
s = H5FL_BLK_MALLOC(str_buf,HDstrlen("foo") + 1);
HDstrcpy(s, "foo");
/* Transfer ownership of dynamically allocated string to ref-counted string */
rs=H5RS_own(s);
CHECK(rs, NULL, "H5RS_own");
/* Get pointer to raw string in ref-counted string */
t=H5RS_GET_STR(rs);
CHECK(t, NULL, "H5RS_GET_STR");
VERIFY(t, s, "transferring");
cmp=HDstrcmp(s,t);
VERIFY(cmp, 0, "HDstrcmp");
/* Increment reference count (should NOT duplicate string) */
ret=H5RS_incr(rs);
CHECK(ret, FAIL, "H5RS_incr");
/* Change the buffer initially wrapped */
*s='F';
/* Get pointer to raw string in ref-counted string */
t=H5RS_GET_STR(rs);
CHECK(t, NULL, "H5RS_GET_STR");
VERIFY(t, s, "transferring");
cmp=HDstrcmp(t,s);
VERIFY(cmp, 0, "HDstrcmp");
/* Decrement reference count for string */
ret=H5RS_decr(rs);
CHECK(ret, FAIL, "H5RS_decr");
ret=H5RS_decr(rs);
CHECK(ret, FAIL, "H5RS_decr");
} /* end test_refstr_own() */
/****************************************************************
**
** test_refstr_finalize(): Test basic H5RS (ref-counted strings) code.
** Wrap up data for ref-counted string testing
**
****************************************************************/
static void
test_refstr_finalize(void)
{
} /* end test_refstr_finalize() */
/****************************************************************
**
** test_refstr(): Main H5RS testing routine.
**
****************************************************************/
void
test_refstr(void)
{
/* Output message about test being performed */
MESSAGE(5, ("Testing Reference Counted Strings\n"));
/* Initialize ref-counted strings testing data */
test_refstr_init();
/* Actual ref-counted strings tests */
test_refstr_create(); /* Test ref-counted string creation */
test_refstr_count(); /* Test ref-counted string counting */
test_refstr_dup(); /* Test ref-counted string duplication */
test_refstr_cmp(); /* Test ref-counted string comparison */
test_refstr_wrap(); /* Test ref-counted string wrapping */
test_refstr_own(); /* Test ref-counted string ownership transfer */
/* Finalize ref-counted strings testing data */
test_refstr_finalize();
} /* end test_refstr() */
Reference in New Issue View Git Blame Copy Permalink