mirror/hdf5
2
0
Fork 0
mirror of https://github.com/HDFGroup/hdf5.git synced 2025-02-23 16:20:57 +08:00
Code Issues Packages Projects Releases Wiki Activity

[svn-r5521] Purpose:

Browse Source 
Code improvement

Description:
    The metadata aggregation code in the library was not terribly smart about
    extending contiguous regions of metadata in the file and would not extend
    them as far as possible.  This causes space in the file to be wasted, also.

Solution:
    Be smarter about extending the space used in the file for metadata by
    checking whether new metadata blocks allocated in the file are at the end
    of the current metadata aggregation region and append them to the metadata
    region if so.  This has the nice side benefit of reducing the number of
    bytes we waste in the file and reducing the size of the file by a small
    amount in some cases.

    This reduces the number of I/O operations which hit the disk for my test
    program from 53 to 19 (i.e. from 393 to 19, overall).

Platforms tested:
    Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
This commit is contained in:
Quincey Koziol 2002-06-04 08:37:51 -05:00
parent 8ae371469a
commit ba26e8f2ad
2 changed files with 39 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
release_docs/RELEASE.txt
View File

@ -168,6 +168,9 @@ Documentation
New Features
============
* Improved internal metadata aggregation, which can reduce the number of
actual I/O calls made, improving performance. Additionally, this can
reduce the size of files produced. QAK - 2002/06/04
* Improved internal metadata caching, which can reduce the number of actual
I/O calls made by a substantial amount, improving
performance. QAK - 2002/06/03

47
src/H5FD.c
View File

@ -1425,22 +1425,47 @@ H5FD_alloc(H5FD_t *file, H5FD_mem_t type, hsize_t size)
if((file->feature_flags&H5FD_FEAT_AGGREGATE_METADATA) && type!=H5FD_MEM_DRAW) {
/* Check if the space requested is larger than the space left in the block */
if(size>file->cur_meta_block_size) {
haddr_t new_meta; /* Address for new metadata */
/* Check if the block asked for is too large for a metadata block */
if(size>=file->def_meta_block_size) {
/* Allocate just enough room for this new block the regular way */
ret_value=H5FD_real_alloc(file,type,size);
/* Allocate more room for this new block the regular way */
new_meta=H5FD_real_alloc(file,type,size);
/* Check if the new metadata is at the end of the current metadata block */
if(file->eoma+file->cur_meta_block_size==new_meta) {
/* Treat the allocation request as if the current metadata block
* grew by the amount allocated and just update the eoma
* address. Don't bother updating the cur_meta_block_size
* since it will just grow and shrink by the same amount.
*/
ret_value=file->eoma;
file->eoma+=size;
} /* end if */
else {
/* Use the new metadata block for the space allocated */
ret_value=new_meta;
} /* end else */
} /* end if */
else {
/*
* Instead of just dropping the remainder of the block on the
* floor and leaving the space in the file unused, we should
* return this small piece of unused space to the free list
* management. - QAK
*/
/* Allocate another metadata block */
file->eoma=H5FD_real_alloc(file,H5FD_MEM_DEFAULT,file->def_meta_block_size);
file->cur_meta_block_size=file->def_meta_block_size;
new_meta=H5FD_real_alloc(file,H5FD_MEM_DEFAULT,file->def_meta_block_size);
/* Check if the new metadata is at the end of the current metadata block */
if(file->eoma+file->cur_meta_block_size==new_meta) {
file->cur_meta_block_size+=file->def_meta_block_size;
} /* end if */
else {
/*
* Instead of just dropping the remainder of the block on the
* floor and leaving the space in the file unused, we should
* return this small piece of unused space to the free list
* management. - QAK
*/
file->eoma=new_meta;
file->cur_meta_block_size=file->def_meta_block_size;
} /* end else */
/* Allocate space out of the metadata block */
ret_value=file->eoma;