mirror of
https://github.com/HDFGroup/hdf5.git
synced 2024-11-27 02:10:55 +08:00
a901d24a8b
Purpose:
Renamed old "HDF5 User's Guide" as "HDF5 User's Guide, Release 1.4.5".
Added navigation bar link to new user's guide on HDF server.
Changed release tag line (in old UG only) back to
"Describes HDF5 Release 1.4.5, February 200 3"
Platforms tested:
IE 5
191 lines
8.2 KiB
HTML
191 lines
8.2 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
|
|
<html>
|
|
<head>
|
|
<title>Data Caching</title>
|
|
|
|
<!-- #BeginLibraryItem "/ed_libs/styles_UG.lbi" -->
|
|
<!--
|
|
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
|
* Copyright by the Board of Trustees of the University of Illinois. *
|
|
* All rights reserved. *
|
|
* *
|
|
* This file is part of HDF5. The full HDF5 copyright notice, including *
|
|
* terms governing use, modification, and redistribution, is contained in *
|
|
* the files COPYING and Copyright.html. COPYING can be found at the root *
|
|
* of the source code distribution tree; Copyright.html can be found at the *
|
|
* root level of an installed copy of the electronic HDF5 document set and *
|
|
* is linked from the top-level documents page. It can also be found at *
|
|
* http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html. If you do not have *
|
|
* access to either file, you may request a copy from hdfhelp@ncsa.uiuc.edu. *
|
|
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
|
-->
|
|
|
|
<link href="ed_styles/UGelect.css" rel="stylesheet" type="text/css">
|
|
<!-- #EndLibraryItem --></head>
|
|
|
|
<body bgcolor="#FFFFFF">
|
|
|
|
|
|
<!-- #BeginLibraryItem "/ed_libs/NavBar_UG.lbi" --><hr>
|
|
<center>
|
|
<table border=0 width=98%>
|
|
<tr><td valign=top align=left>
|
|
<a href="index.html">HDF5 documents and links</a> <br>
|
|
<a href="H5.intro.html">Introduction to HDF5</a> <br>
|
|
<a href="RM_H5Front.html">HDF5 Reference Manual</a> <br>
|
|
<a href="http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/index.html">HDF5 User's Guide for Release 1.6</a> <br>
|
|
<!--
|
|
<a href="Glossary.html">Glossary</a><br>
|
|
-->
|
|
</td>
|
|
<td valign=top align=right>
|
|
And in this document, the
|
|
<a href="H5.user.html"><strong>HDF5 User's Guide from Release 1.4.5:</strong></a>
|
|
<br>
|
|
<a href="Files.html">Files</a>
|
|
<a href="Datasets.html">Datasets</a>
|
|
<a href="Datatypes.html">Datatypes</a>
|
|
<a href="Dataspaces.html">Dataspaces</a>
|
|
<a href="Groups.html">Groups</a>
|
|
<br>
|
|
<a href="References.html">References</a>
|
|
<a href="Attributes.html">Attributes</a>
|
|
<a href="Properties.html">Property Lists</a>
|
|
<a href="Errors.html">Error Handling</a>
|
|
<br>
|
|
<a href="Filters.html">Filters</a>
|
|
<a href="Caching.html">Caching</a>
|
|
<a href="Chunking.html">Chunking</a>
|
|
<a href="MountingFiles.html">Mounting Files</a>
|
|
<br>
|
|
<a href="Performance.html">Performance</a>
|
|
<a href="Debugging.html">Debugging</a>
|
|
<a href="Environment.html">Environment</a>
|
|
<a href="ddl.html">DDL</a>
|
|
</td></tr>
|
|
</table>
|
|
</center>
|
|
<hr>
|
|
<!-- #EndLibraryItem --><h1>Data Caching</h1>
|
|
|
|
<h2>1. Meta Data Caching</h2>
|
|
|
|
<p>The HDF5 library caches two types of data: meta data and raw
|
|
data. The meta data cache holds file objects like the file
|
|
header, symbol table nodes, global heap collections, object
|
|
headers and their messages, etc. in a partially decoded
|
|
state. The cache has a fixed number of entries which is set with
|
|
the file access property list (defaults to 10k) and each entry
|
|
can hold a single meta data object. Collisions between objects
|
|
are handled by preempting the older object in favor of the new
|
|
one.
|
|
|
|
<h2>2. Raw Data Chunk Caching</h2>
|
|
|
|
<p>Raw data chunks are cached because I/O requests at the
|
|
application level typically don't map well to chunks at the
|
|
storage level. The chunk cache has a maximum size in bytes
|
|
set with the file access property list (defaults to 1MB) and
|
|
when the limit is reached chunks are preempted based on the
|
|
following set of heuristics.
|
|
|
|
<ul>
|
|
<li>Chunks which have not been accessed for a long time
|
|
relative to other chunks are penalized.
|
|
<li>Chunks which have been accessed frequently in the recent
|
|
past are favored.
|
|
<li>Chunks which are completely read and not written, completely
|
|
written but not read, or completely read and completely
|
|
written are penalized according to <em>w0</em>, an
|
|
application-defined weight between 0 and 1 inclusive. A weight
|
|
of zero does not penalize such chunks while a weight of 1
|
|
penalizes those chunks more than all other chunks. The
|
|
default is 0.75.
|
|
<li>Chunks which are larger than the maximum cache size do not
|
|
participate in the cache.
|
|
</ul>
|
|
|
|
<p>One should choose large values for <em>w0</em> if I/O requests
|
|
typically do not overlap but smaller values for <em>w0</em> if
|
|
the requests do overlap. For instance, reading an entire 2d
|
|
array by reading from non-overlapping "windows" in a row-major
|
|
order would benefit from a high <em>w0</em> value while reading
|
|
a diagonal accross the dataset where each request overlaps the
|
|
previous request would benefit from a small <em>w0</em>.
|
|
|
|
<h2>3. Data Caching Operations</h2>
|
|
|
|
<p>The cache parameters for both caches are part of a file access
|
|
property list and are set and queried with this pair of
|
|
functions:
|
|
|
|
<dl>
|
|
<dt><code>herr_t H5Pset_cache(hid_t <em>plist</em>, unsigned int
|
|
<em>mdc_nelmts</em>, size_t <em>rdcc_nbytes</em>, double
|
|
<em>w0</em>)</code>
|
|
<dt><code>herr_t H5Pget_cache(hid_t <em>plist</em>, unsigned int
|
|
*<em>mdc_nelmts</em>, size_t *<em>rdcc_nbytes</em>, double
|
|
<em>w0</em>)</code>
|
|
<dd>Sets or queries the meta data cache and raw data chunk cache
|
|
parameters. The <em>plist</em> is a file access property
|
|
list. The number of elements (objects) in the meta data cache
|
|
is <em>mdc_nelmts</em>. The total size of the raw data chunk
|
|
cache and the preemption policy is <em>rdcc_nbytes</em> and
|
|
<em>w0</em>. For <code>H5Pget_cache()</code> any (or all) of
|
|
the pointer arguments may be null pointers.
|
|
</dl>
|
|
|
|
|
|
<!-- #BeginLibraryItem "/ed_libs/NavBar_UG.lbi" --><hr>
|
|
<center>
|
|
<table border=0 width=98%>
|
|
<tr><td valign=top align=left>
|
|
<a href="index.html">HDF5 documents and links</a> <br>
|
|
<a href="H5.intro.html">Introduction to HDF5</a> <br>
|
|
<a href="RM_H5Front.html">HDF5 Reference Manual</a> <br>
|
|
<a href="http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/index.html">HDF5 User's Guide for Release 1.6</a> <br>
|
|
<!--
|
|
<a href="Glossary.html">Glossary</a><br>
|
|
-->
|
|
</td>
|
|
<td valign=top align=right>
|
|
And in this document, the
|
|
<a href="H5.user.html"><strong>HDF5 User's Guide from Release 1.4.5:</strong></a>
|
|
<br>
|
|
<a href="Files.html">Files</a>
|
|
<a href="Datasets.html">Datasets</a>
|
|
<a href="Datatypes.html">Datatypes</a>
|
|
<a href="Dataspaces.html">Dataspaces</a>
|
|
<a href="Groups.html">Groups</a>
|
|
<br>
|
|
<a href="References.html">References</a>
|
|
<a href="Attributes.html">Attributes</a>
|
|
<a href="Properties.html">Property Lists</a>
|
|
<a href="Errors.html">Error Handling</a>
|
|
<br>
|
|
<a href="Filters.html">Filters</a>
|
|
<a href="Caching.html">Caching</a>
|
|
<a href="Chunking.html">Chunking</a>
|
|
<a href="MountingFiles.html">Mounting Files</a>
|
|
<br>
|
|
<a href="Performance.html">Performance</a>
|
|
<a href="Debugging.html">Debugging</a>
|
|
<a href="Environment.html">Environment</a>
|
|
<a href="ddl.html">DDL</a>
|
|
</td></tr>
|
|
</table>
|
|
</center>
|
|
<hr>
|
|
<!-- #EndLibraryItem --><!-- #BeginLibraryItem "/ed_libs/Footer.lbi" --><address>
|
|
<a href="mailto:hdfhelp@ncsa.uiuc.edu">HDF Help Desk</a>
|
|
<br>
|
|
Describes HDF5 Release 1.4.5, February 2003
|
|
</address><!-- #EndLibraryItem --><!-- Created: Tue May 26 15:20:14 EDT 1998 -->
|
|
<!-- hhmts start -->
|
|
Last modified: 13 December 1999
|
|
<!-- hhmts end -->
|
|
|
|
|
|
</body>
|
|
</html>
|