mirror of
https://github.com/HDFGroup/hdf5.git
synced 2024-11-21 01:04:10 +08:00
dd99ecc1bc
A little editorial cleanup. Added link to Ragged.html, Ragged Arrays. Added link to Environment.html, Environmental Variables and Configuration Parameters. Added link to Copyright.html. Ragged.html Added "Use at your own risk" notice. Environment.html New file for environmental variables and configuration parameters. Datatypes.html Added the following datatypes: H5T_NATIVE_HSIZE H5T_NATIVE_HSSIZE H5T_NATIVE_HERR H5T_NATIVE_HBOOL Section 8, "Data Conversions" Added named link tag to catch hyperlink from RM. Revised "Note:" to emphasize limited inate data-conversion capability. Copyright.html New file. Copyright statement, linked into User's Guide and Reference Manual. Glossary.html Unlinked for Beta. General Changed email link at bottom of "user guide level documents" (as contrasted with "developer guide level documents" listed on the lower half of the User's Guide front page) to HDF Help Desk, hdfhelp@ncsa.uiuc.edu. Changed the following function names: From: To: H5version H5get_libversion H5vers_check H5check_version H5Anum_attrs H5Aget_num_attrs H5Fget_create_template() H5Fget_create_plist() H5Fget_access_template() H5Fget_access_plist() H5Gstat H5Gget_stat H5Sextent_npoints H5Sget_simple_extent_npoints H5Sextent_ndims H5Sget_simple_extent_ndims H5Sextent_dims H5Sget_simple_extent_dims H5Sget_class H5Sget_simple_extent_type H5Sselect_npoints H5Sget_select_npoints
95 lines
3.5 KiB
HTML
95 lines
3.5 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
|
|
<html>
|
|
<head>
|
|
<title>Data Caching</title>
|
|
</head>
|
|
|
|
<body>
|
|
<h1>Meta Data Caching</h1>
|
|
|
|
<p>The HDF5 library caches two types of data: meta data and raw
|
|
data. The meta data cache holds file objects like the file
|
|
header, symbol table nodes, global heap collections, object
|
|
headers and their messages, etc. in a partially decoded
|
|
state. The cache has a fixed number of entries which is set with
|
|
the file access property list (defaults to 10k) and each entry
|
|
can hold a single meta data object. Collisions between objects
|
|
are handled by preempting the older object in favor of the new
|
|
one.
|
|
|
|
<h1>Raw Data Chunk Caching</h1>
|
|
|
|
<p>Raw data chunks are cached because I/O requests at the
|
|
application level typically don't map well to chunks at the
|
|
storage level. The chunk cache has a maximum size in bytes
|
|
set with the file access property list (defaults to 1MB) and
|
|
when the limit is reached chunks are preempted based on the
|
|
following set of heuristics.
|
|
|
|
<ul>
|
|
<li>Chunks which have not been accessed for a long time
|
|
relative to other chunks are penalized.
|
|
<li>Chunks which have been accessed frequently in the recent
|
|
past are favored.
|
|
<li>Chunks which are completely read and not written, completely
|
|
written but not read, or completely read and completely
|
|
written are penalized according to <em>w0</em>, an
|
|
application-defined weight between 0 and 1 inclusive. A weight
|
|
of zero does not penalize such chunks while a weight of 1
|
|
penalizes those chunks more than all other chunks. The
|
|
default is 0.75.
|
|
<li>Chunks which are larger than the maximum cache size do not
|
|
participate in the cache.
|
|
</ul>
|
|
|
|
<p>One should choose large values for <em>w0</em> if I/O requests
|
|
typically do not overlap but smaller values for <em>w0</em> if
|
|
the requests do overlap. For instance, reading an entire 2d
|
|
array by reading from non-overlapping "windows" in a row-major
|
|
order would benefit from a high <em>w0</em> value while reading
|
|
a diagonal accross the dataset where each request overlaps the
|
|
previous request would benefit from a small <em>w0</em>.
|
|
|
|
<h1>The API</h1>
|
|
|
|
<p>The cache parameters for both caches are part of a file access
|
|
property list and are set and queried with this pair of
|
|
functions:
|
|
|
|
<dl>
|
|
<dt><code>herr_t H5Pset_cache(hid_t <em>plist</em>, unsigned int
|
|
<em>mdc_nelmts</em>, size_t <em>rdcc_nbytes</em>, double
|
|
<em>w0</em>)</code>
|
|
<dt><code>herr_t H5Pget_cache(hid_t <em>plist</em>, unsigned int
|
|
*<em>mdc_nelmts</em>, size_t *<em>rdcc_nbytes</em>, double
|
|
<em>w0</em>)</code>
|
|
<dd>Sets or queries the meta data cache and raw data chunk cache
|
|
parameters. The <em>plist</em> is a file access property
|
|
list. The number of elements (objects) in the meta data cache
|
|
is <em>mdc_nelmts</em>. The total size of the raw data chunk
|
|
cache and the preemption policy is <em>rdcc_nbytes</em> and
|
|
<em>w0</em>. For <code>H5Pget_cache()</code> any (or all) of
|
|
the pointer arguments may be null pointers.
|
|
</dl>
|
|
|
|
<!--
|
|
<hr>
|
|
<address><a href="mailto:matzke@llnl.gov">Robb Matzke</a></address>
|
|
-->
|
|
<!-- Created: Tue May 26 15:20:14 EDT 1998 -->
|
|
<!-- hhmts start -->
|
|
<!--
|
|
Last modified: Tue May 26 15:38:27 EDT 1998
|
|
-->
|
|
<!-- hhmts end -->
|
|
|
|
<hr>
|
|
<address>
|
|
<a href="mailto:hdfhelp@ncsa.uiuc.edu">HDF Help Desk</a>
|
|
</address>
|
|
|
|
Last modified: 9 September 1998
|
|
|
|
</body>
|
|
</html>
|