mirror of
https://github.com/HDFGroup/hdf5.git
synced 2024-11-27 02:10:55 +08:00
a0c466cd99
Purpose: Bug Fix Description: If an HDF5 file grows larger than its address space, it dies and is unable to write any data. This is more likely to happen since users are able to change the number of bytes used to store addresses in the file. Solution: HDF5 now throws an error instead of dying. In addition, it "reserves" address space for the local heap and for object headers (which do not allocate space immediately). This ensures that after the error occurs, there is enough address space left to flush the entire file to disk, so no data is lost. A more complete explanation is at /doc/html/TechNotes/ReservedFileSpace.html Platforms tested: sleipnir, copper (parallel), verbena, arabica, Windows (Visual Studio 7) Solution: Platforms tested: Misc. update:
310 lines
11 KiB
HTML
310 lines
11 KiB
HTML
<html>
|
|
|
|
<!--
|
|
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
|
* Copyright by the Board of Trustees of the University of Illinois. *
|
|
* All rights reserved. *
|
|
* *
|
|
* This file is part of HDF5. The full HDF5 copyright notice, including *
|
|
* terms governing use, modification, and redistribution, is contained in *
|
|
* the files COPYING and Copyright.html. COPYING can be found at the root *
|
|
* of the source code distribution tree; Copyright.html can be found at the *
|
|
* root level of an installed copy of the electronic HDF5 document set and *
|
|
* is linked from the top-level documents page. It can also be found at *
|
|
* http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html. If you do not have *
|
|
* access to either file, you may request a copy from hdfhelp@ncsa.uiuc.edu. *
|
|
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
|
-->
|
|
|
|
<head><title>
|
|
HDF5 Technical Notes
|
|
</title>
|
|
|
|
<!-- #BeginLibraryItem "/ed_libs/styles_Gen.lbi" -->
|
|
<!--
|
|
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
|
* Copyright by the Board of Trustees of the University of Illinois. *
|
|
* All rights reserved. *
|
|
* *
|
|
* This file is part of HDF5. The full HDF5 copyright notice, including *
|
|
* terms governing use, modification, and redistribution, is contained in *
|
|
* the files COPYING and Copyright.html. COPYING can be found at the root *
|
|
* of the source code distribution tree; Copyright.html can be found at the *
|
|
* root level of an installed copy of the electronic HDF5 document set and *
|
|
* is linked from the top-level documents page. It can also be found at *
|
|
* http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html. If you do not have *
|
|
* access to either file, you may request a copy from hdfhelp@ncsa.uiuc.edu. *
|
|
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
|
-->
|
|
|
|
<link href="ed_styles/GenElect.css" rel="stylesheet" type="text/css">
|
|
<!-- #EndLibraryItem --></head>
|
|
|
|
<body bgcolor="#FFFFFF">
|
|
|
|
|
|
<!-- #BeginLibraryItem "/ed_libs/NavBar_TechN.lbi" --><hr>
|
|
<center>
|
|
<table cellspacing=0 cellpadding=0 border=0 width=98%>
|
|
<tr><td valign=top align=left>
|
|
<a href="index.html">HDF5 documents and links</a> <br>
|
|
<a href="H5.intro.html">Introduction to HDF5</a> <br>
|
|
<!--
|
|
<a href="Glossary.html">Glossary</a><br>
|
|
-->
|
|
</td>
|
|
|
|
<td valign=top align=right>
|
|
<a href="http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/index.html">HDF5 User's Guide</a> <br>
|
|
<a href="ADGuide.html">HDF5 Application Developer's Guide</a> <br>
|
|
<a href="RM_H5Front.html">HDF5 Reference Manual</a> <br>
|
|
|
|
<!--
|
|
And in this document,
|
|
HDF5 Technical Notes
|
|
<br>
|
|
<a href="TechNotes/FreeLists.html">Memory Management and Free Lists</a>
|
|
-->
|
|
|
|
</td></tr>
|
|
</table>
|
|
</center>
|
|
<hr>
|
|
<!-- #EndLibraryItem --><br>
|
|
<br>
|
|
|
|
<center>
|
|
<h1>HDF5 Technical Notes</h1>
|
|
<h3>Technical Notes for HDF5 Library and Driver Developers</h3>
|
|
</center>
|
|
|
|
|
|
<center>
|
|
<table border=0 width=85%>
|
|
|
|
<tr><td valign=top colspan=3>
|
|
This informal volume of technical notes is of interest to
|
|
those who develop and maintain the HDF5 library and
|
|
related, closely-coupled drivers.
|
|
These notes are not generally of interest to applications developers
|
|
and certainly not of interest to users.
|
|
(Some of these documents may be somewhat out of date as they were
|
|
working papers for the design process.)
|
|
<br>
|
|
<br>
|
|
</td></tr>
|
|
|
|
<tr><td valign=top width=40%><a href="TechNotes/MemoryMgmt.html">
|
|
Memory Management</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of memory management issues in HDF5
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/FreeLists.html">
|
|
Memory Management and</a>
|
|
<br>
|
|
<a href="TechNotes/FreeLists.html">Free Lists</a>
|
|
</td><td> </td><td valign=top>
|
|
Notes regarding the implementation of free lists and memory management
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/HeapMgmt.html">Heap Management</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of the H5H heap management fuctions
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/RawDStorage.html">Raw Data Storage</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of the storage of raw HDF5 data
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/VFL.html">Virtual File Layer</a>
|
|
</td><td> </td><td valign=top>
|
|
A description of the HDF5 virtual file layer (VFL),
|
|
a public API for the implementation of custom I/O drivers
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/VFLfunc.html">List of VFL Functions</a>
|
|
</td><td> </td><td valign=top>
|
|
A list of the VFL functions, <code>H5FD*</code>
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/IOPipe.html">I/O Pipeline</a>
|
|
</td><td> </td><td valign=top>
|
|
A description of the raw data I/O pipeline
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/BigDataSmMach.html">
|
|
Large Datasets on Small</a>
|
|
<br>
|
|
<a href="TechNotes/BigDataSmMach.html">Machines</a>
|
|
</td><td> </td><td valign=top>
|
|
A guide to accessing large datasets on small computers
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/MoveDStruct.html">
|
|
Relocating a File Data</a>
|
|
<br>
|
|
<a href="TechNotes/MoveDStruct.html">Structure</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of the issues involved in moving file data structures once
|
|
they have been cached in memory
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/ExternalFiles.html">
|
|
Working with External Files</a>
|
|
</td><td> </td><td valign=top>
|
|
A guide to the use of multiple files with HDF5
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/ObjectHeader.html">Object Headers</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of the H5O object header functions
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/SymbolTables.html">
|
|
Symbol Table Caching Issues</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of issues involving caching of object header messages in
|
|
symbol table entries
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/H4-H5Compat.html">
|
|
HDF4/HDF5 Compatibility</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of compatibility issues between HDF4 and HDF5
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/ChunkingStudy.html">
|
|
Testing the Chunked Layout</a>
|
|
<br>
|
|
<a href="TechNotes/ChunkingStudy.html">of HDF5</a>
|
|
</td><td> </td><td valign=top>
|
|
A white paper discussing the motivation to implement raw data chunking
|
|
in the HDF5 library
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/LibMaint.html">Library Maintenance</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of HDF5 library maintenance issues
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/CodeReview.html">Code Review</a>
|
|
</td><td> </td><td valign=top>
|
|
Code Review 1 and 2
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/Version.html">
|
|
Release Version Numbers</a>
|
|
</td><td> </td><td valign=top>
|
|
A description of HDF5 release version numbers
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/NamingScheme.html">Naming Schemes</a>
|
|
</td><td> </td><td valign=top>
|
|
A discussion of naming schemes for HDF5 library modules, functions,
|
|
datatypes, etc.
|
|
</td></tr>
|
|
|
|
<tr><td valign=top>
|
|
<a href="TechNotes/ThreadSafeLibrary.html">Thread Safe HDF5 Library</a>
|
|
<br>
|
|
<a href="TechNotes/ThreadSafeLibrary.html">Implementation</a>
|
|
</td><td> </td><td valign=top>
|
|
A report on the implementation of a thread safe HDF5 library.
|
|
</td></tr>
|
|
|
|
<tr><td valign=top>
|
|
<a href="TechNotes/openmp-hdf5.html">Using HDF5 with OpenMP</a>
|
|
</td><td> </td><td valign=top>
|
|
A short report on using HDF5 with OpenMP.
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/SWControls.html">HDF5 Software Controls</a>
|
|
</td><td> </td><td valign=top>
|
|
Descriptions of the HDF5 knobs and controls, such as the
|
|
environment variables and settings that control the functionality
|
|
of the HDF5 libraries and tools.
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/Daily_Test_Explained.htm">Daily Test Explained</a>
|
|
</td><td> </td><td valign=top>
|
|
An explanation of the Daily Testing for HDF software conducted.
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/TestReview.html">Test Review</a>
|
|
</td><td> </td><td valign=top>
|
|
Results of reviewing tests for API functions.
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/Basic_perform.html">Basic Performance Tools</a>
|
|
</td><td> </td><td valign=top>
|
|
A description of the three basic performance tools (chunk, iopipe, overhead).
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/VLTypes.html">Variable-Length Datatype Info</a>
|
|
</td><td> </td><td valign=top>
|
|
Description of various aspects of using variable-length datatypes in HDF5.
|
|
</td></tr>
|
|
|
|
<tr><td valign=top><a href="TechNotes/ReservedFileSpace.html">Reserved File Address Space</a>
|
|
</td><td> </td><td valign=top>
|
|
Description of HDF5's internal system for ensuring that files stay within their address space.
|
|
</td></tr>
|
|
|
|
</table>
|
|
</center>
|
|
|
|
<br>
|
|
<br>
|
|
|
|
|
|
<!-- #BeginLibraryItem "/ed_libs/NavBar_TechN.lbi" --><hr>
|
|
<center>
|
|
<table cellspacing=0 cellpadding=0 border=0 width=98%>
|
|
<tr><td valign=top align=left>
|
|
<a href="index.html">HDF5 documents and links</a> <br>
|
|
<a href="H5.intro.html">Introduction to HDF5</a> <br>
|
|
<!--
|
|
<a href="Glossary.html">Glossary</a><br>
|
|
-->
|
|
</td>
|
|
|
|
<td valign=top align=right>
|
|
<a href="http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/index.html">HDF5 User's Guide</a> <br>
|
|
<a href="ADGuide.html">HDF5 Application Developer's Guide</a> <br>
|
|
<a href="RM_H5Front.html">HDF5 Reference Manual</a> <br>
|
|
|
|
<!--
|
|
And in this document,
|
|
HDF5 Technical Notes
|
|
<br>
|
|
<a href="TechNotes/FreeLists.html">Memory Management and Free Lists</a>
|
|
-->
|
|
|
|
</td></tr>
|
|
</table>
|
|
</center>
|
|
<hr>
|
|
<!-- #EndLibraryItem --><address>
|
|
<table width=100% border=0>
|
|
<tr><td align=left valign=top>
|
|
|
|
<!-- #BeginLibraryItem "/ed_libs/Footer.lbi" --><address>
|
|
<a href="mailto:hdfhelp@ncsa.uiuc.edu">HDF Help Desk</a>
|
|
<br>
|
|
Describes HDF5 Release 1.7, the unreleased development branch; working toward HDF5 Release 1.8.0
|
|
</address><!-- #EndLibraryItem -->
|
|
|
|
Last modified: 25 September 2003
|
|
|
|
</td>
|
|
<td align=right valign=top>
|
|
<a href="Copyright.html">Copyright</a>
|
|
</td></tr>
|
|
</table>
|
|
|
|
|
|
</body>
|
|
</html>
|