2021-11-23 22:05:40 +08:00
|
|
|
<html>
|
|
|
|
|
<head>
|
|
|
|
|
<title>Debugging HDF5 Applications</title>
|
|
|
|
|
|
|
|
|
|
<h2>Introduction</h2>
|
|
|
|
|
|
|
|
|
|
<p>The HDF5 library contains a number of debugging features to
|
|
|
|
|
make programmers' lives easier including the ability to print
|
|
|
|
|
detailed error messages, check invariant conditions, display
|
|
|
|
|
timings and other statistics, and trace API function calls and
|
|
|
|
|
return values.
|
|
|
|
|
|
|
|
|
|
</p><dl>
|
|
|
|
|
<dt><b>Error Messages</b>
|
|
|
|
|
</dt><dd>Error messages are normally displayed automatically on the
|
|
|
|
|
standard error stream and include a stack trace of the library
|
|
|
|
|
including file names, line numbers, and function names. The
|
|
|
|
|
application has complete control over how error messages are
|
|
|
|
|
displayed and can disable the display on a permanent or
|
|
|
|
|
temporary basis. Refer to the documentation for the H5E error
|
|
|
|
|
handling package.
|
|
|
|
|
|
|
|
|
|
<br><br>
|
|
|
|
|
</dd><dt><b>Invariant Conditions</b>
|
|
|
|
|
</dt><dd>Unless <code>NDEBUG</code> is defined during compiling, the
|
|
|
|
|
library will include code to verify that invariant conditions
|
|
|
|
|
have the expected values. When a problem is detected the
|
|
|
|
|
library will display the file and line number within the
|
|
|
|
|
library and the invariant condition that failed. A core dump
|
|
|
|
|
may be generated for post mortem debugging. The code to
|
|
|
|
|
perform these checks can be included on a per-package bases.
|
|
|
|
|
|
|
|
|
|
<br><br>
|
|
|
|
|
</dd><dt><b>Timings and Statistics</b>
|
|
|
|
|
</dt><dd>The library can be configured to accumulate certain
|
|
|
|
|
statistics about things like cache performance, datatype
|
|
|
|
|
conversion, data space conversion, and data filters. The code
|
|
|
|
|
is included on a per-package basis and enabled at runtime by
|
|
|
|
|
an environment variable.
|
|
|
|
|
|
|
|
|
|
<br><br>
|
|
|
|
|
</dd><dt><b>API Tracing</b>
|
|
|
|
|
</dt><dd>All API calls made by an application can be displayed and
|
|
|
|
|
include formal argument names and actual values and the
|
|
|
|
|
function return value. This code is also conditionally
|
|
|
|
|
included at compile time and enabled at runtime.
|
|
|
|
|
</dd></dl>
|
|
|
|
|
|
|
|
|
|
<p>The statistics and tracing can be displayed on any output
|
|
|
|
|
stream (including streams opened by the shell) with output from
|
|
|
|
|
different packages even going to different streams.
|
|
|
|
|
|
|
|
|
|
</p><h2>Error Messages</h2>
|
|
|
|
|
|
|
|
|
|
<p>By default any API function that fails will print an error
|
|
|
|
|
stack to the standard error stream.
|
|
|
|
|
|
|
|
|
|
</p><p>
|
|
|
|
|
</p><center>
|
|
|
|
|
<table border="" align="center" width="100%">
|
|
|
|
|
<tbody><tr>
|
|
|
|
|
<td>
|
|
|
|
|
<p><code></code></p><pre><code>
|
|
|
|
|
HDF5-DIAG: Error detected in thread 0. Back trace follows.
|
|
|
|
|
#000: H5F.c line 1245 in H5Fopen(): unable to open file
|
|
|
|
|
major(04): File interface
|
|
|
|
|
minor(10): Unable to open file
|
|
|
|
|
#001: H5F.c line 846 in H5F_open(): file does not exist
|
|
|
|
|
major(04): File interface
|
|
|
|
|
minor(10): Unable to open file
|
|
|
|
|
</code></pre>
|
|
|
|
|
</td>
|
|
|
|
|
</tr>
|
|
|
|
|
</tbody></table>
|
|
|
|
|
</center>
|
|
|
|
|
|
|
|
|
|
<p>The error handling package (H5E) is described
|
|
|
|
|
<a href="./group___h5_e.html">elsewhere</a>.
|
|
|
|
|
|
|
|
|
|
</p><h2>Invariant Conditions</h2>
|
|
|
|
|
|
|
|
|
|
<p>To include checks for invariant conditions the library should
|
|
|
|
|
be configured with <code>--disable-production</code>, the
|
|
|
|
|
default for versions before 1.2. The library designers have made
|
|
|
|
|
every attempt to handle error conditions gracefully but an
|
|
|
|
|
invariant condition assertion may fail in certain cases. The
|
|
|
|
|
output from a failure usually looks something like this:
|
|
|
|
|
|
|
|
|
|
</p><p>
|
|
|
|
|
</p><center>
|
|
|
|
|
<table border="" align="center" width="100%">
|
|
|
|
|
<tbody><tr>
|
|
|
|
|
<td>
|
|
|
|
|
<p><code></code></p><pre><code>
|
|
|
|
|
Assertion failed: H5.c:123: i<NELMTS(H5_debug_g)
|
|
|
|
|
IOT Trap, core dumped.
|
|
|
|
|
</code></pre>
|
|
|
|
|
</td>
|
|
|
|
|
</tr>
|
|
|
|
|
</tbody></table>
|
|
|
|
|
</center>
|
|
|
|
|
|
|
|
|
|
<h2>Timings and Statistics</h2>
|
|
|
|
|
|
|
|
|
|
<p>Code to accumulate statistics is included at compile time by
|
|
|
|
|
using the <code>--enable-debug</code> configure switch. The
|
|
|
|
|
switch can be followed by an equal sign and a comma-separated
|
|
|
|
|
list of package names or else a default list is used.
|
|
|
|
|
|
|
|
|
|
</p><p>
|
|
|
|
|
</p><center>
|
|
|
|
|
<table border="" align="center" width="80%">
|
|
|
|
|
<tbody><tr>
|
|
|
|
|
<th>Name</th>
|
|
|
|
|
<th>Default</th>
|
|
|
|
|
<th>Description</th>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">a</td>
|
|
|
|
|
<td align="center">No</td>
|
|
|
|
|
<td>Attributes</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">ac</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Meta data cache</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">b</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>B-Trees</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">d</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Datasets</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">e</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Error handling</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">f</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Files</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">g</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Groups</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">hg</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Global heap</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">hl</td>
|
|
|
|
|
<td align="center">No</td>
|
|
|
|
|
<td>Local heaps</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">i</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Interface abstraction</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">mf</td>
|
|
|
|
|
<td align="center">No</td>
|
|
|
|
|
<td>File memory management</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">mm</td>
|
|
|
|
|
<td align="center">Yes</td>
|
2021-12-07 22:27:29 +08:00
|
|
|
<td>Library memory management</td>
|
2021-11-23 22:05:40 +08:00
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">o</td>
|
|
|
|
|
<td align="center">No</td>
|
|
|
|
|
<td>Object headers and messages</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">p</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Property lists</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">s</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Data spaces</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">t</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Datatypes</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">v</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Vectors</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align="center">z</td>
|
|
|
|
|
<td align="center">Yes</td>
|
|
|
|
|
<td>Raw data filters</td>
|
|
|
|
|
</tr>
|
|
|
|
|
</tbody></table>
|
|
|
|
|
</center>
|
|
|
|
|
|
|
|
|
|
<p>In addition to including the code at compile time the
|
|
|
|
|
application must enable each package at runtime. This is done
|
|
|
|
|
by listing the package names in the <code>HDF5_DEBUG</code>
|
|
|
|
|
environment variable. That variable may also contain file
|
|
|
|
|
descriptor numbers (the default is `2') which control the output
|
|
|
|
|
for all following packages up to the next file number. The
|
|
|
|
|
word <code>all</code> refers to all packages. Any word my be
|
|
|
|
|
preceded by a minus sign to turn debugging off for the package.
|
|
|
|
|
|
|
|
|
|
</p><p>
|
|
|
|
|
</p><center>
|
|
|
|
|
<table border="" align="center" width="100%">
|
|
|
|
|
<caption align="top"><b>Sample debug specifications</b></caption>
|
|
|
|
|
<tbody><tr valign="top">
|
|
|
|
|
<td><code>all</code></td>
|
|
|
|
|
<td>This causes debugging output from all packages to be
|
|
|
|
|
sent to the standard error stream.</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr valign="top">
|
|
|
|
|
<td><code>all -t -s</code></td>
|
|
|
|
|
<td>Debugging output for all packages except datatypes
|
|
|
|
|
and data spaces will appear on the standard error
|
|
|
|
|
stream.</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr valign="top">
|
|
|
|
|
<td><code>-all ac 255 t,s</code></td>
|
|
|
|
|
<td>This disables all debugging even if the default was to
|
|
|
|
|
debug something, then output from the meta data cache is
|
|
|
|
|
send to the standard error stream and output from data
|
|
|
|
|
types and spaces is sent to file descriptor 255 which
|
|
|
|
|
should be redirected by the shell.</td>
|
|
|
|
|
</tr>
|
|
|
|
|
</tbody></table>
|
|
|
|
|
</center>
|
|
|
|
|
|
|
|
|
|
<p>The components of the <code>HDF5_DEBUG</code> value may be
|
|
|
|
|
separated by any non-lowercase letter.
|
|
|
|
|
|
|
|
|
|
</p><h2>API Tracing</h2>
|
|
|
|
|
|
|
|
|
|
<p>The HDF5 library can trace API calls by printing the
|
|
|
|
|
function name, the argument names and their values, and the
|
|
|
|
|
return value. Some people like to see lots of output during
|
|
|
|
|
program execution instead of using a good symbolic debugger, and
|
|
|
|
|
this feature is intended for their consumption. For example,
|
|
|
|
|
the output from <code>h5ls foo</code> after turning on tracing,
|
|
|
|
|
includes:
|
|
|
|
|
|
|
|
|
|
</p><p>
|
|
|
|
|
</p><center>
|
|
|
|
|
<table border="" align="center" width="100%">
|
|
|
|
|
<tbody><tr>
|
|
|
|
|
<td>
|
|
|
|
|
<code><pre>
|
|
|
|
|
H5Tcopy(type=184549388) = 184549419 (type);
|
|
|
|
|
H5Tcopy(type=184549392) = 184549424 (type);
|
|
|
|
|
H5Tlock(type=184549424) = SUCCEED;
|
|
|
|
|
H5Tcopy(type=184549393) = 184549425 (type);
|
|
|
|
|
H5Tlock(type=184549425) = SUCCEED;
|
|
|
|
|
H5Fopen(filename="foo", flags=0, access=H5P_DEFAULT) = FAIL;
|
|
|
|
|
HDF5-DIAG: Error detected in thread 0. Back trace follows.
|
|
|
|
|
#000: H5F.c line 1245 in H5Fopen(): unable to open file
|
|
|
|
|
major(04): File interface
|
|
|
|
|
minor(10): Unable to open file
|
|
|
|
|
#001: H5F.c line 846 in H5F_open(): file does not exist
|
|
|
|
|
major(04): File interface
|
|
|
|
|
minor(10): Unable to open file
|
|
|
|
|
</pre></code>
|
|
|
|
|
</td>
|
|
|
|
|
</tr>
|
|
|
|
|
</tbody></table>
|
|
|
|
|
</center>
|
|
|
|
|
|
|
|
|
|
<p>The code that performs the tracing must be included in the
|
|
|
|
|
library by specifying the <code>--enable-trace</code>
|
|
|
|
|
configuration switch (the default for versions before 1.2). Then
|
|
|
|
|
the word <code>trace</code> must appear in the value of the
|
|
|
|
|
<code>HDF5_DEBUG</code> variable. The output will appear on the
|
|
|
|
|
last file descriptor before the word <code>trace</code> or two
|
|
|
|
|
(standard error) by default.
|
|
|
|
|
|
|
|
|
|
</p><p>
|
|
|
|
|
</p><center>
|
|
|
|
|
<table border="" align="center" width="100%">
|
|
|
|
|
<tbody><tr>
|
|
|
|
|
<td>To display the trace on the standard error stream:
|
|
|
|
|
<code><pre>$ env HDF5_DEBUG=trace a.out
|
|
|
|
|
</pre></code>
|
|
|
|
|
</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td>To send the trace to a file:
|
|
|
|
|
<code><pre>$ env HDF5_DEBUG="55 trace" a.out 55>trace-output
|
|
|
|
|
</pre></code>
|
|
|
|
|
</td>
|
|
|
|
|
</tr>
|
|
|
|
|
</tbody></table>
|
|
|
|
|
</center>
|
|
|
|
|
|
|
|
|
|
<h3>Performance</h3>
|
|
|
|
|
|
|
|
|
|
<p>If the library was not configured for tracing then there is no
|
|
|
|
|
unnecessary overhead since all tracing code is excluded.
|
|
|
|
|
However, if tracing is enabled but not used there is a small
|
|
|
|
|
penalty. First, code size is larger because of extra
|
|
|
|
|
statically-declared character strings used to store argument
|
|
|
|
|
types and names and extra auto variable pointer in each
|
|
|
|
|
function. Also, execution is slower because each function sets
|
|
|
|
|
and tests a local variable and each API function calls the
|
|
|
|
|
<code>H5_trace()</code> function.
|
|
|
|
|
|
|
|
|
|
</p><p>If tracing is enabled and turned on then the penalties from the
|
|
|
|
|
previous paragraph apply plus the time required to format each
|
|
|
|
|
line of tracing information. There is also an extra call to
|
|
|
|
|
H5_trace() for each API function to print the return value.
|
|
|
|
|
|
|
|
|
|
</p><h3>Safety</h3>
|
|
|
|
|
|
|
|
|
|
<p>The tracing mechanism is invoked for each API function before
|
|
|
|
|
arguments are checked for validity. If bad arguments are passed
|
|
|
|
|
to an API function it could result in a segmentation fault.
|
|
|
|
|
However, the tracing output is line-buffered so all previous
|
|
|
|
|
output will appear.
|
|
|
|
|
|
|
|
|
|
</p><h3>Completeness</h3>
|
|
|
|
|
|
|
|
|
|
<p>There are two API functions that don't participate in
|
|
|
|
|
tracing. They are <code>H5Eprint()</code> and
|
|
|
|
|
<code>H5Eprint_cb()</code> because their participation would
|
|
|
|
|
mess up output during automatic error reporting.
|
|
|
|
|
|
|
|
|
|
</p><p>On the other hand, a number of API functions are called during
|
|
|
|
|
library initialization and they print tracing information.
|
|
|
|
|
|
|
|
|
|
</p><h3>Implementation</h3>
|
|
|
|
|
|
|
|
|
|
<p>For those interested in the implementation here is a
|
|
|
|
|
description. Each API function should have a call to one of the
|
|
|
|
|
<code>H5TRACE()</code> macros immediately after the
|
|
|
|
|
<code>FUNC_ENTER()</code> macro. The first argument is the
|
|
|
|
|
return type encoded as a string. The second argument is the
|
|
|
|
|
types of all the function arguments encoded as a string. The
|
|
|
|
|
remaining arguments are the function arguments. This macro was
|
|
|
|
|
designed to be as terse and unobtrousive as possible.
|
|
|
|
|
|
|
|
|
|
</p><p>In order to keep the <code>H5TRACE()</code> calls synchronized
|
|
|
|
|
with the source code we've written a perl script which gets
|
|
|
|
|
called automatically just before Makefile dependencies are
|
|
|
|
|
calculated for the file. However, this only works when one is
|
|
|
|
|
using GNU make. To reinstrument the tracing explicitly, invoke
|
|
|
|
|
the <code>trace</code> program from the hdf5 bin directory with
|
|
|
|
|
the names of the source files that need to be updated. If any
|
|
|
|
|
file needs to be modified then a backup is created by appending
|
|
|
|
|
a tilde to the file name.
|
|
|
|
|
|
|
|
|
|
</p><p>
|
|
|
|
|
</p><center>
|
|
|
|
|
<table border="" align="center" width="100%">
|
|
|
|
|
<caption align="top"><b>Explicit Instrumentation</b></caption>
|
|
|
|
|
<tbody><tr>
|
|
|
|
|
<td>
|
|
|
|
|
<code><pre>
|
|
|
|
|
$ ../bin/trace *.c
|
|
|
|
|
H5E.c: in function `H5Ewalk_cb':
|
|
|
|
|
H5E.c:336: warning: trace info was not inserted
|
|
|
|
|
</pre></code>
|
|
|
|
|
</td>
|
|
|
|
|
</tr>
|
|
|
|
|
</tbody></table>
|
|
|
|
|
</center>
|
|
|
|
|
|
|
|
|
|
<p>Note: The warning message is the result of a comment of the
|
|
|
|
|
form <code>/*NO TRACE*/</code> somewhere in the function
|
|
|
|
|
body. Tracing information will not be updated or inserted if
|
|
|
|
|
such a comment exists.
|
|
|
|
|
|
|
|
|
|
</p><p>Error messages have the same format as a compiler so that they
|
|
|
|
|
can be parsed from program development environments like
|
|
|
|
|
Emacs. Any function which generates an error will not be
|
|
|
|
|
modified.</p>
|
|
|
|
|
|
|
|
|
|
</body></html>
|
