mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2024-12-09 08:11:38 +08:00
Code Issues Packages Projects Releases Wiki Activity
32a776badf
netcdf-c/nc_test/nc_test.html
Ed Hartnett 18f4bca367 moving to trunk subdir
2010-06-03 13:24:43 +00:00

615 lines
17 KiB
HTML
Raw Blame History

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<title>
Description of nc_test
</title>
</head>
<body bgcolor="#00009C" text=#FFFF00 link=#FFFFFF alink=#00FF00 vlink=#FFFFFF>
<center>
<h1>
Description of <tt>nc_test</tt>
</h1>
</a>
<h2>Harvey Davies</h2>
<h3>7 August 1996</h3>
</center>
<h3>
TERMINOLOGY
</h3>
<dl>
<dt>Test
<dd>Process of testing one netCDF library function.
<dt>Comparison
<dd>Successful match of value of individual variable/attribute
element read (possibly after put) with that expected.
<dt>Error
<dd>Non-zero status returned by netCDF library function.
<dt>Failure
<dd>Unexpected result (including error status). Possible causes:
<ul>
<li>absent data file
<li>wrong permissions
<li>bug in netCDF library function
</ul>
<dt>Blunder
<dd>Bug in <tt>nc_test</tt>.
</dl>
<h3>
PROGRAM OPERATION AND COMMAND-LINE OPTIONS
</h3>
All printed output is written to <tt>stderr</tt>, not <tt>stdout</tt>. This
is intended to facilitate debugging.
<p>
There is a separate test for each netCDF library function (apart from
<tt>test_nc_enddef</tt> which simply calls <tt>test_nc_redef</tt>, which tests
both <tt>nc_redef</tt> and <tt>nc_enddef</tt>). Each test generates one line
of results if there are no failures and additional lines if there are
failures. The simplest such line for no failures is one such as:
<pre>
*** Testing nc_open ... ok
</pre>
Many tests count the number of comparisons and if there are no failures print
a line such as:
<pre>
*** Testing nc_get_var1_short ... 682 good comparisons. ok
</pre>
Each failure produces a line until the limit (per test) specified by the
<tt>-n</tt>
option (default 8) is reached. E.g. running the full test in a read-only
directory produces:
<pre>
*** Testing nc_open ...
FAILURE at line 119 of test_read.c: nc_create: Permission denied
FAILURE at line 124 of test_read.c: nc_open: No such file or directory
FAILURE at line 129 of test_read.c: remove of scratch.nc failed
### 3 FAILURES TESTING nc_open! ###
</pre>
The total number of failures is printed at the end. E.g.
<pre>
Total number of failures: 60
</pre>
If there are no failures then <tt>nc_test</tt> exits with exit-status 0, otherwise 1.
<p>
The following usage instructions are produced by the command
<tt>nc_test -h</tt>:
<pre>
nc_test [-hrv] [-n <MAX_NMPT>]
[-h] Print help
[-c] Create file test.nc (Do not do tests)
[-r] Just do read-only tests
[-v] Verbose mode
[-n <MAX_NMPT>] max. number of messages per test (Default: 8)
</pre>
The <tt>-c</tt> option creates the read-only test file test.nc and does
nothing else. (Originally I had a separate program, but it was simpler to
combine them.) The utility <tt>make</tt> will run <tt>nc_test -c</tt> to
recreate <tt>test.nc</tt> whenever <tt>nc_test</tt> is changed, but I suggest
including <tt>test.nc</tt> in the distribution so users do not normally have
to do this step.
<p>
As mentioned above, it is possible to run the full test in a read-only
directory. (This currently generates 60 failures.) But in a read-only
directory it is better to use the <tt>-r</tt> option which suppresses all
actions which write, so there should be no failures.
<p>
The option <tt>-v</tt> sets verbose mode, in which additional details are
produced after certain failures. (This is not consistent across tests. An
earlier version also produced details of each comparison, whether successful
or not; but this proved too voluminous.)
<p>
The option <tt>-n <i>MAX_NMPT</i></tt> sets the maximum number of messages
printed per test. This defaults to 8.
<h3>
SOURCE FILES
</h3>
<dl>
<dt><tt>nc_test.c</tt>
<dd>Driver
<dt><tt>tests.h</tt>
<dd>Main include file
<dt><tt>error.c</tt>
<dd>Functions concerned with low-level printing and failure handling.
<dt><tt>error.h</tt>
<dd>Include file related to error.c.
<dt><tt>util.c</tt>
<dd>Various utility functions for generating data, comparisons, etc.
<dt><tt>test_get.m4</tt>
<dd>Read-only type-specific tests.
<dt><tt>test_put.m4</tt>
<dd>Write type-specific tests.
<dt><tt>test_read.c</tt>
<dd>Other read-only tests (in fact there is some writing).
<dt><tt>test_write.c</tt>
<dd>Other write tests.
</dl>
<p>
The <tt>wc</tt> utility gives the following counts of lines, words and
characters:
<pre>
22 70 622 error.h
324 1000 9226 tests.h
366 1303 11029 test_get.m4
610 1941 16469 test_put.m4
52 155 1159 error.c
277 692 7645 nc_test.c
1599 5908 45979 test_read.c
1882 6887 55625 test_write.c
728 2623 19434 util.c
5860 20579 167188 total
</pre>
<h3>
DATA FILES
</h3>
Two netCDF files are used. One (<tt>test.nc</tt>) is read-only, except that
the <tt>-c</tt> option creates it. The other (<tt>scratch.nc</tt>) has a
lifetime within the period of any test which does writing.
<p>
The file <tt>tests.h</tt> is used merely as an example of a non-netCDF file.
<h3>
INTERNAL METADATA DESCRIBING TEST DATA
</h3>
The test metadata defining the file <tt>tests.nc</tt> (and for many tests also
<tt>scratch.nc</tt>) is described by parameters (e.g. <tt>NVARS</tt>, the
number of variables) defined in <tt>tests.h</tt> and global variables with the
prefix <tt>dim</tt>, <tt>var</tt>, <tt>att</tt> or <tt>gatt</tt> (global
attribute). E.g. a netCDF variable is described by global variables such as
<tt>var_name</tt>, <tt>var_type</tt> and <tt>var_dimid</tt>. Note that the
data values of netCDF variables and attributes are not stored in this way.
Instead, these are generated by function <tt>hash</tt> each time they are
needed.
<p>
The need for separate global attribute tests is obviated by accessing
<tt>att/gatt</tt>
variables via macros which give global values for a varid of -1. E.g.
<tt>ATT_NAME(varid,j)</tt> gives <tt>gatt_name[j]</tt> if varid is -1,
otherwise it gives <tt>att_name[varid][j]</tt>. Thus attribute tests will
typically vary <tt>varid</tt> from -1 to <tt>NVARS-1</tt>.
<p>
The function <tt>init_gvars</tt> (in <tt>util.c</tt>) defines the values of
these global variables defining the metadata. It defines variable names such
as <tt>sr23</tt> for a 3D short dimensioned <tt>UNLIMITED</tt> (record) x 2 x
3. The maximum dimension size is 4. There is a vector of length 1, 2, 3 and
4 for each data type. However for ranks 2 and 3 (<tt>MAX_RANK</tt>) there is
only one type for each shape, but there is an example of each possible shape
(including one with the <tt>UNLIMITED</tt> dimension). This produces a total
of 136 (<tt>NVARS</tt>) variables.
<p>
The command
<pre>
ncdump -h test.nc | grep char
</pre>
produces the following list of all the character variables:
<pre>
char c ;
char cr(Dr) ;
char c1(D1) ;
char c2(D2) ;
char c3(D3) ;
char c4(D4) ;
char cr1(Dr, D1) ;
char c13(D1, D3) ;
char c31(D3, D1) ;
char c43(D4, D3) ;
char cr21(Dr, D2, D1) ;
char cr33(Dr, D3, D3) ;
char c111(D1, D1, D1) ;
char c123(D1, D2, D3) ;
char c141(D1, D4, D1) ;
char c213(D2, D1, D3) ;
char c231(D2, D3, D1) ;
char c243(D2, D4, D3) ;
char c321(D3, D2, D1) ;
char c333(D3, D3, D3) ;
char c411(D4, D1, D1) ;
char c423(D4, D2, D3) ;
char c441(D4, D4, D1) ;
</pre>
Given that there are two records (<tt>Dr</tt> = 2 currently), one can use this
to calculate the number of character elements as:
1+2+1+2+3+4+2+3+3+12+4+18+1+6+4+6+6+24+6+27+4+24+16 = 179. This corresponds
to the number of comparisons for characters given in output such as:
<pre>
*** Testing nc_get_vara_text ... 179 good comparisons. ok
</pre>
The number of comparisons for numeric types is more difficult to calculate
because it depends on whether the data values are within the ranges for the
external and internal types. However one can do quite a number of consistency
checks. For example the number of comparisons done by the <tt>(void *)</tt>
functions (e.g. <tt>nc_get_var1</tt>) is 1386 which equals the sum of that
for <tt>nc_get_var1_double</tt> (1207, the total number of numeric elements)
and the above value of 179.
<p>
There is one global attribute of each data-type and lengths vary from 1 to 6.
Only the six scalar variables have attributes and their lengths vary from 0 to
5.
<h3>
GENERATION OF TEST DATA VALUES
</h3>
Variable and attribute data values are generated by function <tt>hash</tt> in
file <tt>util.c</tt>. The type <tt>double</tt> result is a function of the
data-type, rank (-1 for attribute) and index vector.
<p>
Vectors are treated specially. Index 0 gives the minimum for the type and
index 1 gives the maximum. For types other than <tt>NC_CHAR</tt> and
<tt>NC_DOUBLE</tt>, index 2 gives a value just less than the minimum and index
3 gives a value just more than the maximum.
<p>
Otherwise <tt>hash</tt> uses an algorithm which always generates a valid value
for the type.
<p>
There is also a 4-argument wrapper function <tt>hash4</tt> which has an
additional argument specifying whether the internal type is <tt>uchar</tt>.
This handles the special <tt>NC_BYTE/uchar</tt> adjustment.
<h3>
ncdump of <tt>test.nc</tt>
</h3>
The following is the start of output from <tt>ncdump test.nc</tt>:
<pre>
netcdf test {
dimensions:
Dr = UNLIMITED ; // (2 currently)
D1 = 1 ;
D2 = 2 ;
D3 = 3 ;
D4 = 4 ;
variables:
char c ;
byte b ;
b:c = "" ;
short s ;
s:b = '\200' ;
s:s = -32768s, 32767s ;
long l ;
l:l = -2147483648, 2147483647, 2147483647 ;
l:f = -3.4028235e+38f, 3.4028235e+38f, -3.4028235e+38f, 3.4028235e+38f ;
l:d = -1.797693134862316e+308, 1.797693134862316e+308, -1., 1., 660. ;
float f ;
double d ;
d:c = "<22>AZ$&" ;
char cr(Dr) ;
byte br(Dr) ;
short sr(Dr) ;
long lr(Dr) ;
float fr(Dr) ;
double dr(Dr) ;
char c1(D1) ;
byte b1(D1) ;
short s1(D1) ;
long l1(D1) ;
float f1(D1) ;
double d1(D1) ;
char c2(D2) ;
byte b2(D2) ;
short s2(D2) ;
long l2(D2) ;
float f2(D2) ;
double d2(D2) ;
char c3(D3) ;
byte b3(D3) ;
short s3(D3) ;
long l3(D3) ;
float f3(D3) ;
double d3(D3) ;
char c4(D4) ;
byte b4(D4) ;
short s4(D4) ;
long l4(D4) ;
float f4(D4) ;
double d4(D4) ;
char cr1(Dr, D1) ;
byte br2(Dr, D2) ;
short sr3(Dr, D3) ;
long lr4(Dr, D4) ;
float f11(D1, D1) ;
double d12(D1, D2) ;
char c13(D1, D3) ;
byte b14(D1, D4) ;
short s21(D2, D1) ;
long l22(D2, D2) ;
float f23(D2, D3) ;
double d24(D2, D4) ;
char c31(D3, D1) ;
byte b32(D3, D2) ;
short s33(D3, D3) ;
long l34(D3, D4) ;
float f41(D4, D1) ;
double d42(D4, D2) ;
char c43(D4, D3) ;
byte b44(D4, D4) ;
short sr11(Dr, D1, D1) ;
long lr12(Dr, D1, D2) ;
float fr13(Dr, D1, D3) ;
double dr14(Dr, D1, D4) ;
char cr21(Dr, D2, D1) ;
byte br22(Dr, D2, D2) ;
short sr23(Dr, D2, D3) ;
long lr24(Dr, D2, D4) ;
float fr31(Dr, D3, D1) ;
double dr32(Dr, D3, D2) ;
char cr33(Dr, D3, D3) ;
byte br34(Dr, D3, D4) ;
short sr41(Dr, D4, D1) ;
long lr42(Dr, D4, D2) ;
float fr43(Dr, D4, D3) ;
double dr44(Dr, D4, D4) ;
char c111(D1, D1, D1) ;
byte b112(D1, D1, D2) ;
short s113(D1, D1, D3) ;
long l114(D1, D1, D4) ;
float f121(D1, D2, D1) ;
double d122(D1, D2, D2) ;
char c123(D1, D2, D3) ;
byte b124(D1, D2, D4) ;
short s131(D1, D3, D1) ;
long l132(D1, D3, D2) ;
float f133(D1, D3, D3) ;
double d134(D1, D3, D4) ;
char c141(D1, D4, D1) ;
byte b142(D1, D4, D2) ;
short s143(D1, D4, D3) ;
long l144(D1, D4, D4) ;
float f211(D2, D1, D1) ;
double d212(D2, D1, D2) ;
char c213(D2, D1, D3) ;
byte b214(D2, D1, D4) ;
short s221(D2, D2, D1) ;
long l222(D2, D2, D2) ;
float f223(D2, D2, D3) ;
double d224(D2, D2, D4) ;
char c231(D2, D3, D1) ;
byte b232(D2, D3, D2) ;
short s233(D2, D3, D3) ;
long l234(D2, D3, D4) ;
float f241(D2, D4, D1) ;
double d242(D2, D4, D2) ;
char c243(D2, D4, D3) ;
byte b244(D2, D4, D4) ;
short s311(D3, D1, D1) ;
long l312(D3, D1, D2) ;
float f313(D3, D1, D3) ;
double d314(D3, D1, D4) ;
char c321(D3, D2, D1) ;
byte b322(D3, D2, D2) ;
short s323(D3, D2, D3) ;
long l324(D3, D2, D4) ;
float f331(D3, D3, D1) ;
double d332(D3, D3, D2) ;
char c333(D3, D3, D3) ;
byte b334(D3, D3, D4) ;
short s341(D3, D4, D1) ;
long l342(D3, D4, D2) ;
float f343(D3, D4, D3) ;
double d344(D3, D4, D4) ;
char c411(D4, D1, D1) ;
byte b412(D4, D1, D2) ;
short s413(D4, D1, D3) ;
long l414(D4, D1, D4) ;
float f421(D4, D2, D1) ;
double d422(D4, D2, D2) ;
char c423(D4, D2, D3) ;
byte b424(D4, D2, D4) ;
short s431(D4, D3, D1) ;
long l432(D4, D3, D2) ;
float f433(D4, D3, D3) ;
double d434(D4, D3, D4) ;
char c441(D4, D4, D1) ;
byte b442(D4, D4, D2) ;
short s443(D4, D4, D3) ;
long l444(D4, D4, D4) ;
// global attributes:
:Gc = "" ;
:Gb = '\200', '\177' ;
:Gs = -32768s, 32767s, 32767s ;
:Gl = -2147483648, 2147483647, 2147483647, 2147483647 ;
:Gf = -3.4028235e+38f, 3.4028235e+38f, -3.4028235e+38f, 3.4028235e+38f, 531.f ;
:Gd = -1.797693134862316e+308, 1.797693134862316e+308, -1., 1., 660., 650. ;
data:
c = "\002" ;
b = 254 ;
s = -5 ;
l = -20 ;
f = -9 ;
d = -10 ;
cr = "\000\377" ;
br = 128, 127 ;
sr = -32768, 32767 ;
lr = -2147483648, 2147483647 ;
fr = -3.402823e+38 , 3.402823e+38 ;
dr = -1.79769313486232e+308, 1.79769313486232e+308 ;
c1 = "" ;
b1 = 128 ;
s1 = -32768 ;
</pre>
<h3>
HANDLING OF FAILURES
</h3>
The macro <tt>IF</tt> is used in place of an ordinary <tt>if</tt> as in:
<pre>
IF (err)
error("nc_open: %s", nc_strerror(err));
</pre>
<tt>IF</tt> acts like an ordinary <tt>if</tt> except for two side effects.
Firstly, it prints a failure message. Secondly, it increments the global
variable <tt>nfails</tt>, which contains a count of the failures in the
current test. The true condition should always correspond to a failure.
There can be an <tt>else</tt>.
<p>
The macro <tt>IF</tt> is defined in file <tt>error.h</tt> as follows:
<pre>
#define IF(EXPR) if (ifFail(EXPR, __LINE__, __FILE__))
</pre>
The function <tt>ifFail</tt> is defined in file <tt>error.c</tt> as follows:
<pre>
int
ifFail(const int expr, const int line, const char *file)
{
if (expr) {
++nfails;
error("\n\tFAILURE at line %d of %s: ", line, file);
}
return expr;
}
</pre>
<p>
The driver calls each test function and prints summary messages using the
following macro <tt>NC_TEST</tt> defined in file <tt>nc_test.c</tt>:
<pre>
#define NC_TEST(func) \
print( "*** Testing " #func " ... ");\
nfails = 0;\
test_ ## func();\
nfailsTotal += nfails;\
if (verbose) \
print("\n"); \
if ( nfails == 0) \
print( "ok\n");\
else\
print( "\n\t### %d FAILURES TESTING %s! ###\n", nfails, #func)
</pre>
<h3>
INDIVIDUAL TESTS
</h3>
The file <tt>scratch.nc</tt> is created at the start, and deleted at the end,
of each test that uses it.
<p>
An attempt has been made to generate all possible errors (including all
possible illegal argument values) and check their error status values. Where
some arguments could be <tt>NULL</tt>, this <tt>NULL</tt> alternative is
typically chosen for odd variable ID values.
<p>
In most cases the get/put tests read/write each variable/attribute element
exactly once.
<p>
The <tt>get_vara/put_vara</tt> tests split each dimension at a randomly
selected point. So a matrix is divided into four sub-matrices.
<p>
The <tt>get_vars/put_vars</tt> tests do the same splitting, but also randomly
select a stride vector for each sub-array. The whole sub-array is covered by
repeating with different start-index-vectors. (The number of different starts
is the product of the elements of the stride vector.)
<p>
The buffer for get/put att, <tt>var1</tt>, <tt>vara</tt> and <tt>vars</tt> is
used for only a single get/put. However the buffer for <tt>varm</tt> tests is
a bit-image of the external variable and each get/put transfers some elements,
filling in the gaps. Otherwise, these <tt>varm</tt> tests are similar to the
<tt>vars</tt> tests.
<h3>
TESTING DONE
</h3>
Tests were run on slim, binnie and laraine. These tests consisted of:
<pre>
nc_test -c # in writable directory
nc_test # in writable directory
nc_test -r # in read-only directory
nc_test # in read-only directory
</pre>
<p>
A full test on slim gave the following timings:
<pre>
real 0m17.67s
user 0m7.81s
sys 0m1.15s
</pre>
<h3>
CURRENT STATUS AND WORK REMAINING
</h3>
Currently there are 95 tests implemented. There are 32 tests not yet
implemented, corresponding to library functions have not yet been
implemented. These correspond to the eight types for each of
<tt>nc_get_vars_</tt><i>TYPE</I>, <tt>nc_get_varm_</tt><i>TYPE</I>,
<tt>nc_put_vars_</tt><i>TYPE</I>, <tt>nc_put_varm_</tt><i>TYPE</I>. However
these will all be generated by four <tt>m4</tt> macros, which will be similar
to the existing <tt>nc_get_vars</tt>, <tt>nc_get_varm</tt>,
<tt>nc_put_vars</tt>, <tt>nc_put_varm</tt>.
<p>
The code to test negative strides is currently commented out, because this
feature has not yet been implemented in the library.
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink