mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2025-03-25 17:40:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix typo in configure.ac. Also, if we have to have multiple copies of

Browse Source
This commit is contained in:
Russ Rew 2012-05-09 17:21:32 +00:00
parent b47d95b121
commit f638446d00
7 changed files with 1008 additions and 579 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
configure.ac
View File

@ -587,7 +587,7 @@ AC_MSG_RESULT($enable_mmap)
# check for mmap and mremap
AC_FUNC_MMAP
AC_CHECK_FUNCS([mremap])
if "x$ac_cv_func_mmap_fixed_mapped" != xyes -o "x$ac_cv_func_mremap" != xyes ; then
if test "x$ac_cv_func_mmap_fixed_mapped" != xyes -o "x$ac_cv_func_mremap" != xyes ; then
echo "mmap function or mremap function is not available: disabling mmap"
enable_mmap=no
fi

3
man4/Doxyfile.in
View File

@ -612,7 +612,8 @@ INPUT = \
@abs_top_srcdir@/libdispatch/dv2i.c \
@abs_top_srcdir@/libdispatch/nc4.c \
@abs_top_srcdir@/libsrc4/nc4file.c \
@abs_top_srcdir@/ncdump/ncdump.c
@abs_top_srcdir@/ncdump/ncdump.c \
@abs_top_srcdir@/ncdump/nccopy.c
# This tag can be used to specify the character encoding of the source files
# that doxygen parses. Internally doxygen uses the UTF-8 encoding, which is

612
man4/guide.doc
View File

@ -3194,78 +3194,85 @@ dataset:
\page guide_ncdump ncdump
The ncdump tool generates the CDL text representation of a netCDF
dataset on standard output, optionally excluding some or all of the
variable data in the output. The output from ncdump is intended to be
acceptable as input to ncgen. Thus ncdump and ncgen can be used as
inverses to transform data representation between binary and text
representations.
The \b ncdump utility generates a text representation of a specified
netCDF file on standard output, optionally excluding some or all of
the variable data in the output. The text representation is in a form
called CDL (network Common Data form Language) that can be viewed,
edited, or serve as input to \b ncgen, a companion program that can
generate a binary netCDF file from a CDL file. Hence \b ncgen and \b
ncdump can be used as inverses to transform the data representation
between binary and text representations. See \b ncgen documentation
for a description of CDL and netCDF representations.
As of netCDF version 4.1, ncdump can also access DAP data sources if
DAP support is enabled in the underlying netCDF library. Instead of
specifying a file name as argument to ncdump, the user specifies a URL
to a DAP source.
\b ncdump may also be used to determine what kind of netCDF file
is used (which variant of the netCDF file format) with the -k
option.
ncdump may also be used as a simple browser for netCDF datasets, to
display the dimension names and lengths; variable names, types, and
If DAP support was enabled when \b ncdump was built, the file name may
specify a DAP URL. This allows \b ncdump to access data sources from
DAP servers, including data in other formats than netCDF. When used
with DAP URLs, \b ncdump shows the translation from the DAP data
model to the netCDF data model.
\b ncdump may also be used as a simple browser for netCDF data files,
to display the dimension names and lengths; variable names, types, and
shapes; attribute names and values; and optionally, the values of data
for all variables or selected variables in a netCDF dataset.
for all variables or selected variables in a netCDF file. For
netCDF-4 files, groups and user-defined types are also included in \b
ncdump output.
ncdump defines a default format used for each type of netCDF variable
data, but this can be overridden if a C_format attribute is defined
for a netCDF variable. In this case, ncdump will use the C_format
attribute to format values for that variable. For example, if
floating-point data for the netCDF variable Z is known to be accurate
to only three significant digits, it might be appropriate to use this
variable attribute:
\b ncdump uses '_' to represent data values that are equal to the
'_FillValue' attribute for a variable, intended to represent
data that has not yet been written. If a variable has no
'_FillValue' attribute, the default fill value for the variable
type is used unless the variable is of byte type.
\b ncdump defines a default display format used for each type of
netCDF data, but this can be changed if a `C_format' attribute
is defined for a netCDF variable. In this case, \b ncdump will
use the `C_format' attribute to format each value. For
example, if floating-point data for the netCDF variable `Z' is
known to be accurate to only three significant digits, it would
be appropriate to use the variable attribute
\code
Z:C_format = "%.3g"
Z:C_format = "%.3g"
\endcode
Ncdump uses '_' to represent data values that are equal to the
_FillValue attribute for a variable, intended to represent data that
has not yet been written. If a variable has no _FillValue attribute,
the default fill value for the variable type is used unless the
variable is of byte type.
@section OPTIONS
UNIX syntax for invoking ncdump:
@par -c
Show the values of \e coordinate \e variables (1D variables with the
same names as dimensions) as well as the declarations of all
dimensions, variables, attribute values, groups, and user-defined
types. Data values of non-coordinate variables are not included in
the output. This is usually the most suitable option to use for a
brief look at the structure and contents of a netCDF file.
\code
ncdump [ -c | -h] [-v var1,...] [-b lang] [-f lang]
[-l len] [ -p fdig[,ddig]] [ -s ] [ -n name] [input-file]
\endcode
where:
<pre>
-c
Show the values of coordinate variables (variables that are also
dimensions) as well as the declarations of all dimensions, variables,
and attribute values. Data values of non-coordinate variables are not
included in the output. This is often the most suitable option to use
for a brief look at the structure and contents of a netCDF dataset.
-h
@par -h
Show only the header information in the output, that is, output only
the declarations for the netCDF dimensions, variables, and attributes
of the input file, but no data values for any variables. The output is
identical to using the '-c' option except that the values of
coordinate variables are not included. (At most one of '-c' or '-h'
options may be present.)
the declarations for the netCDF dimensions, variables, attributes,
groups, and user-defined types of the input file, but no data values
for any variables. The output is identical to using the '-c' option
except that the values of coordinate variables are not included. (At
most one of '-c' or '-h' options may be present.)
-v var1,...
@par -v \a var1,...
@par
The output will include data values for the specified variables, in
addition to the declarations of all dimensions, variables, and
attributes. One or more variables must be specified by name in the
comma-delimited list following this option. The list must be a single
argument to the command, hence cannot contain blanks or other white
space characters. The named variables must be valid netCDF variables
in the input-file. The default, without this option and in the absence
of the '-c' or '-h' options, is to include data values for all
variables in the output.
argument to the command, hence cannot contain unescaped blanks or
other white space characters. The named variables must be valid netCDF
variables in the input-file. A variable within a group in a netCDF-4
file may be specified with an absolute path name, such as
`/GroupA/GroupA2/var'. Use of a relative path name such as `var' or
`grp/var' specifies all matching variable names in the file. The
default, without this option and in the absence of the '-c' or '-h'
options, is to include data values for \e all variables in the output.
-b lang
@par -b [c|f]
A brief annotation in the form of a CDL comment (text beginning with
the characters '//') will be included in the data section of the
output for each 'row' of data, to help identify data values for
@ -3277,30 +3284,46 @@ varying fastest). In either case, the data will be presented in the
same order; only the annotations will differ. This option may be
useful for browsing through large volumes of multidimensional data.
-f lang
@par -f [c|f]
Full annotations in the form of trailing CDL comments (text beginning
with the characters '//') for every data value (except individual
characters in character arrays) will be included in the data
section. If lang begins with 'C' or 'c', then C language conventions
will be used (zero-based indices, last dimension varying fastest). If
lang begins with 'F' or 'f', then FORTRAN language conventions will be
used (one-based indices, first dimension varying fastest). In either
case, the data will be presented in the same order; only the
annotations will differ. This option may be useful for piping data
into other filters, since each data value appears on a separate line,
fully identified. (At most one of '-b' or '-f' options may be
present.)
will be used. If lang begins with 'F' or 'f', then FORTRAN language
conventions will be used. In either case, the data will be presented
in the same order; only the annotations will differ. This option may
be useful for piping data into other filters, since each data value
appears on a separate line, fully identified. (At most one of '-b' or
'-f' options may be present.)
-l len
@par -l \e length
@par
Changes the default maximum line length (80) used in formatting lists
of non-character data values.
-p float_digits[,double_digits]
@par -n \e name
@par
CDL requires a name for a netCDF file, for use by 'ncgen -b' in
generating a default netCDF file name. By default, \b ncdump
constructs this name from the last component of the file name of
the input netCDF file by stripping off any extension it has. Use
the '-n' option to specify a different name. Although the output
file name used by 'ncgen -b' can be specified, it may be wise to
have \b ncdump change the default name to avoid inadvertently
overwriting a valuable netCDF file when using \b ncdump, editing the
resulting CDL file, and using 'ncgen -b' to generate a new netCDF
file from the edited CDL file.
@par -p \e float_digits[, \e double_digits ]
@par
Specifies default precision (number of significant digits) to use in
displaying floating-point or double precision data values for
attributes and variables. If specified, this value overrides the value
of the C_format attribute, if any, for a variable. Floating-point data
will be displayed with float_digits significant digits. If
will be displayed with \e float_digits significant digits. If \e
double_digits is also specified, double-precision values will be
displayed with that many significant digits. In the absence of any
'-p' specifications, floating-point and double-precision data are
@ -3308,271 +3331,364 @@ displayed with 7 and 15 significant digits respectively. CDL files can
be made smaller if less precision is required. If both floating-point
and double precisions are specified, the two values must appear
separated by a comma (no blanks) as a single argument to the command.
(To represent every last bit of precision in a CDL file for all
possible floating-point values would require '-p 9,17'.)
-n name
CDL requires a name for a netCDF dataset, for use by 'ncgen -b' in
generating a default netCDF dataset name. By default, ncdump
constructs this name from the last component of the file name of the
input netCDF dataset by stripping off any extension it has. Use the
'-n' option to specify a different name. Although the output file name
used by 'ncgen -b' can be specified, it may be wise to have ncdump
change the default name to avoid inadvertently overwriting a valuable
netCDF dataset when using ncdump, editing the resulting CDL file, and
using 'ncgen -b' to generate a new netCDF dataset from the edited CDL
file.
@par -k
Show \e kind of netCDF file, that is which format variant the file uses.
Other options are ignored if this option is specified. Output will be
one of 'classic'. '64-bit offset', 'netCDF-4', or 'netCDF-4 classic
model'.
-s
Specifies that special virtual attributes should be output for the
file format variant and for variable properties such as compression,
chunking, and other properties specific to the format implementation
that are primarily related to performance rather than the logical
schema of the data. All the special virtual attributes begin with '_'
followed by an upper-case letter. Currently they include the global
attribute “_Format” and the variable attributes “_Fletcher32”,
“_ChunkSizes”, “_Endianness”, “_DeflateLevel”, “_Shuffle”, “_Storage”,
and “_NoFill”. The ncgen utility recognizes these attributes and
@par -s
Specifies that \e special virtual attributes should be output for the
file format variant and for variable properties such as
compression, chunking, and other properties specific to the format
implementation that are primarily related to performance rather
than the logical schema of the data. All the special virtual
attributes begin with '_' followed by an upper-case
letter. Currently they include the global attribute '_Format' and
the variable attributes '_ChunkSizes', '_DeflateLevel',
'_Endianness', '_Fletcher32', '_NoFill', '_Shuffle', and '_Storage'.
The \b ncgen utility recognizes these attributes and
supports them appropriately.
-t
@par -t
Controls display of time data, if stored in a variable that uses a
udunits compliant time representation such as “days since 1970-01-01”
or “seconds since 2009-03-15 12:01:17”. If this option is specified,
time values are displayed as human-readable date-time strings rather
than numerical values, interpreted in terms of a “calendar” variable
attribute, if specified. Calendar attribute values interpreted with
this option include the CF Conventions values “gregorian” or
“standard”, “proleptic_gregorian”, “noleap” or “365_day”, “all_leap”
or “366_day”, “360_day”, and “julian”.
</pre>
udunits compliant time representation such as 'days since 1970-01-01'
or 'seconds since 2009-03-15 12:01:17'. If this option is specified,
time values are displayed as a human-readable date-time strings rather
than numerical values, interpreted in terms of a 'calendar' variable
attribute, if specified. For numeric attributes of time variables,
the human-readable time value is displayed after the actual value, in
an associated CDL comment. Calendar attribute values interpreted with
this option include the CF Conventions values 'gregorian' or
'standard', 'proleptic_gregorian', 'noleap' or '365_day', 'all_leap'
or '366_day', '360_day', and 'julian'.
<h1>Examples</h1>
@par -i
Same as the '-t' option, except output time data as date-time strings
with ISO-8601 standard 'T' separator, instead of a blank.
Look at the structure of the data in the netCDF dataset foo.nc:
@par -g \e grp1,...
@par
The output will include data values only for the specified groups.
One or more groups must be specified by name in the comma-delimited
list following this option. The list must be a single argument to the
command. The named groups must be valid netCDF groups in the
input-file. The default, without this option and in the absence of the
'-c' or '-h' options, is to include data values for all groups in the
output.
@par -w
For file names that request remote access using DAP URLs, access data
with client-side caching of entire variables.
@par -x
Output XML (NcML) instead of CDL. The NcML does not include data values.
The NcML output option currently only works for netCDF classic model data.
@section EXAMPLES
Look at the structure of the data in the netCDF file foo.nc:
\code
ncdump -c foo.nc
ncdump -c foo.nc
\endcode
Produce an annotated CDL version of the structure and data in the
netCDF dataset foo.nc, using C-style indexing for the annotations:
netCDF file foo.nc, using C-style indexing for the annotations:
\code
ncdump -b c foo.nc > foo.cdl
ncdump -b c foo.nc > foo.cdl
\endcode
Output data for only the variables uwind and vwind from the netCDF
dataset foo.nc, and show the floating-point data with only three
file foo.nc, and show the floating-point data with only three
significant digits of precision:
\code
ncdump -v uwind,vwind -p 3 foo.nc
ncdump -v uwind,vwind -p 3 foo.nc
\endcode
Produce a fully-annotated (one data value per line) listing of the
data for the variable omega, using FORTRAN conventions for indices,
and changing the netCDF dataset name in the resulting CDL file to
and changing the netCDF file name in the resulting CDL file to
omega:
\code
ncdump -v omega -f fortran -n omega foo.nc > Z.cdl
ncdump -v omega -f fortran -n omega foo.nc > Z.cdl
\endcode
Examine the translated DDS for the DAP source from the specified URL.
Examine the translated DDS for the DAP source from the specified URL:
\code
ncdump -h http://test.opendap.org:8080/dods/dts/test.01
ncdump -h http://test.opendap.org:8080/dods/dts/test.01
\endcode
Without dumping all the data, show the special virtual attributes that indicate
performance-related characterisitics of a netCDF-4 file:
\code
ncdump -h -s nc4file.nc
\endcode
@section see_also SEE ALSO
ncgen(1), netcdf(3)
@section string_note NOTE ON STRING OUTPUT
For classic, 64-bit offset or netCDF-4 classic model data, \b ncdump
generates line breaks after embedded newlines in displaying character
data. This is not done for netCDF-4 files, because netCDF-4 supports
arrays of real strings of varying length.
\page guide_nccopy nccopy
The nccopy utility copies an input netCDF file to an output netCDF
file, in any of the four format variants, if possible. For example, if
built with the netCDF-3 library, a netCDF classic file may be copied
to a netCDF 64-bit offset file, permitting larger variables. If built
with the netCDF-4 library, a netCDF classic file may be copied to a
netCDF-4 file or to a netCDF-4 classic model file as well, permitting
data compression, data chunking, efficient schema changes, larger
variable sizes, and use of other netCDF-4 features in case the output
uses the enhanced netCDF model.
The \b nccopy utility copies an input netCDF file in any supported
format variant to an output netCDF file, optionally converting the
output to any compatible netCDF format variant, compressing the data,
or rechunking the data. For example, if built with the netCDF-3
library, a netCDF classic file may be copied to a netCDF 64-bit offset
file, permitting larger variables. If built with the netCDF-4
library, a netCDF classic file may be copied to a netCDF-4 file or to
a netCDF-4 classic model file as well, permitting data compression,
efficient schema changes, larger variable sizes, and use of other
netCDF-4 features.
The nccopy source also serves as an example of a generic netCDF-4
program, with its ability to read any valid netCDF file and handle
strings, arbitrarily nested groups, any user-defined types, (including
arbitrarily nested compound types and variable-length types), and data
of any valid netCDF-4 type. Functions from and fragments of the nccopy
source may be useful to other developers for writing generic programs
that handle or write arbitrary netCDF data.
\b nccopy also serves as an example of a generic netCDF-4 program,
with its ability to read any valid netCDF file and handle nested
groups, strings, and user-defined types, including arbitrarily
nested compound types, variable-length types, and data of any valid
netCDF-4 type.
As of netCDF version 4.1, and if DAP support was enabled when nccopy
was built, the input file name on which nccopy is invoked may specify
a DAP URL. This allows nccopy to convert data on DAP servers to local
netCDF files.
If DAP support was enabled when \b nccopy was built, the file name may
specify a DAP URL. This may be used to convert data on DAP servers to
local netCDF files.
UNIX syntax for invoking nccopy:
\code
nccopy [ -k kind] [ -d n] [ -s ] [ -u ] [ -c chunkspec ]
[ -m bufsize] input output
nccopy [-k kind] [-d n] [-s] [-u] [-w] [-c chunkspec] [-m bufsize]
[-h chunk_cache] [-e cache_elems] [-r] infile outfile
\endcode
where:
@par -k \e kind
Specifies the kind of file to be created (that is, the format variant)
and, by inference, the data model (i.e. netcdf-3 (classic) versus
netcdf-4 (enhanced)). The possible arguments are as follows. \n
'1' or 'classic' => netCDF classic format \n
'2', '64-bit-offset', or '64-bit offset' => netCDF 64-bit format \n
'3', 'hdf5', 'netCDF-4', or 'enhanced' => netCDF-4 format (enhanced data model) \n
'4', 'hdf5-nc3', 'netCDF-4 classic model', or 'enhanced-nc3' => netCDF-4 classic model format \n
<pre>
-k kind
Specifies the kind of file to be created and, by inference, the data
model (i.e. netcdf-3 (classic) versus netcdf-4 (enhanced)). The
possible arguments are as follows.
“1” or “classic” write a netCDF classic file format file
“2”, “64-bit-offset”, or “64-bit offset” write a netCDF 64 bit classic file format file
“3”, “hdf5”, “netCDF-4”, or “enhanced” write a netCDF-4 format file
“4”, “hdf5-nc3”, “netCDF-4 classic model”, or “enhanced-nc3”
write a netCDF-4 classic model file format, which is
restricted to netCDF-3 classic data model but may use
netCDF-4/HDF5 features such as compression and chunking.
@par
If no value for -k is specified, then the output will use the same
format as the input. However, if the input is classic or 64-bit offset
and a compression level is specified with the -d option or chunking is
specified with the -c option, then the output will be netCDF-4 classic
model format. Note that attempting some kinds of format conversion
will result in an error, if the conversion is not possible. For
example, an attempt to copy a netCDF-4 file that uses features of the
enhanced model to any of the other kinds of netCDF formats that use
the classic model will result in an error.
format as the input, except if the input is classic or 64-bit offset
and either chunking or compression is specified, in which case the
output will be netCDF-4 classic model format. Note that attempting
some kinds of format conversion will result in an error, if the
conversion is not possible. For example, an attempt to copy a
netCDF-4 file that uses features of the enhanced model, such as groups
or variable-length strings, to any of the other kinds of netCDF
formats that use the classic model will result in an error.
-d n
Specify deflation level (level of compression) for variable data in
output. 0 corresponds to no compression and 9 to maximum compression,
with higher levels of compression requiring marginally more time to
compress or uncompress than lower levels. Compression achieved may
also depend on chunking parameters, which will use default chunking in
the current nccopy implementation. If this option is specified for a
classic format or 64-bit offset format input file, it is not necessary
to also specify that the output should be netCDF-4 classic model, as
that will be the default. If this option is not specified and the
input file has compressed variables, the compression will still be
preserved in the output, using the same chunking as in the input
unless the -c option is used to specify different chunking.
@par -d \e n
For netCDF-4 output, including netCDF-4 classic model, specify
deflation level (level of compression) for variable data output. 0
corresponds to no compression and 9 to maximum compression, with
higher levels of compression requiring marginally more time to
compress or uncompress than lower levels. Compression achieved may
also depend on output chunking parameters. If this option is
specified for a classic format or 64-bit offset format input file, it
is not necessary to also specify that the output should be netCDF-4
classic model, as that will be the default. If this option is not
specified and the input file has compressed variables, the compression
will still be preserved in the output, using the same chunking as in
the input by default.
Note that nccopy requires all variables to be compressed using the
same compression level, but the API has no such restriction. With a
@par
Note that \b nccopy requires all variables to be compressed using the
same compression level, but the API has no such restriction. With a
program you can customize compression for each variable independently.
-s
Specify shuffling of variable data bytes before compression or after
decompression. This option is ignored unless a non-zero deflation
level is specified. Turning shuffling on sometimes improves
@par -s
For netCDF-4 output, including netCDF-4 classic model,
specify shuffling of variable data bytes before compression or after
decompression. This option is ignored unless a non-zero deflation
level is specified. Turning shuffling on sometimes improves
compression.
-u
@par -u
Convert any unlimited size dimensions in the input to fixed size
dimensions in the output.
dimensions in the output. This can speed up variable-at-a-time
access, but slow down record-at-a-time access to multiple variables
along an unlimited dimension.
-c chunkspec
Specify chunking (multidimensional tiling) for variable data in the
output, useful to specify the units of disk access, compression, or
other filters such as checksums. The chunkspec argument is a string of
comma-separated associations, each specifying a dimension name, a `/'
character, and optionally the corresponding chunk length for that
dimension. No blanks should appear in the chunkspec string, except
possibly escaped blanks that are part of a dimension name. A chunkspec
must name at least one dimension, and may omit dimensions which are
not to be chunked or for which the default chunk length is desired. If
a dimension name is followed by a `/' character but no subsequent
chunk length, the actual dimension length is assumed. If copying a
classic model file to a netCDF-4 output file and not naming all
dimensions in the chunkspec, unnamed dimensions will also use the
actual dimension length for the chunk length.
@par -w
Keep output in memory (as a diskless netCDF file) until output is
closed, at which time output file is written to disk. This can
greatly speedup operations such as converting unlimited dimension to
fixed size (-u option), chunking, rechunking, or compressing the
input. It requires that available memory is large enough to hold the
output file. This option may provide a larger speedup than careful
tuning of the -m, -h, or -e options, and it's certainly a lot simpler.
By default, fixed-size variables smaller than 1024 bytes of data will
not be chunked even if they use dimensions mentioned in the chunkspec
string.
@par -c \e chunkspec
@par
For netCDF-4 output, including netCDF-4 classic model, specify
chunking (multidimensional tiling) for variable data in the output.
This is useful to specify the units of disk access, compression, or
other filters such as checksums. Changing the chunking in a netCDF
file can also greatly speedup access, by choosing chunk shapes that
are appropriate for the most common access patterns.
An example of a chunkspec for variables that use the `m' and `n'
dimensions might be `m/100,n/200' to specify 100 by 200 chunks. To see
the chunking resulting from copying with a chunkspec, use ncdump with
the `-s' option (and typically with the `-h' option) on the output
file.
@par
The chunkspec argument is a string of comma-separated associations,
each specifying a dimension name, a '/' character, and optionally the
corresponding chunk length for that dimension. No blanks should
appear in the chunkspec string, except possibly escaped blanks that
are part of a dimension name. A chunkspec must name at least one
dimension, and may omit dimensions which are not to be chunked or for
which the default chunk length is desired. If a dimension name is
followed by a '/' character but no subsequent chunk length, the actual
dimension length is assumed. If copying a classic model file to a
netCDF-4 output file and not naming all dimensions in the chunkspec,
unnamed dimensions will also use the actual dimension length for the
chunk length. An example of a chunkspec for variables that use
'm' and 'n' dimensions might be 'm/100,n/200' to specify 100 by 200
chunks. To see the chunking resulting from copying with a chunkspec,
use the '-s' option of ncdump on the output file.
Note that nccopy requires variables that share a dimension to also
share the chunk size associated with that dimension, but the API has
no such restriction. With a program you can customize chunking for
each variable independently.
@par
Note that \b nccopy requires variables that share a dimension to also
share the chunk size associated with that dimension, but the
programming interface has no such restriction. If you need to
customize chunking for variables independently, you will need to use
the library API in a custom utility program.
-m bufsize
Specifies the size, in bytes, of the copy buffer used to to copy large
variables, by copying them in smaller pieces, each no larger than
bufsize. A suffix of k, m, or g multiplies the copy buffer size by one
thousand, million, or billion, respectively. The default is 5 million
bytes, but will be increased if necessary to hold at least one chunk
of netCDF-4 chunked variables in the input file. You may want to
specify a value larger than the default for OPeNDAP copies of large
files over high latency networks.
</pre>
@par -m \e bufsize
@par
An integer or floating-point number that specifies the size, in bytes,
of the copy buffer used to copy large variables. A suffix of K, M, G,
or T multiplies the copy buffer size by one thousand, million,
billion, or trillion, respectively. The default is 5 Mbytes,
but will be increased if necessary to hold at least one chunk of
netCDF-4 chunked variables in the input file. You may want to specify
a value larger than the default for copying large files over high
latency networks. Using the '-w' option may provide better
performance, if the output fits in memory.
<h1>Examples</h1>
@par -e \e chunk_cache
@par
For netCDF-4 output, including netCDF-4 classic model, an integer or
floating-point number that specifies the size in bytes of chunk cache
for chunked variables. This is not a property of the file, but merely
a performance tuning parameter for avoiding compressing or
decompressing the same data multiple times while copying and changing
chunk shapes. A suffix of K, M, G, or T multiplies the chunk cache
size by one thousand, million, billion, or trillion, respectively.
The default is 4.194304 Mbytes (or whatever was specified for the
configure-time constant CHUNK_CACHE_SIZE when the netCDF library was
built). Ideally, the \b nccopy utility should accept only one memory
buffer size and divide it optimally between a copy buffer and chunk
cache, but no general algorithm for computing the optimum chunk cache
size has been implemented yet. Using the '-w' option may provide
better performance, if the output fits in memory.
Simple copy, check for corruption or truncation
@par -h \e cache_elems
@par
For netCDF-4 output, including netCDF-4 classic model, specifies
number of elements that the chunk cache can hold. This is not a
property of the file, but merely a performance tuning parameter for
avoiding compressing or decompressing the same data multiple times
while copying and changing chunk shapes. The default is 1009 (or
whatever was specified for the configure-time constant
CHUNK_CACHE_NELEMS when the netCDF library was built). Ideally, the
\b nccopy utility should determine an optimum value for this parameter,
but no general algorithm for computing the optimum number of chunk
cache elements has been implemented yet.
Make a copy of foo1.nc, a netCDF file of any type, to foo2.nc, a
netCDF file of the same type:
@par -r
Read netCDF classic or 64-bit offset input file into a diskless netCDF
file in memory before copying. Requires that input file be small
enough to fit into memory. For \b nccopy, this doesn't seem to provide
any significant speedup, so may not be a useful option.
@section EXAMPLES
@subsection simple_copy Simple Copy
Make a copy of foo1.nc, a netCDF file of any type, to
foo2.nc, a netCDF file of the same type:
\code
nccopy foo1.nc foo2.nc
nccopy foo1.nc foo2.nc
\endcode
Note that the above copy will not be as fast as use of cp or other
simple copy utility, because the file is copied using only the netCDF
API. If the input file has extra bytes after the end of the netCDF
data, those will not be copied, because they are not accessible
through the netCDF interface. If the original file was generated in
'No fill' mode so that fill values are not stored for padding for data
alignment, the output file may have different padding bytes.
This also can be used to check foo1.nc for corruption of metadata or
for truncation, because such problems will usually be detected in
trying to parse and read through the data.
Note that the above copy will not be as fast as use of a simple copy
utility, because the file is copied using only the netCDF API. If the
input file has extra bytes after the end of the netCDF data, those
will not be copied, because they are not accessible through the netCDF
interface. If the original file was generated in `No fill' mode so
that fill values are not stored for padding for data alignment, the
output file may have different padding bytes. Uncompress and convert
to classic format
Convert a netCDF-4 classic model file that uses compression
(compressed.nc) to a netCDF-3 file (classic.nc):
@subsection uncompress Uncompress Data
Convert a netCDF-4 classic model file, compressed.nc, that uses
compression, to a netCDF-3 file classic.nc:
\code
nccopy -k classic compressed.nc classic.nc
nccopy -k classic compressed.nc classic.nc
\endcode
Note that '1' could be used instead of 'classic'.
Note that “1” could be used instead of “classic”.
Copy a subset of data on an OPeNDAP server to a netCDF file
Download the variable `time_bnds' and it's associated attributes from
an OPeNDAP server and copy the result to a netCDF file named `tb.nc':
@subsection remote_access Remote Access to Data Subset
Download the variable 'time_bnds' and its associated attributes from
an OPeNDAP server and copy the result to a netCDF file named 'tb.nc':
\code
nccopy 'http://test.opendap.org/opendap/data/nc/sst.mnmean.nc.gz?time_bnds' tb.nc
nccopy 'http://test.opendap.org/opendap/data/nc/sst.mnmean.nc.gz?time_bnds' tb.nc
\endcode
Note that URLs that name specific variables as command-line arguments
should generally be quoted, to avoid the shell interpreting special
characters in OPeNDAP syntax, such as `?'. Apply compression to a
netCDF file
characters such as '?'.
@subsection compress Compress Data
Compress all the variables in the input file foo.nc, a netCDF file of
any type, to the output file bar.nc:
\code
nccopy -d1 foo.nc bar.nc
nccopy -d1 foo.nc bar.nc
\endcode
If foo.nc was a classic or 64-bit offset netCDF file, bar.nc will be a
netCDF-4 classic model netCDF file, because the classic and 64-bit
offset format variants don't support compression. If foo.nc was a
offset format variants don't support compression. If foo.nc was a
netCDF-4 file with some variables compressed using various deflation
levels, the output will also be a netCDF-4 file of the same type, but
all the variables, including any uncompressed variables in the input,
will now use deflation level 1.
@subsection rechunk Rechunk Data for Faster Access
Assume the input data includes gridded variables that use time, lat,
lon dimensions, with 1000 times by 1000 latitudes by 1000 longitudes,
and that the time dimension varies most slowly. Also assume that
users want quick access to data at all times for a small set of
lat-lon points. Accessing data for 1000 times would typically require
accessing 1000 disk blocks, which may be slow.
Reorganizing the data into chunks on disk that have all the time in
each chunk for a few lat and lon coordinates would greatly speed up
such access. To chunk the data in the input file slow.nc, a netCDF
file of any type, to the output file fast.nc, you could use;
\code
nccopy -c time/1000,lat/40,lon/40 slow.nc fast.nc
\endcode
to specify data chunks of 1000 times, 40 latitudes, and 40 longitudes.
If you had enough memory to contain the output file, you could speed
up the rechunking operation significantly by creating the output in
memory before writing it to disk on close:
\code
nccopy -w -c time/1000,lat/40,lon/40 slow.nc fast.nc
\endcode
\page guide_ncgen3 ncgen3
The ncgen3 tool is the new name for the older, original ncgen utility.

290
ncdump/nccopy.1
View File

@ -9,189 +9,216 @@ optionally compressing or chunking data in the output copy.
nccopy
.nh
\%[-k \fI kind \fP]
\%[-c \fI chunkspec \fP]
\%[-d \fI n \fP]
\%[-s]
\%[-u]
\%[-w]
\%[-c \fI chunkspec \fP]
\%[-m \fI bufsize \fP]
\%[-h \fI chunk_cache \fP]
\%[-e \fI cache_elems \fP]
\%[-r]
\%\fI infile \fP
\%\fI outfile \fP
.hy
.ft
.SH DESCRIPTION
.LP
\fBnccopy\fP
copies an input netCDF file (in any of the four format variants) to an
output netCDF file, in any of the four format variants, if possible.
For example, if built with the netCDF-3 library, a netCDF classic file
may be copied to a netCDF 64-bit offset file, permitting larger
variables.
If built with the netCDF-4 library, a netCDF classic file may be
copied to a netCDF-4 file or to a netCDF-4 classic
model file as well, permitting data compression, later efficient schema changes, larger variable sizes, and use of other netCDF-4
features in case the output uses the enhanced netCDF model.
The \fBnccopy\fP utility copies an input netCDF file in any supported
format variant to an output netCDF file, optionally converting the
output to any compatible netCDF format variant, compressing the data,
or rechunking the data. For example, if built with the netCDF-3
library, a netCDF classic file may be copied to a netCDF 64-bit offset
file, permitting larger variables. If built with the netCDF-4
library, a netCDF classic file may be copied to a netCDF-4 file or to
a netCDF-4 classic model file as well, permitting data compression,
efficient schema changes, larger variable sizes, and use of other
netCDF-4 features.
.LP
\fB nccopy \fP also serves as an example of a generic netCDF-4 program,
\fBnccopy\fP also serves as an example of a generic netCDF-4 program,
with its ability to read any valid netCDF file and handle nested
groups, strings, and any user-defined types, including arbitrarily
groups, strings, and user-defined types, including arbitrarily
nested compound types, variable-length types, and data of any valid
netCDF-4 type. Other generic utility programs can make use of parts
of \fB nccopy \fP for more complex operations on netCDF data.
netCDF-4 type.
.LP
As of netCDF version 4.1, and if DAP support was enabled when
\fB nccopy \fP
was built, the file name may specify a DAP URL. This allows \fB nccopy \fP
to convert data on DAP servers to local netCDF files.
If DAP support was enabled when \b nccopy was built, the file name may
specify a DAP URL. This may be used to convert data on DAP servers to
local netCDF files.
.SH OPTIONS
.IP "\fB -k \fP \fI kind \fP"
Specifies the kind of file to be created (that is, the format variant)
and, by inference,
the data model (i.e. netcdf-3 (classic) versus
netcdf-4 (enhanced)).
The possible arguments are as follows.
and, by inference, the data model (i.e. netcdf-3 (classic) versus
netcdf-4 (enhanced)). The possible arguments are as follows.
.RS
.RS
.IP "'1', 'classic' => netcdf classic file format, netcdf-3 type model."
.IP "'2', '64-bit-offset', '64-bit offset' => netcdf 64 bit classic file format, netcdf-3 type model."
.IP "'3', 'hdf5', 'netCDF-4', 'enhanced' => netcdf-4 file format, netcdf-4 type model."
.IP "'4', 'hdf5-nc3', 'netCDF-4 classic model', 'enhanced-nc3' => netcdf-4 file format, netcdf-3 type model."
.IP "'1' or 'classic' => netcdf classic format"
.IP "'2', '64-bit-offset', or '64-bit offset' => netCDF 64-bit format"
.IP "'3', 'hdf5', 'netCDF-4', or 'enhanced' => netCDF-4 format (enhanced data model)"
.IP "'4', 'hdf5-nc3', 'netCDF-4 classic model', or 'enhanced-nc3' => netCDF-4 classic model format"
.RE
.RE
.IP
If no value for -k is specified, then the output will use the same
format as the input, except if the input is classic or 64-bit offset
and either chunking or compression is specified, in which case the output will be netCDF-4 classic
model format.
Note that attempting some kinds of format
conversion will result in an error, if the conversion is not
possible. For example, an attempt to copy a netCDF-4 file that uses
features of the enhanced model to any of the other kinds of netCDF
and either chunking or compression is specified, in which case the
output will be netCDF-4 classic model format. Note that attempting
some kinds of format conversion will result in an error, if the
conversion is not possible. For example, an attempt to copy a
netCDF-4 file that uses features of the enhanced model, such as
groups or variable-length strings, to any of the other kinds of netCDF
formats that use the classic model will result in an error.
.IP "\fB -d \fP \fI n \fP"
Specify deflation level (level of compression) for variable data in
output. 0 corresponds to no compression and 9 to maximum compression,
with higher levels of compression requiring marginally more time to
For netCDF-4 output, including netCDF-4 classic model, specify
deflation level (level of compression) for variable data output. 0
corresponds to no compression and 9 to maximum compression, with
higher levels of compression requiring marginally more time to
compress or uncompress than lower levels. Compression achieved may
also depend on chunking parameters, which will use default chunking in the current nccopy
implementation. If this option is specified for a classic format or
64-bit offset format input file, it is not necessary to also specify
that the output should be netCDF-4 classic model, as that will
be the default. If this option is not specified and the input file
has compressed variables, the compression will still be preserved
in the output, using the same chunking as in the input.
also depend on output chunking parameters. If this option is
specified for a classic format or 64-bit offset format input file, it
is not necessary to also specify that the output should be netCDF-4
classic model, as that will be the default. If this option is not
specified and the input file has compressed variables, the compression
will still be preserved in the output, using the same chunking as in
the input by default.
.IP
Note that \fB nccopy \fP requires all variables to be compressed using
Note that \fBnccopy\fP requires all variables to be compressed using
the same compression level, but the API has no such restriction. With
a program you can customize compression for each variable independently.
.IP "\fB -s \fP"
Specify shuffling of variable data bytes before compression or after
For netCDF-4 output, including netCDF-4 classic model, specify
shuffling of variable data bytes before compression or after
decompression. This option is ignored unless a non-zero deflation
level is specified. Turning shuffling on sometimes improves
compression.
compression.
.IP "\fB -u \fP"
Convert any unlimited size dimensions in the input to fixed size
dimensions in the output.
.IP "\fB -c \fP \fI chunkspec \fP"
Specify chunking (multidimensional tiling) for variable data in
the output, useful to specify the units of disk access, compression, or
other filters such as checksums.
The \fI chunkspec \fP argument is a string of comma-separated
associations, each specifying a dimension name, a `/' character, and
dimensions in the output. This can speed up variable-at-a-time
access, but slow down record-at-a-time access to multiple variables
along an unlimited dimension.
.IP "\fB -w \fP"
Keep output in memory (as a diskless netCDF file) until output is
closed, at which time output file is written to disk. This can
greatly speedup operations such as converting unlimited dimension to
fixed size (-u option), chunking, rechunking, or compressing the
input. It requires that available memory is large enough to hold the
output file. This option may provide a larger speedup than careful
tuning of the -m, -h, or -e options, and it's certainly a lot simpler.
.IP "\fB -c \fP \fIchunkspec\fP"
For netCDF-4 output, including netCDF-4 classic model, specify
chunking (multidimensional tiling) for variable data in the output.
This is useful to specify the units of disk access, compression, or
other filters such as checksums. Changing the chunking in a netCDF
file can also greatly speedup access, by choosing chunk shapes that
are appropriate for the most common access patterns.
.IP
The \fIchunkspec\fP argument is a string of comma-separated
associations, each specifying a dimension name, a '/' character, and
optionally the corresponding chunk length for that dimension. No
blanks should appear in the chunkspec string, except possibly escaped
blanks that are part of a dimension name. A
chunkspec must name at least one dimension, and may omit dimensions
which are not to be chunked or for which the default chunk length is
desired. If a dimension name is followed by a `/' character but no
subsequent chunk length, the actual dimension length is assumed. If
copying a classic model file to a netCDF-4 output file and not naming
all dimensions in the chunkspec, unnamed dimensions will also use the
actual dimension length for the chunk length.
An example of a chunkspec
for variables that use the `m' and `n' dimensions might be
`m/100,n/200' to specify 100 by 200 chunks. To see the chunking
resulting from copying with a chunkspec, use the `-s'
option of ncdump on the output file.
blanks that are part of a dimension name. A chunkspec must name at
least one dimension, and may omit dimensions which are not to be
chunked or for which the default chunk length is desired. If a
dimension name is followed by a '/' character but no subsequent chunk
length, the actual dimension length is assumed. If copying a classic
model file to a netCDF-4 output file and not naming all dimensions in
the chunkspec, unnamed dimensions will also use the actual dimension
length for the chunk length. An example of a chunkspec for variables
that use 'm' and 'n' dimensions might be 'm/100,n/200' to specify
100 by 200 chunks. To see the chunking resulting from copying with a
chunkspec, use the '-s' option of ncdump on the output file.
.IP
Note that \fBnccopy\fP requires variables that share a dimension to
also share the chunk size associated with that dimension, but the
programming interface has no such restriction. If you need to
customize chunking for variables independently, you will need to use
the library API in a custom utility program.
.IP "\fB -m \fP \fI bufsize \fP"
An integer or floating-point number that specifies the size, in bytes,
of the copy buffer used
to copy large variables. A suffix of K, M, G, or T multiplies
the copy buffer size by one thousand, million, billion, or trillion, respectively.
The default is 5,000,000 bytes,
of the copy buffer used to copy large variables. A suffix of K, M, G,
or T multiplies the copy buffer size by one thousand, million,
billion, or trillion, respectively. The default is 5 Mbytes,
but will be increased if necessary to hold at least one chunk of
netCDF-4 chunked variables in the input file. You may want to specify
a value larger than the default for copying large files over high
latency networks.
latency networks. Using the '-w' option may provide better
performance, if the output fits in memory.
.IP "\fB -h \fP \fI chunk_cache \fP"
An integer or floating-point number that specifies the size in bytes
of chunk cache for chunked variables. This is
not a property of the file, but merely a performance tuning parameter
for avoiding compressing or decompressing the same data multiple times
while copying and changing chunk shapes. A suffix of K, M, G, or T multiplies
the chunk cache size by one thousand, million, billion, or trillion, respectively.
The default is 4,194,304 (or whatever was specified for the
For netCDF-4 output, including netCDF-4 classic model, an integer or
floating-point number that specifies the size in bytes of chunk cache
for chunked variables. This is not a property of the file, but merely
a performance tuning parameter for avoiding compressing or
decompressing the same data multiple times while copying and changing
chunk shapes. A suffix of K, M, G, or T multiplies the chunk cache
size by one thousand, million, billion, or trillion, respectively.
The default is 4.194304 Mbytes (or whatever was specified for the
configure-time constant CHUNK_CACHE_SIZE when the netCDF library was
built). Ideally, the nccopy utility should accept only one memory
buffer size and divide it optimally between a copy buffer and chunk
cache, but no general algorithm for computing the optimum chunk cache
size has been implemented yet.
built). Ideally, the \fBnccopy\fP utility should accept only one
memory buffer size and divide it optimally between a copy buffer and
chunk cache, but no general algorithm for computing the optimum chunk
cache size has been implemented yet. Using the '-w' option may
provide better performance, if the output fits in memory.
.IP "\fB -e \fP \fI cache_elems \fP"
Specifies number of elements that the chunk cache can hold. This is
not a property of the file, but merely a performance tuning parameter
for avoiding compressing or decompressing the same data multiple times
For netCDF-4 output, including netCDF-4 classic model, specifies
number of elements that the chunk cache can hold. This is not a
property of the file, but merely a performance tuning parameter for
avoiding compressing or decompressing the same data multiple times
while copying and changing chunk shapes. The default is 1009 (or
whatever was specified for the configure-time constant
CHUNK_CACHE_NELEMS when the netCDF library was built). Ideally, the
nccopy utility should determine an optimum value for this parameter,
but no general algorithm for computing the optimum number of chunk
cache elements has been implemented yet.
.P
Note that \fB nccopy \fP requires variables that share a dimension to
also share the chunk size associated with that dimension, but the API
has no such restriction. If you need to customize chunking
for each variable independently, you will need to use the library API
in a custom utility program.
\fBnccopy\fP utility should determine an optimum value for this
parameter, but no general algorithm for computing the optimum number
of chunk cache elements has been implemented yet.
.IP "\fB -r \fP"
Read netCDF classic or 64-bit offset input file into a diskless netCDF
file in memory before copying. Requires that input file be small
enough to fit into memory. For \b nccopy, this doesn't seem to provide
any significant speedup, so may not be a useful option.
.SH EXAMPLES
.PP
.LP
Make a copy of foo1.nc, a netCDF file of any type, to foo2.nc, a
netCDF file of the same type:
nccopy foo1.nc foo2.nc
Note that the above copy will not be as fast as use of a
.RS
.HP
nccopy foo1.nc foo2.nc
.RE
.LP
Note that the above copy will not be as fast as use of cp or other
simple copy utility, because the file is copied using
only the netCDF
API. If the input file has extra bytes
after the end of the
netCDF data, those will not be copied, because they are not accessible
through the netCDF interface. If the original file was generated in
`No fill' mode so that fill values are not stored for padding for data
'No fill' mode so that fill values are not stored for padding for data
alignment, the output file may have different padding bytes.
.PP
.LP
Convert a netCDF-4 classic model file, compressed.nc, that uses compression,
to a netCDF-3 file classic.nc:
nccopy -k classic compressed.nc classic.nc
Note that `1' could be used instead of `classic'.
.PP
Download the variable `time_bnds' and it's associated attributes from
an OPeNDAP server and copy the result to a netCDF file named `tb.nc':
nccopy 'http://test.opendap.org/opendap/data/nc/sst.mnmean.nc.gz?time_bnds' tb.nc
.RS
.HP
nccopy -k classic compressed.nc classic.nc
.RE
.LP
Note that '1' could be used instead of 'classic'.
.LP
Download the variable 'time_bnds' and its associated attributes from
an OPeNDAP server and copy the result to a netCDF file named 'tb.nc':
.RS
.HP
nccopy 'http://test.opendap.org/opendap/data/nc/sst.mnmean.nc.gz?time_bnds' tb.nc
.RE
.LP
Note that URLs that name specific variables as command-line arguments
should generally be quoted, to avoid the shell interpreting special
characters such as `?'.
.PP
characters such as '?'.
.LP
Compress all the variables in the input file foo.nc, a netCDF file of any
type, to the output file bar.nc:
nccopy -d1 foo.nc bar.nc
.RS
.HP
nccopy -d1 foo.nc bar.nc
.RE
.LP
If foo.nc was a classic or 64-bit offset netCDF file, bar.nc will be a
netCDF-4 classic model netCDF file, because the classic and 64-bit
offset format variants don't support compression. If foo.nc was a
@ -199,8 +226,31 @@ netCDF-4 file with some variables compressed using various deflation
levels, the output will also be a netCDF-4 file of the same type, but
all the variables, including any uncompressed variables in the input,
will now use deflation level 1.
.LP
Assume the input data includes gridded variables that use time, lat,
lon dimensions, with 1000 times by 1000 latitudes by 1000 longitudes,
and that the time dimension varies most slowly. Also assume that
users want quick access to data at all times for a small set of
lat-lon points. Accessing data for 1000 times would typically require
accessing 1000 disk blocks, which may be slow.
.LP
Reorganizing the data into chunks on disk that have all the time in
each chunk for a few lat and lon coordinates would greatly speed up
such access. To chunk the data in the input file slow.nc, a netCDF
file of any type, to the output file fast.nc, you could use;
.RS
.HP
nccopy -c time/1000,lat/40,lon/40 slow.nc fast.nc
.RE
.LP
to specify data chunks of 1000 times, 40 latitudes, and 40 longitudes.
If you had enough memory to contain the output file, you could speed
up the rechunking operation significantly by creating the output in
memory before writing it to disk on close:
.RS
.HP
nccopy -w -c time/1000,lat/40,lon/40 slow.nc fast.nc
.RE
.SH "SEE ALSO"
.LP
.BR ncdump(1), ncgen (1),
.BR netcdf (3)
.BR ncdump(1), ncgen(1), netcdf(3)

253
ncdump/nccopy.c
View File

@ -1367,21 +1367,270 @@ usage(void)
[-s] add shuffle option to deflation compression\n\
[-c chunkspec] specify chunking for dimensions, e.g. \"dim1/N1,dim2/N2,...\"\n\
[-u] convert unlimited dimensions to fixed-size dimensions in output copy\n\
[-w] write whole output file from diskless netCDF on close\n\
[-m n] set size in bytes of copy buffer, default is 5000000 bytes\n\
[-h n] set size in bytes of chunk_cache for chunked variables\n\
[-e n] set number of elements that chunk_cache can hold\n\
[-r] read whole input file into diskless file on open (classic or 64-bit offset format only)\n\
[-w] write whole output file from diskless netCDF on close\n\
infile name of netCDF input file\n\
outfile name for netCDF output file\n"
/* Don't document this flaky option until it works better */
/* [-x] use experimental computed estimates for variable-specific chunk caches\n\ */
error("%s [-k n] [-d n] [-s] [-c chunkspec] [-u] [-m n] [-h n] [-e n] [-r] [-w] infile outfile\n%s",
error("%s [-k n] [-d n] [-s] [-c chunkspec] [-u] [-w] [-m n] [-h n] [-e n] [-r] infile outfile\n%s",
progname, USAGE);
}
/** @page nccopy \b nccopy tool - Copy a netCDF file, optionally changing format, compression, or chunking in the output.
@section SYNOPSIS
\code
nccopy [-k kind] [-d n] [-s] [-u] [-w] [-c chunkspec] [-m bufsize]
[-h chunk_cache] [-e cache_elems] [-r] infile outfile
\endcode
@section DESCRIPTION
The \b nccopy utility copies an input netCDF file in any supported
format variant to an output netCDF file, optionally converting the
output to any compatible netCDF format variant, compressing the data,
or rechunking the data. For example, if built with the netCDF-3
library, a netCDF classic file may be copied to a netCDF 64-bit offset
file, permitting larger variables. If built with the netCDF-4
library, a netCDF classic file may be copied to a netCDF-4 file or to
a netCDF-4 classic model file as well, permitting data compression,
efficient schema changes, larger variable sizes, and use of other
netCDF-4 features.
\b nccopy also serves as an example of a generic netCDF-4 program,
with its ability to read any valid netCDF file and handle nested
groups, strings, and user-defined types, including arbitrarily
nested compound types, variable-length types, and data of any valid
netCDF-4 type.
If DAP support was enabled when \b nccopy was built, the file name may
specify a DAP URL. This may be used to convert data on DAP servers to
local netCDF files.
@section OPTIONS
@par -k \e kind
Specifies the kind of file to be created (that is, the format variant)
and, by inference, the data model (i.e. netcdf-3 (classic) versus
netcdf-4 (enhanced)). The possible arguments are as follows. \n
'1' or 'classic' => netCDF classic format \n
'2', '64-bit-offset', or '64-bit offset' => netCDF 64-bit format \n
'3', 'hdf5', 'netCDF-4', or 'enhanced' => netCDF-4 format (enhanced data model) \n
'4', 'hdf5-nc3', 'netCDF-4 classic model', or 'enhanced-nc3' => netCDF-4 classic model format \n
@par
If no value for -k is specified, then the output will use the same
format as the input, except if the input is classic or 64-bit offset
and either chunking or compression is specified, in which case the
output will be netCDF-4 classic model format. Note that attempting
some kinds of format conversion will result in an error, if the
conversion is not possible. For example, an attempt to copy a
netCDF-4 file that uses features of the enhanced model, such as groups
or variable-length strings, to any of the other kinds of netCDF
formats that use the classic model will result in an error.
@par -d \e n
For netCDF-4 output, including netCDF-4 classic model, specify
deflation level (level of compression) for variable data output. 0
corresponds to no compression and 9 to maximum compression, with
higher levels of compression requiring marginally more time to
compress or uncompress than lower levels. Compression achieved may
also depend on output chunking parameters. If this option is
specified for a classic format or 64-bit offset format input file, it
is not necessary to also specify that the output should be netCDF-4
classic model, as that will be the default. If this option is not
specified and the input file has compressed variables, the compression
will still be preserved in the output, using the same chunking as in
the input by default.
@par
Note that \b nccopy requires all variables to be compressed using the
same compression level, but the API has no such restriction. With a
program you can customize compression for each variable independently.
@par -s
For netCDF-4 output, including netCDF-4 classic model,
specify shuffling of variable data bytes before compression or after
decompression. This option is ignored unless a non-zero deflation
level is specified. Turning shuffling on sometimes improves
compression.
@par -u
Convert any unlimited size dimensions in the input to fixed size
dimensions in the output. This can speed up variable-at-a-time
access, but slow down record-at-a-time access to multiple variables
along an unlimited dimension.
@par -w
Keep output in memory (as a diskless netCDF file) until output is
closed, at which time output file is written to disk. This can
greatly speedup operations such as converting unlimited dimension to
fixed size (-u option), chunking, rechunking, or compressing the
input. It requires that available memory is large enough to hold the
output file. This option may provide a larger speedup than careful
tuning of the -m, -h, or -e options, and it's certainly a lot simpler.
@par -c \e chunkspec
@par
For netCDF-4 output, including netCDF-4 classic model, specify
chunking (multidimensional tiling) for variable data in the output.
This is useful to specify the units of disk access, compression, or
other filters such as checksums. Changing the chunking in a netCDF
file can also greatly speedup access, by choosing chunk shapes that
are appropriate for the most common access patterns.
@par
The chunkspec argument is a string of comma-separated associations,
each specifying a dimension name, a '/' character, and optionally the
corresponding chunk length for that dimension. No blanks should
appear in the chunkspec string, except possibly escaped blanks that
are part of a dimension name. A chunkspec must name at least one
dimension, and may omit dimensions which are not to be chunked or for
which the default chunk length is desired. If a dimension name is
followed by a '/' character but no subsequent chunk length, the actual
dimension length is assumed. If copying a classic model file to a
netCDF-4 output file and not naming all dimensions in the chunkspec,
unnamed dimensions will also use the actual dimension length for the
chunk length. An example of a chunkspec for variables that use
'm' and 'n' dimensions might be 'm/100,n/200' to specify 100 by 200
chunks. To see the chunking resulting from copying with a chunkspec,
use the '-s' option of ncdump on the output file.
@par
Note that \b nccopy requires variables that share a dimension to also
share the chunk size associated with that dimension, but the
programming interface has no such restriction. If you need to
customize chunking for variables independently, you will need to use
the library API in a custom utility program.
@par -m \e bufsize
@par
An integer or floating-point number that specifies the size, in bytes,
of the copy buffer used to copy large variables. A suffix of K, M, G,
or T multiplies the copy buffer size by one thousand, million,
billion, or trillion, respectively. The default is 5 Mbytes,
but will be increased if necessary to hold at least one chunk of
netCDF-4 chunked variables in the input file. You may want to specify
a value larger than the default for copying large files over high
latency networks. Using the '-w' option may provide better
performance, if the output fits in memory.
@par -e \e chunk_cache
@par
For netCDF-4 output, including netCDF-4 classic model, an integer or
floating-point number that specifies the size in bytes of chunk cache
for chunked variables. This is not a property of the file, but merely
a performance tuning parameter for avoiding compressing or
decompressing the same data multiple times while copying and changing
chunk shapes. A suffix of K, M, G, or T multiplies the chunk cache
size by one thousand, million, billion, or trillion, respectively.
The default is 4.194304 Mbytes (or whatever was specified for the
configure-time constant CHUNK_CACHE_SIZE when the netCDF library was
built). Ideally, the \b nccopy utility should accept only one memory
buffer size and divide it optimally between a copy buffer and chunk
cache, but no general algorithm for computing the optimum chunk cache
size has been implemented yet. Using the '-w' option may provide
better performance, if the output fits in memory.
@par -h \e cache_elems
@par
For netCDF-4 output, including netCDF-4 classic model, specifies
number of elements that the chunk cache can hold. This is not a
property of the file, but merely a performance tuning parameter for
avoiding compressing or decompressing the same data multiple times
while copying and changing chunk shapes. The default is 1009 (or
whatever was specified for the configure-time constant
CHUNK_CACHE_NELEMS when the netCDF library was built). Ideally, the
\b nccopy utility should determine an optimum value for this parameter,
but no general algorithm for computing the optimum number of chunk
cache elements has been implemented yet.
@par -r
Read netCDF classic or 64-bit offset input file into a diskless netCDF
file in memory before copying. Requires that input file be small
enough to fit into memory. For \b nccopy, this doesn't seem to provide
any significant speedup, so may not be a useful option.
@section EXAMPLES
@subsection simple_copy Simple Copy
Make a copy of foo1.nc, a netCDF file of any type, to
foo2.nc, a netCDF file of the same type:
\code
nccopy foo1.nc foo2.nc
\endcode
Note that the above copy will not be as fast as use of cp or other
simple copy utility, because the file is copied using only the netCDF
API. If the input file has extra bytes after the end of the netCDF
data, those will not be copied, because they are not accessible
through the netCDF interface. If the original file was generated in
'No fill' mode so that fill values are not stored for padding for data
alignment, the output file may have different padding bytes.
@subsection uncompress Uncompress Data
Convert a netCDF-4 classic model file, compressed.nc, that uses
compression, to a netCDF-3 file classic.nc:
\code
nccopy -k classic compressed.nc classic.nc
\endcode
Note that '1' could be used instead of 'classic'.
@subsection remote_access Remote Access to Data Subset
Download the variable 'time_bnds' and its associated attributes from
an OPeNDAP server and copy the result to a netCDF file named 'tb.nc':
\code
nccopy 'http://test.opendap.org/opendap/data/nc/sst.mnmean.nc.gz?time_bnds' tb.nc
\endcode
Note that URLs that name specific variables as command-line arguments
should generally be quoted, to avoid the shell interpreting special
characters such as '?'.
@subsection compress Compress Data
Compress all the variables in the input file foo.nc, a netCDF file of
any type, to the output file bar.nc:
\code
nccopy -d1 foo.nc bar.nc
\endcode
If foo.nc was a classic or 64-bit offset netCDF file, bar.nc will be a
netCDF-4 classic model netCDF file, because the classic and 64-bit
offset format variants don't support compression. If foo.nc was a
netCDF-4 file with some variables compressed using various deflation
levels, the output will also be a netCDF-4 file of the same type, but
all the variables, including any uncompressed variables in the input,
will now use deflation level 1.
@subsection rechunk Rechunk Data for Faster Access
Assume the input data includes gridded variables that use time, lat,
lon dimensions, with 1000 times by 1000 latitudes by 1000 longitudes,
and that the time dimension varies most slowly. Also assume that
users want quick access to data at all times for a small set of
lat-lon points. Accessing data for 1000 times would typically require
accessing 1000 disk blocks, which may be slow.
Reorganizing the data into chunks on disk that have all the time in
each chunk for a few lat and lon coordinates would greatly speed up
such access. To chunk the data in the input file slow.nc, a netCDF
file of any type, to the output file fast.nc, you could use;
\code
nccopy -c time/1000,lat/40,lon/40 slow.nc fast.nc
\endcode
to specify data chunks of 1000 times, 40 latitudes, and 40 longitudes.
If you had enough memory to contain the output file, you could speed
up the rechunking operation significantly by creating the output in
memory before writing it to disk on close:
\code
nccopy -w -c time/1000,lat/40,lon/40 slow.nc fast.nc
\endcode
@section see_also SEE ALSO
netcdf(3), ncgen(1), netcdf(3)
*/
int
main(int argc, char**argv)
{

231
ncdump/ncdump.1
View File

@ -7,18 +7,14 @@ ncdump \- Convert netCDF file to text form (CDL)
.HP
ncdump
.nh
\%[-c]
\%[-h]
\%[-chistxw]
\%[-v \fIvar1,...\fP]
\%[-g \fIgrp1,...\fP]
\%[-b \fIlang\fP]
\%[-f \fIlang\fP]
\%[-l \fIlen\fP]
\%[-n \fIname\fP]
\%[-p \fIf_digits[,d_digits]\fP]
\%[-x]
\%[-s]
\%[-t]
\%[-g \fIgrp1,...\fP]
\%\fIfile\fP
.br
.ft B
@ -30,83 +26,82 @@ ncdump
.hy
.ft
.SH DESCRIPTION
\fBncdump\fP generates a text representation of a specified netCDF file on
standard output. The text representation is in a form called CDL
(``network Common Data form Language'') that can be viewed, edited, or serve
as input to \fBncgen\fP. \fBncgen\fP is a companion program that can
generate a binary netCDF file from a CDL file. Hence \fBncgen\fP and
\fBncdump\fP can be used as inverses to transform the data representation
between binary and text representations. See \fBncgen\fP for a description
of CDL and netCDF representations.
.LP
As of NetCDF version 4.1, and if DAP support was enabled when \fBncdump\fP
was built, the file name may specify a DAP URL. This allows \fBncdump\fP
to print out data sources from DAP servers. When used with
the \fB-h\fP option, \fBncdump\fP can be used to show the translation
from the DAP DDS data model to the NetCDF data model.
The \fBncdump\fP utility generates a text representation of a
specified netCDF file on standard output, optionally excluding some or
all of the variable data in the output. The text representation is in
a form called CDL (network Common Data form Language) that can be
viewed, edited, or serve as input to \fBncgen\fP, a companion program
that can generate a binary netCDF file from a CDL file. Hence
\fBncgen\fP and \fBncdump\fP can be used as inverses to transform the
data representation between binary and text representations. See
\fBncgen\fP documentation for a description of CDL and netCDF
representations.
.LP
\fBncdump\fP may also be used to determine what kind of netCDF file is used
(which variant of the netCDF file format) with the -k option.
.LP
If DAP support was enabled when \fBncdump\fP was built, the file name
may specify a DAP URL. This allows \fBncdump\fP to access data sources
from DAP servers, including data in other formats than netCDF. When
used with DAP URLs, \fBncdump\fP shows the translation from the DAP
data model to the netCDF data model.
.LP
\fBncdump\fP may also be used as a simple browser for netCDF data
files, to display the dimension names and lengths; variable names, types,
and shapes; attribute names and values; and optionally, the values of
data for all variables or selected variables in a netCDF file. For
netCDF-4 files, groups and user-defined types are also included in
ncdump output.
.LP
\fBncdump\fP uses `_' to represent data values that are equal to the
`_FillValue' attribute for a variable, intended to represent data that
has not yet been written. If a variable has no `_FillValue'
attribute, the default fill value for the variable type is used unless
the variable is of byte type.
.LP
\fBncdump\fP defines a default display format used for each type of
netCDF data, but
this can be changed if a `C_format' attribute is defined for a netCDF
variable. In this case, \fBncdump\fP will use the `C_format' attribute to
format each value. For example, if floating-point data for the netCDF
variable `Z' is known to be accurate to only three significant digits, it
would be appropriate to use the variable attribute
netCDF data, but this can be changed if a `C_format' attribute is
defined for a netCDF variable. In this case, \fBncdump\fP will use
the `C_format' attribute to format each value. For example, if
floating-point data for the netCDF variable `Z' is known to be
accurate to only three significant digits, it would be appropriate to
use the variable attribute
.RS
.HP
Z:C_format = "%.3g"
.RE
.LP
\fBncdump\fP may also be used as a simple browser for netCDF data
files, to display the dimension names and sizes; variable names, types,
and shapes; attribute names and values; and optionally, the values of
data for all variables or selected variables in a netCDF file.
.LP
\fBncdump\fP uses `_' to represent data values that are equal to the
`_FillValue' attribute for a variable, intended to represent data that
has not yet been written. If a variable has no `_FillValue' attribute, the
default fill value for the variable type is used if the variable is not of
byte type.
.LP
\fBncdump\fP may also be used to determine what kind of netCDF file is used
(which variant of the netCDF file format) with the -k option.
.SH OPTIONS
.IP "\fB-c\fP"
Show the values of \fIcoordinate\fP variables (variables that are also
dimensions) as well as the declarations of all dimensions, variables, and
attribute values. Data values of non-coordinate variables are not included
in the output. This is the most suitable option to use for a brief look at
the structure and contents of a netCDF file.
Show the values of \fIcoordinate\fP variables (1D variables with the
same names as dimensions) as well as the declarations of all
dimensions, variables, attribute values, groups, and user-defined
types. Data values of non-coordinate variables are not included in
the output. This is usually the most suitable option to use for a brief look
at the structure and contents of a netCDF file.
.IP "\fB-h\fP"
Show only the \fIheader\fP information in the output, that is the
declarations of dimensions, variables, and attributes but no data values for
Show only the \fIheader\fP information in the output, that is, output only
the declarations for the dimensions, variables, attributes,
groups, and user-defined types of the input file, but no data values for
any variables. The output is identical to using the \fB-c\fP option except
that the values of coordinate variables are not included. (At most one of
\fB-c\fP or \fB-h\fP options may be present.)
.IP "\fB-v\fP \fIvar1,...,varn\fP"
The output will include data values for the specified variables, in addition
to the declarations of all dimensions, variables, and attributes. One or
more variables must be specified by name in the comma-delimited list
following this option. The list must be a single argument to the command. The named
variables must be valid netCDF variables in the input-file. A variable
within a group in a netCDF-4 file may be specified with an absolute
path name, such as `/GroupA/GroupA2/var'. Use of a relative path name
such as `var' or `grp/var' specifies all matching
variable names in the file.
The default,
without this option and in the absence of the \fB-c\fP or \fB-h\fP
options, is to include data values for \fIall\fP variables in the output.
.IP "\fB-g\fP \fIgrp1,...,grpn\fP"
For netCDF-4 files, the output will include data values only for the
specified groups. One or more groups must be specified by name in the
comma-delimited list following this option. The list must be a single
argument to the command. The named groups must be valid netCDF groups
in the input-file. A group in a netCDF-4 file may be specified with
an absolute or relative path name. Use of a relative path name
specifies all matching group names in the file. The default, without
this option and in the absence of the \fB-c\fP or \fB-h\fP options, is
to include data values for \fIall\fP groups in the output.
.IP "\fB-b\fP \fIlang\fP"
.IP "\fB-v\fP \fIvar1,...\fP"
The output will include data values for the specified variables, in
addition to the declarations of all dimensions, variables, and
attributes. One or more variables must be specified by name in the
comma-delimited list following this option. The list must be a single
argument to the command, hence cannot contain unescaped blanks or
other white space characters. The named variables must be valid
netCDF variables in the input-file. A variable within a group in a
netCDF-4 file may be specified with an absolute path name, such as
`/GroupA/GroupA2/var'. Use of a relative path name such as `var' or
`grp/var' specifies all matching variable names in the file. The
default, without this option and in the absence of the \fB-c\fP or
\fB-h\fP options, is to include data values for \fIall\fP variables in
the output.
.IP "\fB-b\fP \fI[c|f]\fP"
A brief annotation in the form of a CDL comment (text beginning with the
characters ``//'') will be included in the data section of the output for
each `row' of data, to help identify data values for multidimensional
@ -115,26 +110,26 @@ conventions will be used (zero-based indices, last dimension varying
fastest). If \fIlang\fP begins with `F' or `f', then Fortran language
conventions will be used (one-based indices, first dimension varying
fastest). In either case, the data will be presented in the same order;
only the annotations will differ. This option is useful for browsing
only the annotations will differ. This option may be useful for browsing
through large volumes of multidimensional data.
.IP "\fB-f\fP \fIlang\fP"
Full annotations in the form of trailing CDL comments (text beginning with
the characters ``//'') for every data value (except individual characters in
character arrays) will be included in the data section. If \fIlang\fP
begins with `C' or `c', then C language conventions will be used (zero-based
indices, last dimension varying fastest). If \fIlang\fP begins with `F' or
`f', then Fortran language conventions will be used (one-based indices,
first dimension varying fastest). In either case, the data will be
presented in the same order; only the annotations will differ. This option
may be useful for piping data into other filters, since each data value
appears on a separate line, fully identified.
.IP "\fB-l\fP \fIlen\fP"
.IP "\fB-f\fP \fI[c|f]\fP"
Full annotations in the form of trailing CDL comments (text beginning
with the characters ``//'') for every data value (except individual
characters in character arrays) will be included in the data section.
If \fIlang\fP begins with `C' or `c', then C language conventions will
be used. If \fIlang\fP begins with `F' or `f', then Fortran language
conventions will be used. In either case, the data will be presented
in the same order; only the annotations will differ. This option may
be useful for piping data into other filters, since each data value
appears on a separate line, fully identified. (At most one of '-b' or
'-f' options may be present.)
.IP "\fB-l\fP \fIlength\fP"
Changes the default maximum line length (80) used in formatting lists of
non-character data values.
.IP "\fB-n\fP \fIname\fP"
CDL requires a name for a netCDF data set, for use by \fBncgen -b\fP in
CDL requires a name for a netCDF file, for use by \fBncgen -b\fP in
generating a default netCDF file name. By default, \fIncdump\fP constructs
this name from the last component of the path name of the input netCDF file
this name from the last component of the file name of the input netCDF file
by stripping off any extension it has. Use the \fB-n\fP option to specify a
different name. Although the output file name used by \fBncgen -b\fP can be
specified, it may be wise to have \fIncdump\fP change the default name to
@ -142,25 +137,23 @@ avoid inadvertently overwriting a valuable netCDF file when using
\fBncdump\fP, editing the resulting CDL file, and using \fBncgen -b\fP to
generate a new netCDF file from the edited CDL file.
.IP "\fB-p\fP \fIfloat_digits[,double_digits]\fP"
Specifies default precision (number of significant digits) to use in displaying
floating-point or double precision data values for attributes and variables.
If specified, this value overrides the value of the `C_format' attribute for
any variable that has such an attribute.
Floating-point data will be displayed with
\fIfloat_digits\fP significant digits. If \fIdouble_digits\fP is also
specified, double-precision values will be displayed with that many
significant digits. In the absence of any
\fB-p\fP specifications, floating-point and double-precision data are
displayed with 7 and 15 significant digits respectively. CDL files can be
made smaller if less precision is required. If both floating-point and
double-precision precisions are specified, the two values must appear
separated by a comma (no blanks) as a single argument to the command.
If you really want every last bit of precision from the netCDF file
represented in the CDL file for all possible floating-point values, you will
have to specify this with \fB-p 9,17\fP (according to Theorem 15 of the
paper listed under REFERENCES).
Specifies default precision (number of significant digits) to use in
displaying floating-point or double precision data values for
attributes and variables. If specified, this value overrides the
value of the C_format attribute, if any, for a variable.
Floating-point data will be displayed with \fIfloat_digits\fP
significant digits. If \fIdouble_digits\fP is also specified,
double-precision values will be displayed with that many significant
digits. In the absence of any \fB-p\fP specifications, floating-point
and double-precision data are displayed with 7 and 15 significant
digits respectively. CDL files can be made smaller if less precision
is required. If both floating-point and double precisions are
specified, the two values must appear separated by a comma (no blanks)
as a single argument to the command. (To represent every last bit of
precision in a CDL file for all possible floating-point values
would require \fB-p 9,17\fP.)
.IP "\fB-k\fP"
Show what kind of netCDF file the pathname references, one of
Show kind of netCDF file the pathname references, one of
`classic', `64-bit offset',`netCDF-4', or `netCDF-4 classic model'. Before version
3.6, there was only one kind of netCDF file, designated as `classic'
(also know as format variant 1). Large file support introduced
@ -175,9 +168,6 @@ The string output by using the `-k' option may be provided as the
value of the `-k' option to ncgen(1) to
specify exactly what kind of netCDF file to generate, when you want to
override the default inferred from the CDL.
.IP "\fB-x\fP"
Output XML (NcML) instead of CDL. The NcML does not include data values.
The NcML output option currently only works for netCDF classic model data.
.IP "\fB-s\fP"
Output special virtual attributes that provide performance-related
information about the file format and variable properties for netCDF-4
@ -222,6 +212,25 @@ actual value, in an associated CDL comment. Calendar attribute
values interpreted with this option include the CF Conventions values
`gregorian' or `standard', `proleptic_gregorian', `noleap' or `365_day',
`all_leap' or `366_day', `360_day', and `julian'.
.IP "\fB-i\fP"
Same as the '-t' option, except output time data as date-time strings
with ISO-8601 standard 'T' separator, instead of a blank.
.IP "\fB-g\fP \fIgrp1,...\fP"
For netCDF-4 files, the output will include data values only for the
specified groups. One or more groups must be specified by name in the
comma-delimited list following this option. The list must be a single
argument to the command. The named groups must be valid netCDF groups
in the input-file. A group in a netCDF-4 file may be specified with
an absolute or relative path name. Use of a relative path name
specifies all matching group names in the file. The default, without
this option and in the absence of the \fB-c\fP or \fB-h\fP options, is
to include data values for \fIall\fP groups in the output.
.IP "\fB-w\fP"
For file names that request remote access using DAP URLs, access data
with client-side caching of entire variables.
.IP "\fB-x\fP"
Output XML (NcML) instead of CDL. The NcML does not include data values.
The NcML output option currently only works for netCDF classic model data.
.SH EXAMPLES
.LP
Look at the structure of the data in the netCDF file `\fBfoo.nc\fP':
@ -252,12 +261,6 @@ netCDF dataset name in the resulting CDL file to `omega':
.HP
ncdump -v omega -f fortran -n omega foo.nc > Z.cdl
.RE
.SH REFERENCES
\fIWhat
Every Computer Scientist should Know About Floating-Point Arithmetic\fP, D.
Goldberg, \fBACM Computing Surveys, Vol. 23, No. 1\fP, March 1991, pp. 5-48.
.BR
Climate and Forecast Metadata Conventions, http://www.cfconventions.org.
.SH "SEE ALSO"
.LP
.BR ncgen (1),
@ -266,11 +269,11 @@ Climate and Forecast Metadata Conventions, http://www.cfconventions.org.
.LP
Character arrays that contain a null-byte are treated like C strings, so no
characters after the null byte appear in the output.
.LP
Multidimensional character string arrays are not handled well, since the CDL
syntax for breaking a long character string into several shorter lines is
weak.
.LP
There should be a way to specify that the data should be displayed in
`record' order, that is with the all the values for `record' variables
together that have the same value of the record dimension.

196
ncdump/ncdump.c
View File

@ -2047,43 +2047,55 @@ void adapt_url_for_cache(char **pathp) {
}
#endif
/** @page ncdump ncdump tool
@section NAME
ncdump - Convert netCDF file to text form (CDL)
/** @page ncdump \b ncdump tool - Convert netCDF file to text form (CDL)
@section SYNOPSIS
\code
ncdump [-chistxw] [-v var1,...] [-b lang] [-f lang]
[-l len] [-n name] [-p n[,n]] [-g grp1,...] file
ncdump -k file
ncdump -k file
\endcode
@section DESCRIPTION
ncdump generates a text representation of a specified netCDF
file on standard output. The text representation is in a
form called CDL (network Common Data form Language) that
can be viewed, edited, or serve as input to ncgen. ncgen is a
companion program that can generate a binary netCDF file from a
CDL file. Hence ncgen and ncdump can be used as inverses to
transform the data representation between binary and text
representations. See ncgen for a description of CDL and
netCDF representations.
The \b ncdump utility generates a text representation of a specified
netCDF file on standard output, optionally excluding some or all of
the variable data in the output. The text representation is in a form
called CDL (network Common Data form Language) that can be viewed,
edited, or serve as input to \b ncgen, a companion program that can
generate a binary netCDF file from a CDL file. Hence \b ncgen and \b
ncdump can be used as inverses to transform the data representation
between binary and text representations. See \b ncgen documentation
for a description of CDL and netCDF representations.
As of NetCDF version 4.1, and if DAP support was enabled when
ncdump was built, the file name may specify a DAP URL. This
allows ncdump to print out data sources from DAP servers. When
used with the -h option, ncdump can be used to show the
translation from the DAP DDS data model to the NetCDF data
model.
\b ncdump may also be used to determine what kind of netCDF file
is used (which variant of the netCDF file format) with the -k
option.
ncdump defines a default display format used for each type of
If DAP support was enabled when \b ncdump was built, the file name may
specify a DAP URL. This allows \b ncdump to access data sources from
DAP servers, including data in other formats than netCDF. When used
with DAP URLs, \b ncdump shows the translation from the DAP data
model to the netCDF data model.
\b ncdump may also be used as a simple browser for netCDF data files,
to display the dimension names and lengths; variable names, types, and
shapes; attribute names and values; and optionally, the values of data
for all variables or selected variables in a netCDF file. For
netCDF-4 files, groups and user-defined types are also included in \b
ncdump output.
\b ncdump uses '_' to represent data values that are equal to the
'_FillValue' attribute for a variable, intended to represent
data that has not yet been written. If a variable has no
'_FillValue' attribute, the default fill value for the variable
type is used unless the variable is of byte type.
\b ncdump defines a default display format used for each type of
netCDF data, but this can be changed if a `C_format' attribute
is defined for a netCDF variable. In this case, ncdump will
is defined for a netCDF variable. In this case, \b ncdump will
use the `C_format' attribute to format each value. For
example, if floating-point data for the netCDF variable `Z' is
known to be accurate to only three significant digits, it would
@ -2093,51 +2105,41 @@ be appropriate to use the variable attribute
Z:C_format = "%.3g"
\endcode
ncdump may also be used as a simple browser for netCDF data
files, to display the dimension names and sizes; variable
names, types, and shapes; attribute names and values; and
optionally, the values of data for all variables or selected
variables in a netCDF file.
ncdump uses '_' to represent data values that are equal to the
'_FillValue' attribute for a variable, intended to represent
data that has not yet been written. If a variable has no
'_FillValue' attribute, the default fill value for the variable
type is used if the variable is not of byte type.
ncdump may also be used to determine what kind of netCDF file
is used (which variant of the netCDF file format) with the -k
option.
@section OPTIONS
\b -c <br>
Show the values of \e coordinate \e variables (1D variables with the same
names as dimensions) as well as the declarations of all dimensions,
variables, and attribute values. Data values of non-coordinate
variables are not included in the output. This is the most suitable
option to use for a brief look at the structure and contents of a
netCDF file.
@par -c
Show the values of \e coordinate \e variables (1D variables with the
same names as dimensions) as well as the declarations of all
dimensions, variables, attribute values, groups, and user-defined
types. Data values of non-coordinate variables are not included in
the output. This is usually the most suitable option to use for a
brief look at the structure and contents of a netCDF file.
\b -h <br>
@par -h
Show only the header information in the output, that is, output only
the declarations for the netCDF dimensions, variables, and attributes
of the input file, but no data values for any variables. The output is
identical to using the '-c' option except that the values of
coordinate variables are not included. (At most one of '-c' or '-h'
options may be present.)
the declarations for the netCDF dimensions, variables, attributes,
groups, and user-defined types of the input file, but no data values
for any variables. The output is identical to using the '-c' option
except that the values of coordinate variables are not included. (At
most one of '-c' or '-h' options may be present.)
\b -v \e var1,... <br> The output will include data values for the
specified variables, in addition to the declarations of all
dimensions, variables, and attributes. One or more variables must be
specified by name in the comma-delimited list following this
option. The list must be a single argument to the command, hence
cannot contain unescaped blanks or other white space characters. The
named variables must be valid netCDF variables in the input-file. The
@par -v \a var1,...
@par
The output will include data values for the specified variables, in
addition to the declarations of all dimensions, variables, and
attributes. One or more variables must be specified by name in the
comma-delimited list following this option. The list must be a single
argument to the command, hence cannot contain unescaped blanks or
other white space characters. The named variables must be valid netCDF
variables in the input-file. A variable within a group in a netCDF-4
file may be specified with an absolute path name, such as
`/GroupA/GroupA2/var'. Use of a relative path name such as `var' or
`grp/var' specifies all matching variable names in the file. The
default, without this option and in the absence of the '-c' or '-h'
options, is to include data values for all variables in the output.
options, is to include data values for \e all variables in the output.
\b -b [c|f] <br>
@par -b [c|f]
A brief annotation in the form of a CDL comment (text beginning with
the characters '//') will be included in the data section of the
output for each 'row' of data, to help identify data values for
@ -2149,7 +2151,7 @@ varying fastest). In either case, the data will be presented in the
same order; only the annotations will differ. This option may be
useful for browsing through large volumes of multidimensional data.
\b -f [c|f] <br>
@par -f [c|f]
Full annotations in the form of trailing CDL comments (text beginning
with the characters '//') for every data value (except individual
characters in character arrays) will be included in the data
@ -2161,28 +2163,34 @@ be useful for piping data into other filters, since each data value
appears on a separate line, fully identified. (At most one of '-b' or
'-f' options may be present.)
\b -l \e length <br>
@par -l \e length
@par
Changes the default maximum line length (80) used in formatting lists
of non-character data values.
\b -n \e name <br>
@par -n \e name
@par
CDL requires a name for a netCDF file, for use by 'ncgen -b' in
generating a default netCDF file name. By default, ncdump
generating a default netCDF file name. By default, \b ncdump
constructs this name from the last component of the file name of
the input netCDF file by stripping off any extension it has. Use
the '-n' option to specify a different name. Although the output
file name used by 'ncgen -b' can be specified, it may be wise to
have ncdump change the default name to avoid inadvertently
overwriting a valuable netCDF file when using ncdump, editing the
have \b ncdump change the default name to avoid inadvertently
overwriting a valuable netCDF file when using \b ncdump, editing the
resulting CDL file, and using 'ncgen -b' to generate a new netCDF
file from the edited CDL file.
\b -p \e float_digits[,double_digits] <br>
@par -p \e float_digits[, \e double_digits ]
@par
Specifies default precision (number of significant digits) to use in
displaying floating-point or double precision data values for
attributes and variables. If specified, this value overrides the value
of the C_format attribute, if any, for a variable. Floating-point data
will be displayed with float_digits significant digits. If
will be displayed with \e float_digits significant digits. If \e
double_digits is also specified, double-precision values will be
displayed with that many significant digits. In the absence of any
'-p' specifications, floating-point and double-precision data are
@ -2190,14 +2198,16 @@ displayed with 7 and 15 significant digits respectively. CDL files can
be made smaller if less precision is required. If both floating-point
and double precisions are specified, the two values must appear
separated by a comma (no blanks) as a single argument to the command.
(To represent every last bit of precision in a CDL file for all
possible floating-point values would requires '-p 9,17'.)
\b -k <br>
@par -k
Show \e kind of netCDF file, that is which format variant the file uses.
Other options are ignored if this option is specified. Output will be
one of 'classic'. '64-bit offset', 'netCDF-4', or 'netCDF-4 classic
model'.
\b -s <br>
@par -s
Specifies that \e special virtual attributes should be output for the
file format variant and for variable properties such as
compression, chunking, and other properties specific to the format
@ -2207,10 +2217,10 @@ attributes begin with '_' followed by an upper-case
letter. Currently they include the global attribute '_Format' and
the variable attributes '_ChunkSizes', '_DeflateLevel',
'_Endianness', '_Fletcher32', '_NoFill', '_Shuffle', and '_Storage'.
The ncgen utility recognizes these attributes and
The \b ncgen utility recognizes these attributes and
supports them appropriately.
\b -t <br>
@par -t
Controls display of time data, if stored in a variable that uses a
udunits compliant time representation such as 'days since 1970-01-01'
or 'seconds since 2009-03-15 12:01:17'. If this option is specified,
@ -2223,11 +2233,13 @@ this option include the CF Conventions values 'gregorian' or
'standard', 'proleptic_gregorian', 'noleap' or '365_day', 'all_leap'
or '366_day', '360_day', and 'julian'.
\b -i <br>
@par -i
Same as the '-t' option, except output time data as date-time strings
with ISO-8601 standard 'T' separator, instead of a blank.
\b -g \e grp1,... <br>
@par -g \e grp1,...
@par
The output will include data values only for the specified groups.
One or more groups must be specified by name in the comma-delimited
list following this option. The list must be a single argument to the
@ -2236,11 +2248,11 @@ input-file. The default, without this option and in the absence of the
'-c' or '-h' options, is to include data values for all groups in the
output.
\b -w <br>
@par -w
For file names that request remote access using DAP URLs, access data
with client-side caching of entire variables.
\b -x <br>
@par -x
Output XML (NcML) instead of CDL. The NcML does not include data values.
The NcML output option currently only works for netCDF classic model data.
@ -2276,31 +2288,29 @@ omega:
ncdump -v omega -f fortran -n omega foo.nc > Z.cdl
\endcode
Examine the translated DDS for the DAP source from the specified URL.
Examine the translated DDS for the DAP source from the specified URL:
\code
ncdump -h http://test.opendap.org:8080/dods/dts/test.01
\endcode
Without dumping all the data, show the special virtual attributes that indicate
performance-related characterisitics of a netCDF-4 file:
\code
ncdump -h -s nc4file.nc
\endcode
@section see_also SEE ALSO
ncgen(1)
ncgen(1), netcdf(3)
@section string_note NOTE ON STRING OUTPUT
For classic, 64-bit offset or netCDF-4 classic model data, ncdump generates
line breaks after embedded newlines in displaying the character data. This
is not done for netCDF-4 files, because that would create an extra string in
a list of strings when read by ncgen, which supports real strings as a new
primitive type.
@section REFERENCES
What Every Computer Scientist should Know About Floating-Point
Arithmetic, D. Goldberg, ACM Computing Surveys, Vol. 23, No. 1, March
1991, pp. 5-48.
[Climate and Forecast Metadata Conventions](http://www.cfconventions.org/)
For classic, 64-bit offset or netCDF-4 classic model data, \b ncdump
generates line breaks after embedded newlines in displaying character
data. This is not done for netCDF-4 files, because netCDF-4 supports
arrays of real strings of varying length.
*/
int
main(int argc, char *argv[])