From 40a7398b21ccfd9941f52cc209a3c8fa3896212f Mon Sep 17 00:00:00 2001 From: Russ Rew Date: Sat, 13 Dec 2014 19:50:21 -0700 Subject: [PATCH] Added nccopy -kind_code documentation and deprecated -k format_number, need to do the same for ncgen --- docs/guide.dox | 81 +++++++++++++++++++++++++++++++------------------ ncdump/nccopy.1 | 37 +++++++++++----------- 2 files changed, 70 insertions(+), 48 deletions(-) diff --git a/docs/guide.dox b/docs/guide.dox index f5c544749..06d7ae66a 100644 --- a/docs/guide.dox +++ b/docs/guide.dox @@ -1234,7 +1234,7 @@ application is also included that provides a graphical user interface to capabilities similar to the C-based ncdump and ncgen utilities, as well as CF-compliance checking and many other features. -The + \section remote_client The Remote Data Access Client @@ -3256,9 +3256,9 @@ Copy a netCDF file, optionally changing format, compression, or chunking in the \subsection nccopy_SYNOPSIS nccopy synopsis \code -nccopy [-k kind] [-d n] [-s] [-c chunkspec] [-u] [-w] [-[v|V] var1,...] - [-[g|G] grp1,...] [-m bufsize] [-h chunk_cache] [-e cache_elems] - [-r] infile outfile +nccopy [-k kind_name] [-kind_code] [-d n] [-s] [-c chunkspec] [-u] [-w] + [-[v|V] var1,...] [-[g|G] grp1,...] [-m bufsize] [-h chunk_cache] + [-e cache_elems] [-r] infile outfile \endcode \subsection nccopy_DESCRIPTION nccopy description @@ -3274,6 +3274,17 @@ a netCDF-4 classic model file as well, permitting data compression, efficient schema changes, larger variable sizes, and use of other netCDF-4 features. +If no output format is specified, with either \b -k \Ikind_name +or \b -\Ikind_code, then the output will use the same +format as the input, unless the input is classic or 64-bit offset +and either chunking or compression is specified, in which case the +output will be netCDF-4 classic model format. Attempting +some kinds of format conversion will result in an error, if the +conversion is not possible. For example, an attempt to copy a +netCDF-4 file that uses features of the enhanced model, such as +groups or variable-length strings, to any of the other kinds of netCDF +formats that use the classic model will result in an error. + \b nccopy also serves as an example of a generic netCDF-4 program, with its ability to read any valid netCDF file and handle nested groups, strings, and user-defined types, including arbitrarily @@ -3286,25 +3297,34 @@ local netCDF files. \subsection nccopy_OPTIONS nccopy options -\par -k \e kind -Specifies the kind of file to be created (that is, the format variant) -and, by inference, the data model (i.e. netcdf-3 (classic) versus +\par -k \e kind_name +Use format name to specify the kind of file to be created +and, by inference, the data model (i.e. netcdf-3 (classic) or netcdf-4 (enhanced)). The possible arguments are as follows. \n - '1' or 'classic' => netCDF classic format \n - '2', '64-bit-offset', or '64-bit offset' => netCDF 64-bit format \n - '3', 'hdf5', 'netCDF-4', or 'enhanced' => netCDF-4 format (enhanced data model) \n - '4', 'hdf5-nc3', 'netCDF-4 classic model', or 'enhanced-nc3' => netCDF-4 classic model format \n + 'nc3' or 'classic' => netCDF classic format \n + 'nc6' or '64-bit-offset' => netCDF 64-bit format \n + 'nc4' or 'netCDF-4' => netCDF-4 format (enhanced data model) \n + 'nc7' or 'netCDF-4 classic model' => netCDF-4 classic model format \n \par -If no value for -k is specified, then the output will use the same -format as the input, except if the input is classic or 64-bit offset -and either chunking or compression is specified, in which case the -output will be netCDF-4 classic model format. Note that attempting -some kinds of format conversion will result in an error, if the -conversion is not possible. For example, an attempt to copy a -netCDF-4 file that uses features of the enhanced model, such as groups -or variable-length strings, to any of the other kinds of netCDF -formats that use the classic model will result in an error. +Note: The old format numbers '1', '2', '3', '4', equivalent +to the format names 'nc3', 'nc6', 'nc4', or 'nc7' respectively, are +also still accepted but deprecated, due to easy confusion between +format numbers and format names. + +\par -k \e kind_code +Use format numeric code (instead of format name) to specify the kind of file to be created +and, by inference, the data model (i.e. netcdf-3 (classic) versus +netcdf-4 (enhanced)). The numeric codes are: + 3 => netcdf classic format \n + 6 => netCDF 64-bit format \n + 4 => netCDF-4 format (enhanced data model) \n + 7 => netCDF-4 classic model format \n + +The numeric code "7" is used because "7=3+4", specifying the format +that uses the netCDF-3 data model for compatibility and the netCDF-4 +storage format for performance. Credit is due to NCO for use of these +numeric codes instead of the old and confusing format numbers. \par -d \e n For netCDF-4 output, including netCDF-4 classic model, specify @@ -3322,10 +3342,11 @@ the input by default. \par Note that \b nccopy requires all variables to be compressed using the -same compression level, but the API has no such restriction. With a -program you can customize compression for each variable independently. +same compression level, but the API has no such restriction. With +a program you can customize compression for each variable independently. -\par -s For netCDF-4 output, including netCDF-4 classic model, specify +\par -s +For netCDF-4 output, including netCDF-4 classic model, specify shuffling of variable data bytes before compression or after decompression. Shuffling refers to interlacing of bytes in a chunk so that the first bytes of all values are contiguous in storage, followed @@ -3360,21 +3381,21 @@ file can also greatly speedup access, by choosing chunk shapes that are appropriate for the most common access patterns. \par -The chunkspec argument is a string of comma-separated associations, +The \e chunkspec argument is a string of comma-separated associations, each specifying a dimension name, a '/' character, and optionally the corresponding chunk length for that dimension. No blanks should appear in the chunkspec string, except possibly escaped blanks that -are part of a dimension name. A chunkspec names dimensions along -which chunking is to take place, and omits dimensions which are -not to be chunked or for -which the default chunk length is desired. If a dimension name is +are part of a dimension name. A chunkspec names at +least one dimension, and may omit dimensions which are not to be +chunked or for which the default chunk length is desired. If a dimension name is followed by a '/' character but no subsequent chunk length, the actual dimension length is assumed. If copying a classic model file to a netCDF-4 output file and not naming all dimensions in the chunkspec, unnamed dimensions will also use the actual dimension length for the chunk length. An example of a chunkspec for variables that use 'm' and 'n' dimensions might be 'm/100,n/200' to specify 100 by 200 -chunks. +chunks. To see the chunking resulting from copying with a +chunkspec, use the '-s' option of ncdump on the output file. \par The chunkspec '/' that omits all dimension names and @@ -3517,7 +3538,7 @@ compression, to a netCDF-3 file classic.nc: \code nccopy -k classic compressed.nc classic.nc \endcode -Note that '1' could be used instead of 'classic'. +Note that 'nc3' could be used instead of 'classic'.

Remote Access to Data Subset

diff --git a/ncdump/nccopy.1 b/ncdump/nccopy.1 index f47986e61..fac1459da 100644 --- a/ncdump/nccopy.1 +++ b/ncdump/nccopy.1 @@ -1,8 +1,7 @@ .\" $Id: nccopy.1 400 2010-08-27 21:02:52Z russ $ .TH NCCOPY 1 "2012-03-08" "Release 4.2" "UNIDATA UTILITIES" .SH NAME -nccopy \- Copy a netCDF file to specified variant of netCDF format, -optionally compressing or chunking data in the output copy. +nccopy \- Copy a netCDF file, optionally changing format, compression, or chunking in the output. .SH SYNOPSIS .ft B .HP @@ -42,7 +41,7 @@ If no output format is specified, with either \fB-k \fP \fI kind_name\fP or \fB-\fP\fIkind_code\fP, then the output will use the same format as the input, unless the input is classic or 64-bit offset and either chunking or compression is specified, in which case the -output will be netCDF-4 classic model format. Note that attempting +output will be netCDF-4 classic model format. Attempting some kinds of format conversion will result in an error, if the conversion is not possible. For example, an attempt to copy a netCDF-4 file that uses features of the enhanced model, such as @@ -62,7 +61,7 @@ local netCDF files. .IP "\fB -k \fP \fI kind_name \fP" Use format name to specify the kind of file to be created and, by inference, the data model (i.e. netcdf-3 (classic) or -netcdf-4 (enhanced)). The possible arguments are as follows. +netcdf-4 (enhanced)). The possible arguments are: .RS .RS .IP "'nc3' or 'classic' => netCDF classic format" @@ -79,7 +78,7 @@ format numbers and format names. .IP "[\fB-\fP\fIkind_code\fP]" Use format numeric code (instead of format name) to specify the kind of file to be created and, by inference, the data model (i.e. netcdf-3 (classic) versus -netcdf-4 (enhanced)). The possible arguments are as follows. +netcdf-4 (enhanced)). The .RS .RS .IP "3 => netcdf classic format" @@ -88,11 +87,10 @@ netcdf-4 (enhanced)). The possible arguments are as follows. .IP "7 => netCDF-4 classic model format" .RE .RE -The mnemonic for the numeric code "7" is "7=3+4" to specify the format +The numeric code "7" is used because "7=3+4", specifying the format that uses the netCDF-3 data model for compatibility and the netCDF-4 -storage format (based on HDF5) for performance. Credit is due to NCO for -use of these numeric codes instead of the old and confusing format -numbers. +storage format for performance. Credit is due to NCO for use of these +numeric codes instead of the old and confusing format numbers. .IP "\fB -d \fP \fI n \fP" For netCDF-4 output, including netCDF-4 classic model, specify deflation level (level of compression) for variable data output. 0 @@ -107,16 +105,19 @@ specified and the input file has compressed variables, the compression will still be preserved in the output, using the same chunking as in the input by default. .IP -Note that specifying output deflation level with \fBnccopy\fP results in -all output variables compressed using -the same compression level, but the API has no such restriction. With +Note that \fBnccopy\fP requires all variables to be compressed using the +same compression level, but the API has no such restriction. With a program you can customize compression for each variable independently. .IP "\fB -s \fP" For netCDF-4 output, including netCDF-4 classic model, specify shuffling of variable data bytes before compression or after -decompression. This option is ignored unless a non-zero deflation -level is specified. Turning shuffling on sometimes improves -compression. +decompression. Shuffling refers to interlacing of bytes in a chunk so +that the first bytes of all values are contiguous in storage, followed +by all the second bytes, and so on, which often improves compression. +This option is ignored unless a non-zero deflation level is specified. +Using -d0 to specify no deflation on input data that has been +compressed and shuffled turns off both compression and shuffling in +the output. .IP "\fB -u \fP" Convert any unlimited size dimensions in the input to fixed size dimensions in the output. This can speed up variable-at-a-time @@ -142,7 +143,7 @@ The \fIchunkspec\fP argument is a string of comma-separated associations, each specifying a dimension name, a '/' character, and optionally the corresponding chunk length for that dimension. No blanks should appear in the chunkspec string, except possibly escaped -blanks that are part of a dimension name. A chunkspec must name at +blanks that are part of a dimension name. A chunkspec names at least one dimension, and may omit dimensions which are not to be chunked or for which the default chunk length is desired. If a dimension name is followed by a '/' character but no subsequent chunk @@ -214,7 +215,7 @@ performance, if the output fits in memory. .IP "\fB -h \fP \fI chunk_cache \fP" For netCDF-4 output, including netCDF-4 classic model, an integer or floating-point number that specifies the size in bytes of chunk cache -for each chunked variable. This is not a property of the file, but merely +allocated for each chunked variable. This is not a property of the file, but merely a performance tuning parameter for avoiding compressing or decompressing the same data multiple times while copying and changing chunk shapes. A suffix of K, M, G, or T multiplies the chunk cache @@ -228,7 +229,7 @@ cache size has been implemented yet. Using the '-w' option may provide better performance, if the output fits in memory. .IP "\fB -e \fP \fI cache_elems \fP" For netCDF-4 output, including netCDF-4 classic model, specifies -number of chunkss that the chunk cache can hold. A suffix of K, M, G, +number of chunks that the chunk cache can hold. A suffix of K, M, G, or T multiplies the number of chunks that can be held in the cache by one thousand, million, billion, or trillion, respectively. This is not a property of the file, but merely a performance tuning parameter for