Merge branch 'bug-endianness', which really had to do with nccopy -d0

and chunking issues.
This commit is contained in:
Russ Rew 2014-03-23 17:26:43 -06:00
commit aca696afb0
3 changed files with 32 additions and 11 deletions

View File

@ -3348,12 +3348,15 @@ Note that \b nccopy requires all variables to be compressed using the
same compression level, but the API has no such restriction. With a
program you can customize compression for each variable independently.
\par -s
For netCDF-4 output, including netCDF-4 classic model,
specify shuffling of variable data bytes before compression or after
decompression. This option is ignored unless a non-zero deflation
level is specified. Turning shuffling on sometimes improves
compression.
\par -s For netCDF-4 output, including netCDF-4 classic model, specify
shuffling of variable data bytes before compression or after
decompression. Shuffling refers to interlacing of bytes in a chunk so
that the first bytes of all values are contiguous in storage, followed
by all the second bytes, and so on, which often improves compression.
This option is ignored unless a non-zero deflation level is specified.
Using -d0 to specify no deflation on input data that has been
compressed and shuffled turns off both compression and shuffling in
the output.
\par -u
Convert any unlimited size dimensions in the input to fixed size
@ -3384,8 +3387,9 @@ The chunkspec argument is a string of comma-separated associations,
each specifying a dimension name, a '/' character, and optionally the
corresponding chunk length for that dimension. No blanks should
appear in the chunkspec string, except possibly escaped blanks that
are part of a dimension name. A chunkspec must name at least one
dimension, and may omit dimensions which are not to be chunked or for
are part of a dimension name. A chunkspec names dimensions along
which chunking is to take place, and omits dimensions which are
not to be chunked or for
which the default chunk length is desired. If a dimension name is
followed by a '/' character but no subsequent chunk length, the actual
dimension length is assumed. If copying a classic model file to a
@ -3393,7 +3397,13 @@ netCDF-4 output file and not naming all dimensions in the chunkspec,
unnamed dimensions will also use the actual dimension length for the
chunk length. An example of a chunkspec for variables that use
'm' and 'n' dimensions might be 'm/100,n/200' to specify 100 by 200
chunks. To see the chunking resulting from copying with a chunkspec,
chunks.
\par
The chunkspec '/' that omits all dimension names and
corresponding chunk lengths specifies that no chunking is to occur in
the output, so can be used to unchunk all the chunked variables.
To see the chunking resulting from copying with a chunkspec,
use the '-s' option of ncdump on the output file.
\par

View File

@ -823,7 +823,6 @@ copy_var(int igrp, int varid, int ogrp)
/* define the output variable */
NC_CHECK(nc_def_var(ogrp, name, o_typeid, ndims, odimids, &o_varid));
/* attach the variable attributes to the output variable */
NC_CHECK(copy_atts(igrp, varid, ogrp, o_varid));
#ifdef USE_NETCDF4

View File

@ -44,7 +44,16 @@ echo "*** Test nccopy -d1 -s can compress a netCDF-4 file even more ..."
if test `wc -c < tmp.nc` -ge `wc -c < tst_inflated4.nc`; then
exit 1
fi
rm tst_deflated.nc tst_inflated.nc tst_inflated4.nc tmp.nc
echo "*** Test nccopy -d0 turns off compression, shuffling of compressed, shuffled file ..."
./nccopy -d0 tst_inflated4.nc tmp.nc
./ncdump -sh tmp.nc > tmp.cdl
if fgrep '_DeflateLevel' < tmp.cdl ; then
exit 1
fi
if fgrep '_Shuffle' < tmp.cdl ; then
exit 1
fi
rm tst_deflated.nc tst_inflated.nc tst_inflated4.nc tmp.nc tmp.cdl
echo "*** Testing nccopy -d1 -s on ncdump/*.nc files"
for i in $TESTFILES ; do
@ -66,6 +75,9 @@ diff tmp.cdl tmp-chunked.cdl
./nccopy -c dim0/,dim1/,dim2/,dim3/,dim4/,dim5/,dim6/ tmp-chunked.nc tmp-unchunked.nc
./ncdump -n tmp tmp-unchunked.nc > tmp-unchunked.cdl
diff tmp.cdl tmp-unchunked.cdl
./nccopy -c / tmp-chunked.nc tmp-unchunked.nc
./ncdump -n tmp tmp-unchunked.nc > tmp-unchunked.cdl
diff tmp.cdl tmp-unchunked.cdl
# echo "*** Test that nccopy compression with chunking can improve compression"
rm tst_chunking.nc tmp.nc tmp.cdl tmp-chunked.nc tmp-chunked.cdl tmp-unchunked.nc tmp-unchunked.cdl