netcdf-c/nczarr_test/run_newformat.sh
Dennis Heimbigner d953899559 Move to Version 2 NCZarr Extended Meta-Data
re: https://github.com/zarr-developers/zarr-specs/issues/41

After discussions with the Zarr community, it was decided to
convert to a new representation of the NCZarr meta-data extensions: version 2.
These extensions store information necessary to mapping the Zarr data model
to the netcdf-4 data model.

The basic change is to remove the NCZarr specific objects: .nczarr, .nczgroup, .nczarray, and .nczattr.
The contents of these objects is moved into the corresponding existing Zarr objects as special keys. The mapping is as follows:

* ''.nczarr'' => ''/.zgroup/_NCZARR_SUPERBLOCK_''
* ''.nczgroup => ''.zgroup/_NCZARR_GROUP_''
* ''.nczarray => ''.zarray/_NCZARR_ARRAY_''
* ''.nczattr => ''.zattr/_NCZARR_ATTR_''

Backward compatibility is maintained by looking for the object ''/.nczarr''
and if found, then assuming that the dataset is in the older version 1 format.
This compatibility only supports reading of such version 1 datasets.

Documentation and test cases are also added.

Misc. Other Changes:
1. The json parsing code was added to the general library instead of nczarr only (ncjson.c, ncjson.h).
2. Improved support for different platform paths by allowing conversion
   to a single common path representation.
3. Add some new error codes.
4. Modify nccopy usage to mention the new chunking specification.
2021-07-17 16:55:30 -06:00

43 lines
1.0 KiB
Bash
Executable File

#!/bin/sh
if test "x$srcdir" = x ; then srcdir=`pwd`; fi
. ../test_common.sh
. "$srcdir/test_nczarr.sh"
set -e
echo ""
echo "*** Testing backward compatibilty between nczarr meta data format V1 vs V2"
testcaseold() {
zext=$1
fileargs ${srcdir}/ref_oldformat
${NCDUMP} -n ref_oldformat "$fileurl" > ./tmp_oldformat.cdl
diff -w ${srcdir}/ref_oldformat.cdl ./tmp_oldformat.cdl
}
testcasecvt() {
zext=$1
fileargs ${srcdir}/ref_oldformat
${NCCOPY} "$fileurl" "file://tmp_newformat.file#mode=nczarr,file"
${NCDUMP} -n ref_oldformat "file://tmp_newformat.file#mode=nczarr,file" > ./tmp_newformat.cdl
diff -w ${srcdir}/ref_oldformat.cdl ./tmp_newformat.cdl
}
testcasepure() {
zext=$1
fileargs ${srcdir}/ref_oldformat
${NCCOPY} "$fileurl" "file://tmp_newformat.file#mode=nczarr,file"
${NCDUMP} -n ref_oldformat "file://tmp_newformat.file#mode=zarr,file" > ./tmp_newpure.cdl
diff -w ${srcdir}/ref_newformatpure.cdl ./tmp_newpure.cdl
}
if test "x$FEATURE_NCZARR_ZIP" = xyes ; then
testcaseold zip
testcasecvt zip
testcasepure zip
fi
exit 0