netcdf-c/nczarr_test/run_ut_mapapi.sh
Dennis Heimbigner d953899559 Move to Version 2 NCZarr Extended Meta-Data
re: https://github.com/zarr-developers/zarr-specs/issues/41

After discussions with the Zarr community, it was decided to
convert to a new representation of the NCZarr meta-data extensions: version 2.
These extensions store information necessary to mapping the Zarr data model
to the netcdf-4 data model.

The basic change is to remove the NCZarr specific objects: .nczarr, .nczgroup, .nczarray, and .nczattr.
The contents of these objects is moved into the corresponding existing Zarr objects as special keys. The mapping is as follows:

* ''.nczarr'' => ''/.zgroup/_NCZARR_SUPERBLOCK_''
* ''.nczgroup => ''.zgroup/_NCZARR_GROUP_''
* ''.nczarray => ''.zarray/_NCZARR_ARRAY_''
* ''.nczattr => ''.zattr/_NCZARR_ATTR_''

Backward compatibility is maintained by looking for the object ''/.nczarr''
and if found, then assuming that the dataset is in the older version 1 format.
This compatibility only supports reading of such version 1 datasets.

Documentation and test cases are also added.

Misc. Other Changes:
1. The json parsing code was added to the general library instead of nczarr only (ncjson.c, ncjson.h).
2. Improved support for different platform paths by allowing conversion
   to a single common path representation.
3. Add some new error codes.
4. Modify nccopy usage to mention the new chunking specification.
2021-07-17 16:55:30 -06:00

101 lines
2.3 KiB
Bash
Executable File

#!/bin/sh
if test "x$srcdir" = x ; then srcdir=`pwd`; fi
. ../test_common.sh
. "$srcdir/test_nczarr.sh"
#TR="-T10"
set -e
# Test map implementations for consistency at the zmap API
# level. This allows testing of implementations that do not admit
# of easy examination of the actual storage. For example it is
# not easy to look inside S3 storage except using the aws-cli, if
# available.
CMD="${execdir}/ut_mapapi${ext}"
testmapcreate() {
echo ""; echo "*** Test zmap create -k $1"
extfor "$1"
tag=mapapi
base="tmp_$tag"
fileargs $base
deletemap $1 $file
# Create the test file
$CMD $TR -k$1 -x create -f $file
cdl="ut_${tag}_create_${zext}.cdl"
ref="ref_ut_${tag}_create.cdl"
${ZMD} $fileurl > ./$cdl
diff -wb $srcdir/$ref ./$cdl
# delete the test file
$CMD $TR -k$1 -x delete -f $file
rm -f $cdl
mapstillexists $1
if test mapstillexists = 0 ; then exit 1; fi
# re-create the test file
$CMD -k$1 -x create -o $file
}
testmapmeta() {
echo ""; echo "*** Test zmap read/write meta -k $1"
extfor "$1"
tag=mapapi
base="tmp_$tag"
fileargs $base
$CMD $TR -k$1 -x simplemeta -f $file
cdl="ut_${tag}_meta_${zext}.cdl"
ref="ref_ut_${tag}_meta.cdl"
${ZMD} $fileurl > ./$cdl
diff -wb ${srcdir}/$ref ./$cdl
}
testmapdata() {
echo ""; echo "*** Test zmap read/write data -k $1"
extfor "$1"
tag=mapapi
base="tmp_$tag"
fileargs $base
$CMD $TR -k$1 -x "simpledata" -f $file
cdl="ut_${tag}_data_${zext}.cdl"
ref="ref_ut_${tag}_data.cdl"
${ZMD} $fileurl > ./$cdl
diff -wb ${srcdir}/$ref ./$cdl
}
testmapsearch() {
echo ""; echo "*** Test zmap search -k $1"
extfor "$1"
tag=mapapi
base="tmp_$tag"
fileargs $base
txt=ut_${tag}_search_$zext.txt
ref=ref_ut_${tag}_search.txt
rm -f $txt
$CMD $TR -k$1 -x "search" -f $file > $txt
diff -wb ${srcdir}/$ref ./$txt
}
main() {
echo ""
echo "*** Map Unit Testing"
echo ""; echo "*** Test zmap_file"
testmapcreate file; testmapmeta file; testmapdata file; testmapsearch file
if test "x$FEATURE_NCZARR_ZIP" = xyes ; then
echo ""; echo "*** Test zmap_zip"
testmapcreate zip; testmapmeta zip; testmapdata zip; testmapsearch zip
fi
if test "x$FEATURE_S3TESTS" = xyes ; then
echo ""; echo "*** Test zmap_s3sdk"
testmapcreate s3; testmapmeta s3; testmapdata s3; testmapsearch s3
fi
}
main