mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2025-02-05 16:20:10 +08:00
Code Issues Packages Projects Releases Wiki Activity
73a94362dd
netcdf-c/nczarr_test
History
Dennis Heimbigner d953899559 Move to Version 2 NCZarr Extended Meta-Data 
re: https://github.com/zarr-developers/zarr-specs/issues/41

After discussions with the Zarr community, it was decided to
convert to a new representation of the NCZarr meta-data extensions: version 2.
These extensions store information necessary to mapping the Zarr data model
to the netcdf-4 data model.

The basic change is to remove the NCZarr specific objects: .nczarr, .nczgroup, .nczarray, and .nczattr.
The contents of these objects is moved into the corresponding existing Zarr objects as special keys. The mapping is as follows:

* ''.nczarr'' => ''/.zgroup/_NCZARR_SUPERBLOCK_''
* ''.nczgroup => ''.zgroup/_NCZARR_GROUP_''
* ''.nczarray => ''.zarray/_NCZARR_ARRAY_''
* ''.nczattr => ''.zattr/_NCZARR_ATTR_''

Backward compatibility is maintained by looking for the object ''/.nczarr''
and if found, then assuming that the dataset is in the older version 1 format.
This compatibility only supports reading of such version 1 datasets.

Documentation and test cases are also added.

Misc. Other Changes:
1. The json parsing code was added to the general library instead of nczarr only (ncjson.c, ncjson.h).
2. Improved support for different platform paths by allowing conversion
   to a single common path representation.
3. Add some new error codes.
4. Modify nccopy usage to mention the new chunking specification.
2021-07-17 16:55:30 -06:00
..
refs This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
bm_chunks3.c Additional Fixes to NCZarr 2020-12-16 20:48:02 -07:00
bm_utils.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
CMakeLists.txt Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
Makefile.am Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
ncdumpchunks.c Merge branch 'master' into eliminate_need_for_hdf5-1.6-API 2021-05-28 07:11:13 -06:00
ref_avail1.cdl More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_avail1.dmp More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_avail1.txt More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_fillonly.cdl Fix error where not converting fill data 2020-09-12 14:49:59 -06:00
ref_misc1.cdl Fix some additional edges cases for mapping slices to chunks 2020-12-19 21:17:46 -07:00
ref_misc1.dmp More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_misc2.cdl Upgrade the nczarr code to match Zarr V2 2021-04-24 19:48:15 -06:00
ref_nczarr2zarr.cdl Upgrade the nczarr code to match Zarr V2 2021-04-24 19:48:15 -06:00
ref_ndims.cdl Fix some additional edges cases for mapping slices to chunks 2020-12-19 21:17:46 -07:00
ref_ndims.dmp More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_newformatpure.cdl Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
ref_oldformat.cdl Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
ref_oldformat.zip Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
ref_perdimspecs.cdl Provide a Number of fixes/improvements to NCZarr 2020-11-19 17:01:04 -07:00
ref_power_901_constants.cdl Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
ref_power_901_constants.zip Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
ref_purezarr_base.cdl Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_purezarr.cdl Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_quotes.cdl Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
ref_quotes.zip Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
ref_rem.cdl Fix some additional edges cases for mapping slices to chunks 2020-12-19 21:17:46 -07:00
ref_rem.dmp More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_skip.cdl Additional Fixes to NCZarr 2020-12-16 20:48:02 -07:00
ref_skip.txt More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_skipw.cdl Fix some additional edges cases for mapping slices to chunks 2020-12-19 21:17:46 -07:00
ref_t_meta_dim1.cdl This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_t_meta_var1.cdl This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_ut_json_build.txt This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_ut_json_parse.txt This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_ut_map_create.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_map_readdata_nz4.txt This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_ut_map_readmeta2.txt Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_ut_map_readmeta_nz4.txt This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_ut_map_readmeta_nzf.txt This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_ut_map_readmeta.txt Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_map_search_nz4.txt Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries"" 2020-08-17 19:15:47 -06:00
ref_ut_map_search_nzf.txt Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries"" 2020-08-17 19:15:47 -06:00
ref_ut_map_search.txt Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_ut_map_writedata.cdl Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_ut_map_writemeta2.cdl Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_ut_map_writemeta.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_mapapi_create.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_mapapi_data.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_mapapi_meta.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_mapapi_search_nz4.txt Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries"" 2020-08-17 19:15:47 -06:00
ref_ut_mapapi_search_nzf.txt Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries"" 2020-08-17 19:15:47 -06:00
ref_ut_mapapi_search_s3.txt This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_ut_mapapi_search.txt Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_proj.txt More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_ut_testmap_create.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_whole.cdl More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_whole.txt More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_xarray.cdl Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
run_chunkcases.sh Upgrade the nczarr code to match Zarr V2 2021-04-24 19:48:15 -06:00
run_fillonlyz.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_interop.sh Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
run_it_chunks1.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_misc.sh Allow .rc tests to work in parallel by isolation 2021-04-25 22:02:29 -06:00
run_nccopyz.sh More modification in support of #1940 2021-03-09 10:49:30 -07:00
run_ncgen4.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_newformat.sh Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
run_perf_chunks1.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_purezarr.sh Upgrade the nczarr code to match Zarr V2 2021-04-24 19:48:15 -06:00
run_s3_cleanup.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_ut_chunk.sh Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries"" 2020-08-17 19:15:47 -06:00
run_ut_map.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_ut_mapapi.sh Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
run_ut_misc.sh Revert "Revert "Fix nczarr-experimental: improve build support, disengage hdf5 vs netcdf4 flags, and find AWS libraries"" 2020-08-17 19:15:47 -06:00
s3util.c Fix Mingw versus XGetopt (again) 2021-05-26 14:27:27 -06:00
test_nczarr_utils.h Fix Mingw versus XGetopt (again) 2021-05-26 14:27:27 -06:00
test_nczarr.sh Allow .rc tests to work in parallel by isolation 2021-04-25 22:02:29 -06:00
tst_chunkcases.c Add zip file support 2021-01-28 20:11:01 -07:00
tst_fillonlyz.c Add zip file support 2021-01-28 20:11:01 -07:00
tst_utils.c Merge branch 'master' into eliminate_need_for_hdf5-1.6-API 2021-05-28 07:11:13 -06:00
tst_utils.h Add zip file support 2021-01-28 20:11:01 -07:00
tst_zchunks2.c Additional Fixes to NCZarr 2020-12-16 20:48:02 -07:00
tst_zchunks3.c Add zip file support 2021-01-28 20:11:01 -07:00
tst_zchunks.c Additional Fixes to NCZarr 2020-12-16 20:48:02 -07:00
ut_chunking.c Fix some additional edges cases for mapping slices to chunks 2020-12-19 21:17:46 -07:00
ut_includes.h Add zip file support 2021-01-28 20:11:01 -07:00
ut_json.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
ut_map.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
ut_mapapi.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
ut_projections.c Fix some additional edges cases for mapping slices to chunks 2020-12-19 21:17:46 -07:00
ut_projtest.h This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ut_test.c Fix Mingw versus XGetopt (again) 2021-05-26 14:27:27 -06:00
ut_test.h Provide a Number of fixes/improvements to NCZarr 2020-11-19 17:01:04 -07:00
ut_util.c Add zip file support 2021-01-28 20:11:01 -07:00
ut_util.h Fix some additional edges cases for mapping slices to chunks 2020-12-19 21:17:46 -07:00
z4dump.c This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
zhex.c Add zip file support 2021-01-28 20:11:01 -07:00
zisjson.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
zmapio.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
zs3parse.c Fix Mingw versus XGetopt (again) 2021-05-26 14:27:27 -06:00