mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2025-01-18 15:55:12 +08:00
Code Issues Packages Projects Releases Wiki Activity
f6e25b695e
netcdf-c/nczarr_test
History
Dennis Heimbigner f6e25b695e Fix additional S3 support issues 
re: https://github.com/Unidata/netcdf-c/issues/2117
re: https://github.com/Unidata/netcdf-c/issues/2119

* Modify libsrc to allow byte-range reading of netcdf-3 files in private S3 buckets; this required using the aws sdk. Also add a test case.
* The aws sdk can sometimes cause problems if the Awd::ShutdownAPI function is not called. So at optional atexit() support to ensure it is called. This is disabled for Windows.
* Add documentation to nczarr.md on how to build and use the aws sdk under windows. Currently it builds, but testing fails.
* Switch testing from stratus to the Unidata bucket on S3.
* Improve support for the s3: url protocol.
* Add a s3 specific utility code file: ds3util.c
* Modify NC_infermodel to attempt to read the magic number of byte-ranged files in S3.

## Misc.

* Move and rename the core S3 SDK wrapper code (libnczarr/zs3sdk.cpp) to libdispatch since it now used in libsrc as well as libnczarr.
* Add calls to nc_finalize in the utilities in case atexit is disabled.
* Add header only json parser to the distribution rather than as a built source.
2021-10-29 20:06:37 -06:00
..
refs
bm_chunks3.c
bm_timer.h Add missing files so --enable-benchmarks works again. 2021-08-07 21:19:40 -06:00
bm_utils.c Add missing files so --enable-benchmarks works again. 2021-08-07 21:19:40 -06:00
bm_utils.h Add missing files so --enable-benchmarks works again. 2021-08-07 21:19:40 -06:00
CMakeLists.txt Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
Makefile.am Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ncdumpchunks.c Merge branch 'master' into eliminate_need_for_hdf5-1.6-API 2021-05-28 07:11:13 -06:00
ref_any.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_avail1.cdl More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_avail1.dmp More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_avail1.txt More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_blosc_zmap.txt Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_byte_fill_value_null.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_byte_fill_value_null.zarr.zip Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_byte.cdl Add test cases 2021-08-10 14:15:24 -06:00
ref_byte.zarr.zip Add test cases 2021-08-10 14:15:24 -06:00
ref_bzip2.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_fillonly.cdl
ref_filtered.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_groups_regular.cdl Add test cases 2021-08-10 14:15:24 -06:00
ref_groups.h5 Add test cases 2021-08-10 14:15:24 -06:00
ref_misc1.cdl
ref_misc1.dmp More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_misc2.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_multi.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_nczarr2zarr.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_ndims.cdl
ref_ndims.dmp More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_newformatpure.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_oldformat.cdl Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
ref_oldformat.zip Fix bad chunkpath calculation 2021-07-18 16:20:22 -06:00
ref_perdimspecs.cdl
ref_power_901_constants.cdl Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
ref_power_901_constants.zip Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
ref_purezarr_base.cdl Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_purezarr.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_quotes.cdl Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
ref_quotes.zip Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
ref_rem.cdl
ref_rem.dmp More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_skip.cdl
ref_skip.txt More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_skipw.cdl
ref_t_meta_dim1.cdl
ref_t_meta_var1.cdl
ref_ut_json_build.txt
ref_ut_json_parse.txt
ref_ut_map_create.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_map_readdata_nz4.txt
ref_ut_map_readmeta2.txt Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_ut_map_readmeta_nz4.txt
ref_ut_map_readmeta_nzf.txt
ref_ut_map_readmeta.txt Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_map_search_nz4.txt
ref_ut_map_search_nzf.txt
ref_ut_map_search.txt Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_ut_map_writedata.cdl Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_ut_map_writemeta2.cdl Add support for the XArray Zarr _ARRAY_DIMENSIONS attribute 2021-02-24 13:46:11 -07:00
ref_ut_map_writemeta.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_mapapi_create.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_mapapi_data.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_mapapi_meta.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_mapapi_search_nz4.txt
ref_ut_mapapi_search_nzf.txt
ref_ut_mapapi_search_s3.txt
ref_ut_mapapi_search.txt Add zip file support 2021-01-28 20:11:01 -07:00
ref_ut_proj.txt More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_ut_testmap_create.cdl Add zip file support 2021-01-28 20:11:01 -07:00
ref_whole.cdl More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_whole.txt More NCZarr optimizations 2021-01-06 13:35:59 -07:00
ref_xarray.cdl Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ref_zarr_test_data.cdl.gz Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
run_chunkcases.sh Upgrade the nczarr code to match Zarr V2 2021-04-24 19:48:15 -06:00
run_fillonlyz.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_filter.sh Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
run_interop.sh Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
run_it_chunks1.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_misc.sh Add test cases 2021-08-10 14:15:24 -06:00
run_nccopyz.sh More modification in support of #1940 2021-03-09 10:49:30 -07:00
run_ncgen4.sh Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
run_nczarr_fill.sh Add test cases 2021-08-10 14:15:24 -06:00
run_newformat.sh Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
run_perf_chunks1.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_purezarr.sh Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
run_s3_cleanup.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_specific_filters.sh Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
run_ut_chunk.sh
run_ut_map.sh Add zip file support 2021-01-28 20:11:01 -07:00
run_ut_mapapi.sh Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
run_ut_misc.sh
s3util.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
test_nczarr_utils.h Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
test_nczarr.sh Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
testfilter_misc.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
testfilter_multi.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
testfilter_order.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
testfilter_repeat.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
testfilter.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
tst_chunkcases.c Add zip file support 2021-01-28 20:11:01 -07:00
tst_fillonlyz.c Add zip file support 2021-01-28 20:11:01 -07:00
tst_nczfilter.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
tst_utils.c Merge branch 'master' into eliminate_need_for_hdf5-1.6-API 2021-05-28 07:11:13 -06:00
tst_utils.h Add zip file support 2021-01-28 20:11:01 -07:00
tst_zchunks2.c
tst_zchunks3.c Add zip file support 2021-01-28 20:11:01 -07:00
tst_zchunks.c
ut_chunking.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ut_includes.h Fix bad chunkpath calculation 2021-07-18 16:20:22 -06:00
ut_json.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ut_map.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ut_mapapi.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ut_projections.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ut_projtest.h
ut_test.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ut_test.h Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ut_util.c Add zip file support 2021-01-28 20:11:01 -07:00
ut_util.h
z4dump.c
zhex.c Add zip file support 2021-01-28 20:11:01 -07:00
zisjson.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
zmapio.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
zs3parse.c Fix Mingw versus XGetopt (again) 2021-05-26 14:27:27 -06:00