netcdf-c/libdispatch
Dennis Heimbigner f6e25b695e Fix additional S3 support issues
re: https://github.com/Unidata/netcdf-c/issues/2117
re: https://github.com/Unidata/netcdf-c/issues/2119

* Modify libsrc to allow byte-range reading of netcdf-3 files in private S3 buckets; this required using the aws sdk. Also add a test case.
* The aws sdk can sometimes cause problems if the Awd::ShutdownAPI function is not called. So at optional atexit() support to ensure it is called. This is disabled for Windows.
* Add documentation to nczarr.md on how to build and use the aws sdk under windows. Currently it builds, but testing fails.
* Switch testing from stratus to the Unidata bucket on S3.
* Improve support for the s3: url protocol.
* Add a s3 specific utility code file: ds3util.c
* Modify NC_infermodel to attempt to read the magic number of byte-ranged files in S3.

## Misc.

* Move and rename the core S3 SDK wrapper code (libnczarr/zs3sdk.cpp) to libdispatch since it now used in libsrc as well as libnczarr.
* Add calls to nc_finalize in the utilities in case atexit is disabled.
* Add header only json parser to the distribution rather than as a built source.
2021-10-29 20:06:37 -06:00
..
awsincludes.h Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
CMakeLists.txt Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
datt.c This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
dattget.c added anchors for reading_attributes and writing_attributes, and refs to them, also changed order of files in Doxygen.in 2019-11-08 05:19:51 -07:00
dattinq.c cleanup and minor fixes for attribute inq functions 2019-11-07 09:46:23 -07:00
dattput.c added anchors for reading_attributes and writing_attributes, and refs to them, also changed order of files in Doxygen.in 2019-11-08 05:19:51 -07:00
dauth.c Fix new lgtm alerts 2021-09-28 14:19:07 -06:00
daux.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
dcompound.c
dcopy.c Regularize the scoping of dimensions 2021-05-31 15:51:12 -06:00
dcrc32.c Make use of clock_gettime be conditional. 2020-12-06 18:19:53 -07:00
dcrc32.h Provide a Number of fixes/improvements to NCZarr 2020-11-19 17:01:04 -07:00
dcrc64.c Make use of clock_gettime be conditional. 2020-12-06 18:19:53 -07:00
ddim.c formatting cleanup 2019-02-25 07:58:55 -07:00
ddispatch.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
defcheck.c
denum.c
derror.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
dfile.c Codify cross-platform file paths 2021-03-04 13:41:31 -07:00
dfilter.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
dfilterx.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
dgroup.c
dhttp.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
dinfermodel.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
dinternal.c
dnotnc3.c final removal 2019-08-15 07:05:10 -06:00
dnotnc4.c adding quantize functions to all the dispatch tables 2021-08-24 01:26:44 -06:00
doffsets.c Thread safety: step 1: cleanup 2019-03-30 14:06:20 -06:00
dopaque.c
dparallel.c documentation improvements for nc_var_par_access() 2020-03-02 16:36:56 -07:00
dpathmgr.c Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
drc.c (1) improve INI parser (2) Fix make discheck 2021-09-30 13:45:09 -06:00
dreadonly.c removed NC from open in dispatch table 2019-08-01 14:30:20 -06:00
ds3util.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
dstring.c
dtype.c
durlmodel.c Prepare for the path management code 2020-10-13 19:12:15 -06:00
dutf8.c
dutil.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
dv2i.c
dvar.c now nsd of 0 is NC_EINVAL for nc_def_var_quantize() 2021-09-10 06:10:20 -06:00
dvarget.c removed non-relaxed coord bounds from test code 2019-11-26 06:20:34 -07:00
dvarinq.c Merge branch 'main' into ejh_quantize_2 2021-09-07 10:44:45 -06:00
dvarput.c now always relax! 2019-11-26 05:36:16 -07:00
dvlen.c
ezxml_extra.c Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
ezxml.c Patch errors 2021-10-12 14:03:48 -06:00
json.c
Makefile.am Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
nc.c Change (again), and hopefully simplify, the file model inference algorithm. 2019-09-29 12:59:28 -06:00
ncbytes.c Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
ncexhash.c Additional Fixes to NCZarr 2020-12-16 20:48:02 -07:00
nchashmap.c Avoid infinite loop for finding large prime values 2021-10-14 15:55:04 -06:00
ncjson.c Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
nclist.c Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
nclistmgr.c Change (again), and hopefully simplify, the file model inference algorithm. 2019-09-29 12:59:28 -06:00
nclog.c Add zip file support 2021-01-28 20:11:01 -07:00
ncs3sdk.cpp Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ncsettings.hdr
nctime.c Fix for :60 seconds in ncdump 2021-01-26 10:24:43 +02:00
ncuri.c Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
ncxcache.c Move to Version 2 NCZarr Extended Meta-Data 2021-07-17 16:55:30 -06:00
netcdf.m4
stubdef.c
u8.c
u8.h
utf8proc_data.c
utf8proc.c
utf8proc.h Spelling fixes 2019-09-18 08:03:01 -06:00