netcdf-c/libnczarr
Dennis Heimbigner 49737888ca Improve S3 Documentation and Support
## Improvements to S3 Documentation
* Create a new document *quickstart_paths.md* that give a summary of the legal path formats used by netcdf-c. This includes both file paths and URL paths.
* Modify *nczarr.md* to remove most of the S3 related text.
* Move the S3 text from *nczarr.md* to a new document *cloud.md*.
* Add some S3-related text to the *byterange.md* document.

Hopefully, this will make it easier for users to find the information they want.

## Rebuild NCZarr Testing
In order to avoid problems with running make check in parallel, two changes were made:
1. The *nczarr_test* test system was rebuilt. Now, for each test.
any generated files are kept in a test-specific directory, isolated
from all other test executions.
2. Similarly, since the S3 test bucket is shared, any generated S3 objects
are isolated using a test-specific key path.

## Other S3 Related Changes
* Add code to ensure that files created on S3 are reclaimed at end of testing.
* Used the bash "trap" command to ensure S3 cleanup even if the test fails.
* Cleanup the S3 related configure.ac flag set since S3 is used in several places. So now one should use the option *--enable-s3* instead of *--enable-nczarr-s3*, although the latter is still kept as a deprecated alias for the former.
* Get some of the github actions yml to work with S3; required fixing various test scripts adding a secret to access the Unidata S3 bucket.
* Cleanup S3 portion of libnetcdf.settings.in and netcdf_meta.h.in and test_common.in.
* Merge partial S3 support into dhttp.c.
* Create an experimental s3 access library especially for use with Windows. It is enabled by using the options *--enable-s3-internal* (automake) or *-DENABLE_S3_INTERNAL=ON* (CMake). Also add a unit-test for it.
* Move some definitions from ncrc.h to ncs3sdk.h

## Other Changes
* Provide a default implementation of strlcpy and move this and similar defaults into *dmissing.c*.
2023-04-25 17:15:06 -06:00
..
obsolete Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
CMakeLists.txt Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
Makefile.am Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
SimpleBinStream.h
zarr.c Check at nc_open if file appears to be in NCZarr/Zarr format. 2023-03-13 13:24:14 -06:00
zarr.h Add support for Zarr string type to NCZarr 2022-08-27 20:21:13 -06:00
zattr.c Add support for Zarr string type to NCZarr 2022-08-27 20:21:13 -06:00
zcache.h Add support for Zarr string type to NCZarr 2022-08-27 20:21:13 -06:00
zchunking.c More NCZarr optimizations 2021-01-06 13:35:59 -07:00
zchunking.h Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
zclose.c Add support for Zarr string type to NCZarr 2022-08-27 20:21:13 -06:00
zcreate.c Support MSYS2/Mingw platform 2021-12-23 22:18:56 -07:00
zcvt.c Fix some problems with PR https://github.com/Unidata/netcdf-c/pull/2492 2022-09-03 14:21:48 -06:00
zdebug.c Fix byterange handling of some URLS 2023-03-02 19:51:02 -07:00
zdebug.h Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
zdim.c Provide a Number of fixes/improvements to NCZarr 2020-11-19 17:01:04 -07:00
zdispatch.c Enhance/Fix filter support 2022-03-14 12:39:37 -06:00
zdispatch.h Enhance/Fix filter support 2022-03-14 12:39:37 -06:00
zfile.c Merge branch 'main' into bloscfix.dmh 2022-09-06 13:49:18 -06:00
zfilter.c Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
zfilter.h I made a major update to this PR with the following changes: 2022-05-19 22:00:40 -06:00
zgrp.c Provide a Number of fixes/improvements to NCZarr 2020-11-19 17:01:04 -07:00
zincludes.h Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
zinternal.c Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
zinternal.h Fix some problems with PR https://github.com/Unidata/netcdf-c/pull/2492 2022-09-03 14:21:48 -06:00
zmap_file.c Support MSYS2/Mingw platform 2021-12-23 22:18:56 -07:00
zmap_s3sdk.c Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
zmap_zip.c Support MSYS2/Mingw platform 2021-12-23 22:18:56 -07:00
zmap.c Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
zmap.h Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
zodom.c More NCZarr optimizations 2021-01-06 13:35:59 -07:00
zodom.h More NCZarr optimizations 2021-01-06 13:35:59 -07:00
zopen.c Support MSYS2/Mingw platform 2021-12-23 22:18:56 -07:00
zprov.c Additional Fixes to NCZarr 2020-12-16 20:48:02 -07:00
zprovenance.h
zs3sdk.cpp Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
zsync.c Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
ztype.c Add support for Zarr string type to NCZarr 2022-08-27 20:21:13 -06:00
zutil.c Fix DAP4 remotetest server 2022-11-15 20:29:21 -07:00
zvar.c Fix infinite loop in file inferencing 2022-12-18 13:18:00 -07:00
zwalk.c Fix some problems with PR https://github.com/Unidata/netcdf-c/pull/2492 2022-09-03 14:21:48 -06:00
zxcache.c Add support for Zarr string type to NCZarr 2022-08-27 20:21:13 -06:00