Go to file
Dennis Heimbigner f6e25b695e Fix additional S3 support issues
re: https://github.com/Unidata/netcdf-c/issues/2117
re: https://github.com/Unidata/netcdf-c/issues/2119

* Modify libsrc to allow byte-range reading of netcdf-3 files in private S3 buckets; this required using the aws sdk. Also add a test case.
* The aws sdk can sometimes cause problems if the Awd::ShutdownAPI function is not called. So at optional atexit() support to ensure it is called. This is disabled for Windows.
* Add documentation to nczarr.md on how to build and use the aws sdk under windows. Currently it builds, but testing fails.
* Switch testing from stratus to the Unidata bucket on S3.
* Improve support for the s3: url protocol.
* Add a s3 specific utility code file: ds3util.c
* Modify NC_infermodel to attempt to read the magic number of byte-ranged files in S3.

## Misc.

* Move and rename the core S3 SDK wrapper code (libnczarr/zs3sdk.cpp) to libdispatch since it now used in libsrc as well as libnczarr.
* Add calls to nc_finalize in the utilities in case atexit is disabled.
* Add header only json parser to the distribution rather than as a built source.
2021-10-29 20:06:37 -06:00
.github remove actions debug 2021-09-28 14:09:38 -06:00
cmake Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
conda.recipe Attempting appveyor support. Based on work by @ocefpaf, see https://github.com/Unidata/netcdf-c/pull/347 for more information. 2018-04-24 11:28:22 -06:00
ctest_scripts Added a ctest script with DAP tests enabled. 2020-02-11 15:09:29 -07:00
dap4_test ckp 2021-10-26 22:52:23 -06:00
docs Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
examples Remove need for HDF5-1.6 API being defined 2021-04-28 13:59:24 -06:00
fuzz Adds fuzz target for oss-fuzz integration 2021-03-31 11:27:27 +02:00
h5_test Remove need for HDF5-1.6 API being defined 2021-04-28 13:59:24 -06:00
hdf4_test Spelling fixes 2019-09-18 08:03:01 -06:00
include Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
libdap2 Merge branch 'main' into ejh_quantize_2 2021-09-07 10:44:45 -06:00
libdap4 Merge branch 'master' into zarrs3.dmh 2021-10-07 15:10:03 -06:00
libdispatch Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
libhdf4 perparing to apply bitgroom algorithm 2021-08-25 01:31:26 -06:00
libhdf5 Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
liblib Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
libncpoco Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
libnczarr Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
libsrc Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
libsrc4 Merge branch 'main' into ejh_quantize_2 2021-09-07 10:44:45 -06:00
libsrcp adding quantize functions to all the dispatch tables 2021-08-24 01:26:44 -06:00
nc_perf tinker with data algorithm for tst_compress_par.c 2021-09-09 07:53:55 -06:00
nc_test Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
nc_test4 Merge branch 'master' into zarrs3.dmh 2021-10-07 15:10:03 -06:00
ncdap_test Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
ncdump Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
ncgen update merge 2021-10-26 20:53:05 -06:00
ncgen3 Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
nctest Spelling fixes 2019-09-18 08:03:01 -06:00
nczarr_test Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
NUG Removed NUG files, added a README.md file directing people to the correct location. 2020-10-06 11:13:56 -06:00
oc2 Fix FreeBSD fileno problem in the ncgen parsers 2021-09-28 14:03:19 -06:00
plugins (1) improve INI parser (2) Fix make discheck 2021-09-30 13:45:09 -06:00
unit_test Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
.gitignore Fix JSON quoted string processing in libnczarr 2021-05-06 16:39:44 -06:00
acinclude.m4 Merge branch 'master' into ejh_cleanup 2018-04-02 18:11:58 -06:00
appveyor.yml unblock dmh appveyor 2020-12-02 18:17:07 -07:00
bootstrap Renamed autogen file. 2019-01-16 11:22:17 -07:00
cmake_uninstall.cmake.in Misc. files updated with copyright stanza. 2018-12-06 15:51:35 -07:00
CMakeInstallation.cmake Move away from deprecated PackageManager to the productbuild generator. 2019-02-05 13:08:30 -07:00
CMakeLists.txt Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
COMPILE.cmake.txt
config.h.cmake.in Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
config.h.cmake.in.old-works c: remove __CHAR_UNSIGNED__ 2018-02-14 17:24:49 -05:00
configure.ac Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
COPYRIGHT Modifying COPYRIGHT to reflect a standardized license, the BSD 3-clause license. 2018-12-04 15:17:54 -07:00
CTestConfig.cmake.in Adding new security stance to cdash instance. 2019-11-18 10:58:44 -07:00
CTestCustom.cmake Misc. files updated with copyright stanza. 2018-12-06 15:51:35 -07:00
dods.m4 Spelling fixes 2019-09-18 08:03:01 -06:00
FixBundle.cmake.in Misc. files updated with copyright stanza. 2018-12-06 15:51:35 -07:00
INSTALL.md Added a note about running autoreconf -if in support of https://github.com/Unidata/netcdf-c/issues/1397 2019-08-27 11:24:25 -06:00
lib_flags.am Fix out of tree builds with configure - add $(top_builddir)/include to include directory search 2021-04-10 17:57:55 -06:00
libnetcdf.settings.in Merge branch 'master' into ejh_quantize_2 2021-08-26 23:05:59 -06:00
Makefile.am Significantly Improve Amazon S3 Cloud Storage Support 2021-09-27 18:36:33 -06:00
mclean Fix pull request https://github.com/Unidata/netcdf-c/pull/374 (dap4.dmh) 2017-04-03 21:39:44 -06:00
nc-config.cmake.in Improve the building of NCZARR S3 support in CMake and Autoconf 2020-10-16 15:04:51 -06:00
nc-config.in Improve the building of NCZARR S3 support in CMake and Autoconf 2020-10-16 15:04:51 -06:00
netcdf.pc.in Fix typo, exploring solution to https://github.com/Unidata/netcdf-c/issues/1324 2019-02-27 14:14:53 -07:00
netCDFConfig.cmake.in Correct typo. 2020-01-24 16:53:16 -07:00
PostInstall.cmake
postinstall.sh.in Fix typo postinstall. 2020-08-06 15:27:20 -04:00
README.md Update links to netCDF-java documentation. 2021-02-26 15:11:10 -07:00
RELEASE_NOTES.md Fix additional S3 support issues 2021-10-29 20:06:37 -06:00
test_common.in Add filter support to NCZarr 2021-09-02 17:04:26 -06:00
test_prog.c Misc. files updated with copyright stanza. 2018-12-06 15:51:35 -07:00
test-driver-verbose The Jetstream remote test server is now working. 2018-06-26 13:58:45 -06:00
travis.yml Additional Fixes to NCZarr 2020-12-16 20:48:02 -07:00
wjna

Unidata NetCDF

Build Status Coverity Scan Build Status

About

The Unidata network Common Data Form (netCDF) is an interface for scientific data access and a freely-distributed software library that provides an implementation of the interface. The netCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data. The current netCDF software provides C interfaces for applications and data. Separate software distributions available from Unidata provide Java, Fortran, Python, and C++ interfaces. They have been tested on various common platforms.

Properties

NetCDF files are self-describing, network-transparent, directly accessible, and extendible. Self-describing means that a netCDF file includes information about the data it contains. Network-transparent means that a netCDF file is represented in a form that can be accessed by computers with different ways of storing integers, characters, and floating-point numbers. Direct-access means that a small subset of a large dataset may be accessed efficiently, without first reading through all the preceding data. Extendible means that data can be appended to a netCDF dataset without copying it or redefining its structure.

Use

NetCDF is useful for supporting access to diverse kinds of scientific data in heterogeneous networking environments and for writing application software that does not depend on application-specific formats. For information about a variety of analysis and display packages that have been developed to analyze and display data in netCDF form, see

More information

For more information about netCDF, see

Latest releases

You can obtain a copy of the latest released version of netCDF software for various languages:

Copyright and licensing information can be found here, as well as in the COPYRIGHT file accompanying the software

Installation

To install the netCDF-C software, please see the file INSTALL in the netCDF-C distribution, or the (usually more up-to-date) document:

Documentation

A language-independent User's Guide for netCDF, and some other language-specific user-level documents are available from:

A mailing list, netcdfgroup@unidata.ucar.edu, exists for discussion of the netCDF interface and announcements about netCDF bugs, fixes, and enhancements. For information about how to subscribe, see the URL

Feedback

We appreciate feedback from users of this package. Please send comments, suggestions, and bug reports to support-netcdf@unidata.ucar.edu.