mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2024-12-21 08:39:46 +08:00
Code Issues Packages Projects Releases Wiki Activity
7121588ac0
netcdf-c/ncdump
History
Dennis Heimbigner 49737888ca Improve S3 Documentation and Support 
## Improvements to S3 Documentation
* Create a new document *quickstart_paths.md* that give a summary of the legal path formats used by netcdf-c. This includes both file paths and URL paths.
* Modify *nczarr.md* to remove most of the S3 related text.
* Move the S3 text from *nczarr.md* to a new document *cloud.md*.
* Add some S3-related text to the *byterange.md* document.

Hopefully, this will make it easier for users to find the information they want.

## Rebuild NCZarr Testing
In order to avoid problems with running make check in parallel, two changes were made:
1. The *nczarr_test* test system was rebuilt. Now, for each test.
any generated files are kept in a test-specific directory, isolated
from all other test executions.
2. Similarly, since the S3 test bucket is shared, any generated S3 objects
are isolated using a test-specific key path.

## Other S3 Related Changes
* Add code to ensure that files created on S3 are reclaimed at end of testing.
* Used the bash "trap" command to ensure S3 cleanup even if the test fails.
* Cleanup the S3 related configure.ac flag set since S3 is used in several places. So now one should use the option *--enable-s3* instead of *--enable-nczarr-s3*, although the latter is still kept as a deprecated alias for the former.
* Get some of the github actions yml to work with S3; required fixing various test scripts adding a secret to access the Unidata S3 bucket.
* Cleanup S3 portion of libnetcdf.settings.in and netcdf_meta.h.in and test_common.in.
* Merge partial S3 support into dhttp.c.
* Create an experimental s3 access library especially for use with Windows. It is enabled by using the options *--enable-s3-internal* (automake) or *-DENABLE_S3_INTERNAL=ON* (CMake). Also add a unit-test for it.
* Move some definitions from ncrc.h to ncs3sdk.h

## Other Changes
* Provide a default implementation of strlcpy and move this and similar defaults into *dmissing.c*.
2023-04-25 17:15:06 -06:00
..
cdl Adding a missing file to make dist. 2022-02-01 11:46:01 -07:00
expected Adding a missing reference file. 2022-02-01 11:48:27 -07:00
acpget.c Improve UTF8 Support On Windows 2022-02-08 20:53:30 -07:00
bom.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
cdl.h ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
chunkspec.c Make utilities support NC_COMPACT 2020-02-29 12:06:21 -07:00
chunkspec.h Make utilities support NC_COMPACT 2020-02-29 12:06:21 -07:00
CMakeLists.txt "Simplify" XGetopt usage 2023-04-09 13:10:41 -06:00
depend
dimmap.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
dimmap.h ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
dumplib.c Merge branch 'master' into ncdumpvlenbug.dmh 2021-05-27 14:50:05 -06:00
dumplib.h Fixed Clang -Wstrict-prototypes warnings 2022-03-01 23:21:24 -05:00
env
indent.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
indent.h Fixed Clang -Wstrict-prototypes warnings 2022-03-01 23:21:24 -05:00
inttags4.cdl
inttags.cdl
L512.bin
list.c Extend nccopy -F option syntax. 2019-02-08 18:48:17 -07:00
list.h Extend nccopy -F option syntax. 2019-02-08 18:48:17 -07:00
Makefile.am Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
nc4print.c Codify cross-platform file paths 2021-03-04 13:41:31 -07:00
nc4printer.c Use the built-in HDF5 byte-range reader, if available. 2020-09-24 14:33:58 -06:00
nccomps.h ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
nccopy.1 More fixes to the nccopy filter x chunking algorithm 2021-01-31 15:10:39 -07:00
nccopy.c Enhance/Fix filter support 2022-03-14 12:39:37 -06:00
ncdump.1 Enhance/Fix filter support 2022-03-14 12:39:37 -06:00
ncdump.c Cleanup DAP4 testing 2022-11-13 13:15:11 -07:00
ncdump.h Cleanup DAP4 testing 2022-11-13 13:15:11 -07:00
ncfilteravail.c Two fixes for the price of one! 2022-04-28 15:51:20 -06:00
nchdf5version.c 1. Fix an additional flaw in fill_value handling where non-atomic default values were not properly being handled. 2022-01-10 15:27:16 -07:00
nciter.c This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
nciter.h ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
ncpathcvt.c Initialize potentially uninitialized variable 2022-06-07 14:28:01 -06:00
nctime0.c update against main 2023-01-28 13:45:35 -07:00
nctime0.h ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
nctrunc.c Improve UTF8 Support On Windows 2022-02-08 20:53:30 -07:00
ncvalidator.c Merge branch 'Unidata:main' into main 2022-05-13 14:16:15 +02:00
ocprint.c Improve UTF8 Support On Windows 2022-02-08 20:53:30 -07:00
printfqn.c Two fixes for the price of one! 2022-04-28 15:51:20 -06:00
ref1.ncml Docs migration 2021-11-11 10:47:49 -07:00
ref_ctest1_nc4.cdl
ref_ctest1_nc4c.cdl
ref_ctest64.c This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_ctest_small_3.c Add test cases 2019-11-03 12:03:13 -07:00
ref_ctest_small_4.c Add test cases 2019-11-03 12:03:13 -07:00
ref_ctest_special_atts_4.c 1. Fix an additional flaw in fill_value handling where non-atomic default values were not properly being handled. 2022-01-10 22:26:19 -07:00
ref_ctest.c This PR adds EXPERIMENTAL support for accessing data in the 2020-06-28 18:02:47 -06:00
ref_inttags4.cdl
ref_inttags.cdl
ref_keyword1.cdl Allow use of type keywords as identifier in formats that do not support that type. 2020-06-05 17:03:29 -06:00
ref_keyword2.cdl Allow use of type keywords as identifier in formats that do not support that type. 2020-06-05 17:03:29 -06:00
ref_keyword3.cdl Fix the handling of certain alias types on CDL files. 2021-04-13 16:56:43 -06:00
ref_keyword4.cdl Fix the handling of certain alias types on CDL files. 2021-04-13 16:56:43 -06:00
ref_nc_test_netcdf4_4_0.nc
ref_nc_test_netcdf4.cdl
ref_nccopy3_subset.nc
ref_nccopy_w.cdl Fix nccopy to use NC_PERSIST so that -w actually persists the output. 2019-03-15 12:05:27 -06:00
ref_no_ncproperty.nc Simplify libhdf5/nc5info.c to move to lazy parsing 2019-03-09 20:35:57 -07:00
ref_null_byte_padding_test.nc
ref_pathcvt.txt Support MSYS2/Mingw platform 2021-12-23 22:18:56 -07:00
ref_provenance_v1.nc There was a request to extend the provenance information 2018-08-25 21:44:41 -06:00
ref_rcapi.txt Make public a limited API for programmatic access to internal .rc tables 2022-06-17 14:35:12 -06:00
ref_rcmerge1.txt Fix a number of OS specific bugs 2021-11-03 12:49:54 -06:00
ref_rcmerge2.txt Fix a number of OS specific bugs 2021-11-03 12:49:54 -06:00
ref_rcmerge3.txt Fix a number of OS specific bugs 2021-11-03 12:49:54 -06:00
ref_roman_szip_simple.cdl added some ncdump tests for szip when it is present 2020-07-02 13:59:37 -06:00
ref_roman_szip_unlim.cdl added some ncdump tests for szip when it is present 2020-07-02 13:59:37 -06:00
ref_test_360_day_1900.cdl
ref_test_360_day_1900.nc
ref_test_365_day_1900.cdl
ref_test_365_day_1900.nc
ref_test_366_day_1900.cdl
ref_test_366_day_1900.nc
ref_test_corrupt_magic.nc
ref_times_nc4.cdl Added an ncdump -t calendars test using variable length string 2023-01-08 15:58:20 -07:00
ref_times.cdl Test ncdump time unit name case-insensitivity 2020-05-14 06:42:12 -06:00
ref_tst_charfill.cdl
ref_tst_comp2.cdl
ref_tst_comp.cdl
ref_tst_compounds2.cdl
ref_tst_compounds2.nc
ref_tst_compounds3.cdl
ref_tst_compounds3.nc
ref_tst_compounds4.cdl
ref_tst_compounds4.nc
ref_tst_enum_data.cdl
ref_tst_enum_undef.cdl Provide a default enum const when fill value does not match any enum const. 2022-07-17 14:32:31 -06:00
ref_tst_fillbug.cdl
ref_tst_format_att_64.cdl
ref_tst_format_att.cdl
ref_tst_group_data_v23.cdl
ref_tst_group_data.cdl
ref_tst_grp_spec0.cdl
ref_tst_grp_spec.cdl
ref_tst_irish_rover.nc
ref_tst_mud4_chars.cdl
ref_tst_mud4-bc.cdl
ref_tst_mud4.cdl
ref_tst_nans.cdl
ref_tst_nc4_utf8_4.cdl
ref_tst_ncf213.cdl Enhance/Fix filter support 2022-03-14 12:39:37 -06:00
ref_tst_nofilters.cdl More fixes to the nccopy filter x chunking algorithm 2021-01-31 15:10:39 -07:00
ref_tst_noncoord.cdl
ref_tst_opaque_data.cdl
ref_tst_perdimspecs.cdl Fix nccopy -c dim/x to actually use the dim/x value. 2020-09-01 13:44:24 -06:00
ref_tst_radix.cdl Fix ncgen handling of octal constants (with leading 0). 2019-02-25 20:57:23 -07:00
ref_tst_small.cdl
ref_tst_solar_1.cdl
ref_tst_solar_2.cdl
ref_tst_special_atts3.cdl
ref_tst_special_atts.cdl Enhance/Fix filter support 2022-03-14 12:39:37 -06:00
ref_tst_string_data.cdl
ref_tst_unicode.cdl
ref_tst_utf8_4.cdl
ref_tst_utf8.cdl Fix various problem around VLEN's 2022-01-08 18:30:00 -07:00
ref_tst_vlen_data2.cdl
ref_tst_vlen_data.cdl
rewrite-scalar.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
run_back_comp_tests.sh Fix race condition in ncdump (and other) tests. 2022-11-08 20:12:38 -07:00
run_cygutf8.sh Improve UTF8 Support On Windows 2022-02-08 20:53:30 -07:00
run_ncgen_nc4_tests.sh Fix various problem around VLEN's 2022-01-08 18:30:00 -07:00
run_ncgen_tests.sh replace USE_CDF5 with ENABLE_CDF5 2018-06-29 21:17:07 -05:00
run_tests.sh clean up 2019-03-12 09:54:16 -06:00
run_utf8_nc4_tests.sh
run_utf8_tests.sh
scope_ancestor_only.cdl Regularize the scoping of dimensions 2021-05-31 15:51:12 -06:00
scope_ancestor_subgroup.cdl Regularize the scoping of dimensions 2021-05-31 15:51:12 -06:00
scope_group_only.cdl Regularize the scoping of dimensions 2021-05-31 15:51:12 -06:00
scope_preorder.cdl Regularize the scoping of dimensions 2021-05-31 15:51:12 -06:00
small2.cdl
small.cdl
test0.cdl
test_corrupt_magic.cdl
test_keywords.sh Support MSYS2/Mingw platform 2021-12-23 22:18:56 -07:00
test_ncdump.sh Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
test_radix.sh Fix ncgen handling of octal constants (with leading 0). 2019-02-25 20:57:23 -07:00
test_rcmerge.sh Make public a limited API for programmatic access to internal .rc tables 2022-06-17 14:35:12 -06:00
test_scope.sh Improve filter installation process to avoid use of an extra shell script 2022-05-14 16:05:48 -06:00
test_unicode_directory.sh Codify cross-platform file paths 2021-03-04 13:41:31 -07:00
test_unicode_path.sh Improve UTF8 Support On Windows 2022-02-08 20:53:30 -07:00
testpathcvt.sh Update to latest main 2023-04-04 18:37:20 -06:00
tst_64bit.sh replace USE_CDF5 with ENABLE_CDF5 2018-06-29 21:17:07 -05:00
tst_bom.sh Fix race condition in ncdump (and other) tests. 2022-11-08 20:12:38 -07:00
tst_brecs.cdl
tst_bug321.cdl
tst_calendars_nc4.cdl Added an ncdump -t calendars test using variable length string 2023-01-08 15:58:20 -07:00
tst_calendars_nc4.sh Added an ncdump -t calendars test using variable length string 2023-01-08 15:58:20 -07:00
tst_calendars.cdl Test ncdump time unit name case-insensitivity 2020-05-14 06:42:12 -06:00
tst_calendars.sh
tst_charfill.cdl
tst_charfill.sh
tst_chunking.c Explicitly disallow variable length type compression 2022-02-19 16:47:31 -07:00
tst_comp2.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_comp.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_compress.c Add a comment 2021-08-16 10:06:45 -06:00
tst_create_files.c added some ncdump tests for szip when it is present 2020-07-02 13:59:37 -06:00
tst_ctests.sh Add test cases 2019-11-03 12:03:13 -07:00
tst_cygutf8.c Improve UTF8 Support On Windows 2022-02-08 20:53:30 -07:00
tst_dimsizes.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_dimsizes.sh Add complete bitgroom support to NCZarr 2022-01-24 15:22:24 -07:00
tst_enum_data.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_enum_undef.c Provide a default enum const when fill value does not match any enum const. 2022-07-17 14:32:31 -06:00
tst_fileinfo.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_fileinfo.sh Fix undefined references when using Visual Studio 2020-05-18 19:36:28 -06:00
tst_fillbug.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_fillbug.sh
tst_formatx3.sh replace USE_CDF5 with ENABLE_CDF5 2018-06-29 21:17:07 -05:00
tst_formatx4.sh
tst_group_data.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_grp_spec.sh
tst_h_rdc0.c Remove need for HDF5-1.6 API being defined 2021-04-28 13:59:24 -06:00
tst_h_scalar.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_h_scalar.sh
tst_hdf5_offset.sh Fix undefined references when using Visual Studio 2020-05-18 19:36:28 -06:00
tst_inmemory_nc3.sh
tst_inmemory_nc4.sh
tst_inttags4.sh
tst_inttags.sh
tst_iter.sh
tst_lengths.sh replace USE_CDF5 with ENABLE_CDF5 2018-06-29 21:17:07 -05:00
tst_mslp.cdl
tst_mud.sh
tst_nans.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_nccopy3_subset.sh Swapped /bin/bash for /bin/sh to test on osx. 2018-08-12 23:01:08 -06:00
tst_nccopy3.sh Improve filter installation process to avoid use of an extra shell script 2022-05-14 16:05:48 -06:00
tst_nccopy4.sh Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
tst_nccopy5.sh Fix race condition in ncdump (and other) tests. 2022-11-08 20:12:38 -07:00
tst_nccopy_w3.sh Fix nccopy to use NC_PERSIST so that -w actually persists the output. 2019-03-15 12:05:27 -06:00
tst_nccopy_w4.sh Fix nccopy to use NC_PERSIST so that -w actually persists the output. 2019-03-15 12:05:27 -06:00
tst_ncgen4_classic.sh
tst_ncgen4_cycle.sh Fix handling of enum constants nested in compound types. 2021-09-12 18:03:57 -06:00
tst_ncgen4_diff.sh Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
tst_ncgen4.sh Swapped /bin/bash for /bin/sh to test on osx. 2018-08-12 23:01:08 -06:00
tst_ncgen_shared.sh Fix various problem around VLEN's 2022-01-08 18:30:00 -07:00
tst_ncml.cdl
tst_netcdf4_4.sh Enhance/Fix filter support 2022-03-14 12:39:37 -06:00
tst_netcdf4.sh Improve S3 Documentation and Support 2023-04-25 17:15:06 -06:00
tst_null_byte_padding.sh
tst_opaque_data.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_output.sh typo 2022-05-19 22:43:08 -06:00
tst_radix.cdl Fix ncgen handling of octal constants (with leading 0). 2019-02-25 20:57:23 -07:00
tst_rcapi.c Fix errors: 2022-06-20 14:09:05 -06:00
tst_rcmerge.c Make public a limited API for programmatic access to internal .rc tables 2022-06-17 14:35:12 -06:00
tst_special_atts.c Make utilities support NC_COMPACT 2020-02-29 12:06:21 -07:00
tst_string_data.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_unicode.c Explicitly disallow variable length type compression 2022-02-19 16:47:31 -07:00
tst_utf8.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
tst_vlen_data.c Ensure that netcdf_json.h does not interfere with ncjson. 2022-07-05 22:03:52 -06:00
tst_vlen_demo.c ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00
utils.c update against main 2023-01-28 13:45:35 -07:00
utils.h update against main 2023-01-28 13:45:35 -07:00
vardata.c Fix various problem around VLEN's 2022-01-08 18:30:00 -07:00
vardata.h ncdump, other directories updated with copyright stanza. 2018-12-06 15:36:53 -07:00