netcdf-c/include/ncdap.h

70 lines
2.4 KiB
C
Raw Normal View History

Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
/*********************************************************************
* Copyright 2018, UCAR/Unidata
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
* See netcdf/COPYRIGHT file for copying and redistribution conditions.
*********************************************************************/
#ifndef NCDAP_H
#define NCDAP_H 1
#ifndef nullfree
#define nullfree(m) {if((m)!=NULL) {free(m);} else {}}
#endif
/**************************************************/
/*
Collect single bit flags that
affect the operation of the system.
*/
typedef unsigned int NCFLAGS;
# define SETFLAG(controls,flag) (((controls).flags) |= (flag))
# define CLRFLAG(controls,flag) (((controls).flags) &= ~(flag))
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
# define FLAGSET(controls,flag) (((controls.flags) & (flag)) != 0)
/* Defined flags */
#define NCF_NC3 (0x0001) /* DAP->netcdf-3 */
#define NCF_NC4 (0x0002) /* DAP->netcdf-4 */
#define NCF_NCDAP (0x0004) /* Do libnc-dap mimic */
#define NCF_CACHE (0x0008) /* Cache enabled/disabled */
#define NCF_UPGRADE (0x0010) /* Do proper type upgrades */
#define NCF_UNCONSTRAINABLE (0x0020) /* Not a constrainable URL */
#define NCF_SHOWFETCH (0x0040) /* show fetch calls */
#define NCF_ONDISK (0x0080) /* cause oc to store data on disk */
#define NCF_WHOLEVAR (0x0100) /* retrieve only whole variables (as opposed to partial variable) into cache */
#define NCF_PREFETCH (0x0200) /* Cache prefetch enabled/disabled */
#define NCF_PREFETCH_EAGER (0x0400) /* Do eager prefetch; 0=>lazy */
#define NCF_PREFETCH_ALL (0x0800) /* Prefetch all variables */
/* Allow _FillValue/Variable type mismatch */
#define NCF_FILLMISMATCH (0x1000)
Yet another fix for DAP2 double URL encoding. re: https://github.com/Unidata/netcdf-c/issues/1876 and: https://github.com/Unidata/netcdf-c/pull/1835 and: https://github.com/Unidata/netcdf4-python/issues/1041 The change in PR 1835 was correct with respect to using %20 instead of '+' for encoding blanks. However, it was a mistake to assume everything was unencoded and then to do encoding ourselves. The problem is that different servers do different things, with Columbia being an outlier. So, I have added a set of client controls that can at least give the caller some control over this. The caller can append the following fragment to his URL to control what gets encoded before sending it to the server. The syntax is as follows: ```` https://<host>/<path>/<query>#encode=path|query|all|none ```` The possible values: * path -- URL encode (i.e. %xx encode) as needed in the path part of the URL. * query -- URL encode as needed in the query part of the URL. * all -- equivalent to ````#encode=path,query````. * none -- do not url encode any part of the URL sent to the server; not strictly necessary, so mostly for completeness. Note that if "encode=" is used, then before it is processed, all encoding is turned of so that ````#encode=path```` will only encode the path and not the query. The default is ````#encode=query````, so the path is left untouched, but the query is always encoded. Internally, this required changes to pass the encode flags down into the OC2 library. Misc. Unrelated Changes: * Shut up those irritating warning from putget.m4
2020-11-06 02:04:56 +08:00
/* Hack to control URL encoding */
#define NCF_ENCODE_PATH (0x2000)
Cleanup DAP4 testing NOTE: This PR should not be included in 4.9.1 since additional DAP4 related PRs will be forthcoming. This PR makes major changes to libdap4 and dap4_test driven by changes to TDS. * Enable DAP4 * Clean up the test input files and the test baseline comparison files. This entails: * Remove a multitude of unused test input and baseline data files; among them are dap4_test/: daptestfiles, dmrtestfiles, nctestfiles, and misctestfiles. * Define a canonical set of test input files and record in dap4_test/cdltestfiles. * Use the cdltestfiles to generate the .nc test inputs. This set of .nc files is then moved to the d4ts (DAP4 test server) war file in the tds repository. This set then becomes the canonical set of DAP4 test sources. * Scrape d4ts to obtain copies of the raw streams of DAP4 encoded data. The .dmr and .dap streams are then stored in dap4_test/rawtestfiles. * Disable some remote server tests until those servers are fixed. * Add an option to ncdump (-XF) that forces the type of the _FillValue attribute; this is primarily to simplify testing of fill mismatch. * Minor bug fixes to ncgen. * Changes to libdap4: * Replace old checksum hack with the dap4.checksum flag. * Support the dap4.XXX controls. * Cleanup _FillValue handling, especially var-attribute type mismatches. * Fix enum handling based on changes to netcdf-java. * Changes to dap4_test: * Add getopt support to various test support programs. * Remove unneeded shell scripts. * Add new scripts: test_curlopt.sh
2022-11-14 04:15:11 +08:00
#define NCF_ENCODE_QUERY (0x4000)
#define NCF_FILLMISMATCH_FAIL (0x8000)
/* Put these at top bits for now */
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
/*COLUMBIA_HACK*/
#define NCF_COLUMBIA (0x80000000) /* Hack for columbia server */
/* OPeNDAP Hyrax Hack */
#define NCF_HYRAX (0x40000000) /* Hack for hyrax server -- use checksum, but ignore */
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
/* Define all the default on flags */
#define DFALT_ON_FLAGS (NCF_CACHE|NCF_PREFETCH|NCF_FILLMISMATCH)
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
#define ALL_NCF_FLAGS (0xffff|NCF_COLUMBIA|NCF_HYRAX)
Cleanup DAP4 testing NOTE: This PR should not be included in 4.9.1 since additional DAP4 related PRs will be forthcoming. This PR makes major changes to libdap4 and dap4_test driven by changes to TDS. * Enable DAP4 * Clean up the test input files and the test baseline comparison files. This entails: * Remove a multitude of unused test input and baseline data files; among them are dap4_test/: daptestfiles, dmrtestfiles, nctestfiles, and misctestfiles. * Define a canonical set of test input files and record in dap4_test/cdltestfiles. * Use the cdltestfiles to generate the .nc test inputs. This set of .nc files is then moved to the d4ts (DAP4 test server) war file in the tds repository. This set then becomes the canonical set of DAP4 test sources. * Scrape d4ts to obtain copies of the raw streams of DAP4 encoded data. The .dmr and .dap streams are then stored in dap4_test/rawtestfiles. * Disable some remote server tests until those servers are fixed. * Add an option to ncdump (-XF) that forces the type of the _FillValue attribute; this is primarily to simplify testing of fill mismatch. * Minor bug fixes to ncgen. * Changes to libdap4: * Replace old checksum hack with the dap4.checksum flag. * Support the dap4.XXX controls. * Cleanup _FillValue handling, especially var-attribute type mismatches. * Fix enum handling based on changes to netcdf-java. * Changes to dap4_test: * Add getopt support to various test support programs. * Remove unneeded shell scripts. * Add new scripts: test_curlopt.sh
2022-11-14 04:15:11 +08:00
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
typedef struct NCCONTROLS {
NCFLAGS flags;
} NCCONTROLS;
/* Misc. Constants */
#ifndef DFALTPACKETSIZE
#define DFALTPACKETSIZE 0x20000 /*approximately 100k bytes*/
#endif
#ifndef DFALTUSERAGENT
#define DFALTUSERAGENT "netCDF"
#endif
#endif /*NCDAP_H*/