Commit Graph

704 Commits

Author SHA1 Message Date
Dennis Heimbigner
b488c272d5 Fix conflicts with master 2020-02-27 14:06:45 -07:00
Dennis Heimbigner
44d0dcaad2 Add support for multiple filters per variable.
re: https://github.com/Unidata/netcdf-c/issues/1584

Support has been added for multiple filters per variable.  This
affects a number of components in netcdf. The new APIs are
documented in NUG/filters.md.

The primary changes are:
* A set of new functions are provided (see __include/netcdf_filter.h__).
    - Obtain a list of the filters associated with a variable
    - Obtain the parameters for a specific filter.
* The existing __nc_inq_var_filter__ function now returns info
  about the first defined filter.
* The utilities (ncgen, ncdump, and nccopy) now support
  an extended format for specifying a sequence of filters.
  The general form is __<filter>|<filter>..._.
* The ncdump **_Filter** attribute now dumps a list of all the
  filters associated with a variable using the above new format.
* Filter specifications can now use a filter name instead of number
  for filters known to the netcdf library, which in turn is taken
  from the HDF5 filter registration page.
* New errors are defined: NC_EFILTER and NC_ENOFILTER. The latter
  is returned if an attempt is made to access an unknown filter.
* Internally, the dispatch table has been extended to add a function
  to handle all of the filter functions.
* New, filter-related, tests were added to nc_test4.
* A new plugin was added to the plugins directory to help with testing.

Notes:
1. The shuffle and fletcher32 filters are not part of the multifilter system.

Misc. changes:
1. A debug module was added to libhdf5 to help catch error locations.
2020-02-16 12:59:33 -07:00
Edward Hartnett
164de982bd merged changes from master branch 2020-02-11 04:05:35 -07:00
Edward Hartnett
da904f6438 raised NC_MAX_HDF4_NAME length to NC_MAX_NAME 2020-02-08 09:21:01 -07:00
Edward Hartnett
96182a9236 trying to fix windows build 2020-02-07 11:57:56 -07:00
Edward Hartnett
8057a552ef move nc_def_var_szip function so it will appear in the documentation 2020-02-07 09:09:01 -07:00
Edward Hartnett
1817790c6b rely completely on nc_def_var_filter for setting szip 2020-02-05 10:32:16 -07:00
Ward Fisher
2e7234ff5b
Merge pull request #1511 from mathstuf/fix-windows-define
nc4internal: detect Windows using the correct define name
2020-01-27 17:43:43 -07:00
Edward Hartnett
0a0282716d added to libnetcdf.settings and netcdf_meta.h 2020-01-16 09:19:24 -07:00
Edward Hartnett
e94615a0e5
Merge branch 'master' into ejh_szip 2020-01-16 08:49:12 -07:00
Ward Fisher
8771d0bdf4
Merge pull request #1582 from NOAA-GSD/ejh_parallel_zlib
Allow user to turn on zlib, shuffle, and/or fletcher32 filters with parallel I/O for HDF5-1.10.2+
2020-01-13 16:06:51 -07:00
Dennis Heimbigner
748d26c114 Add support for CURLOPT_CONNECTTIMEOUT
I see that there is no way to set CURLOPT_CONNECTTIMEOUT,
but there is support for CURLOPT_TIMEOUT.
So, accept the line 'HTTP.CONNECTTIMEOUT'
in .rc file to allow user to set CURLOPT_CONNECTTIMEOUT.
2020-01-09 11:48:04 -07:00
Edward Hartnett
9406f69d7e moved nc_def_var_szip prototype to netcdf_filter.h 2020-01-06 16:54:54 -07:00
Edward Hartnett
fe7132be91 fixed deflate_level comment 2020-01-06 16:31:49 -07:00
Edward Hartnett
184507be5f now using members in NC_VAR_INFO_T to hold szip info 2020-01-06 08:46:03 -07:00
Edward Hartnett
e703a7678c first stab at re-adding nc_def_var_szip() 2020-01-03 11:38:45 -07:00
Edward Hartnett
808a0e2be9 merged ejh_parallel_zlib 2020-01-02 14:25:31 -07:00
Dennis Heimbigner
5d254f80b0 Guard nctestserver.h against multiple inclusions 2019-12-31 16:04:56 -07:00
Dennis Heimbigner
7223c4a5aa Avoid spurious test failures when servers fail.
re: https://github.com/Unidata/netcdf-c/issues/1451

The situation with the various DAP (and other) remote test
servers is currently in a state of flux.  For example, Unidata
admin is planning to forcibly shift the remote test server to
remotetest.unidata.ucar.edu soon.  In addition, the server
test.opendap.org has shown some recent instability.

The result is that various DAP (and byterange) tests can fail
unexpectedly. This is an irritant to users and reveals nothing
about test sucess or failure.

Solve by modifying tests to report server inaccessibility and
otherwise pretend to succeed.

This puts an onus on Unidata to detect such server failures, but
will not cause users to see spurious failures. [Note. Do similar
fix for netcdf-java]. The check is:
1. export SETX=1 to cause all the shell scripts to trace
2. search the log files for the phrase "WARNING" (in upper case)
and see if it is complaining about not finding a server.

Misc. Changes
-------------
1. Added a pingurl program to see if a server was up.
2. modified some test case url targets
2019-12-31 15:42:58 -07:00
Edward Hartnett
995cfdad96 merged master 2019-12-20 11:16:11 -07:00
Edward Hartnett
b9b5a8ff43 fixed include issue 2019-12-20 07:43:39 -07:00
Edward Hartnett
82df2876b6 starting to support compact storage 2019-12-04 07:53:37 -07:00
Edward Hartnett
6a902fe210 further removal of RELAX_COORD_BOUNDS macro 2019-11-26 06:45:26 -07:00
Ward Fisher
788d20df59
Merge pull request #1531 from NetCDF-World-Domination-Council/ejh_update_netcdf_meta
Add a dispatch version number to netcdf_meta.h and libnetcdf.settings, in case we decide to change dispatch table in future
2019-11-15 16:38:54 -07:00
edwardhartnett
a0f984f997 added dispatch version to netcdf_meta.h and libnetcdf.settings 2019-11-15 10:06:46 -07:00
edwardhartnett
559e933123 fixed header file 2019-11-15 09:39:00 -07:00
Ben Boeckel
464e2953a0 windows: detect Windows using the correct define name 2019-11-07 07:55:47 -05:00
Ward Fisher
91dfcd0248 Fixed another instance of netcdf_par.h being included before netcdf.h 2019-11-04 14:13:51 -07:00
Ward Fisher
ea51c67baf Removed previous kludgy fix. 2019-11-04 14:09:25 -07:00
Ward Fisher
72d889d15a A better solution to previous fix, in line with documented usage of netcdf.h and other header files. 2019-11-04 14:07:50 -07:00
Ward Fisher
726030dabd Added EXTERNL definition in support of https://github.com/Unidata/netcdf-c/issues/1507 2019-11-04 13:38:14 -07:00
Dennis Heimbigner
9062af4697 cleanup netcdf_mem.h and netcdf_dispatch.h 2019-10-28 10:56:24 -06:00
Dennis Heimbigner
5ec10a97e9 CMake: install netcdf_filter.h and netcdf_dispatch.h 2019-10-28 10:52:05 -06:00
Dennis Heimbigner
40cd180957 Fix includes/netcdf_xxx.h files to not include ncexternl.h
re: https://github.com/Unidata/netcdf-c/issues/1502

If you plan to use any of the netcdf_XXX.h files, then it it is assumed
that you first include netcdf.h, followed by any netcdf_XXX.h files
you need. The EXTERNL (and other things) are defined in netcdf.h
so they should be available for use by the netcdf_XXX.h files.

This PR ensures that the above assumption is true.  It also adds
some comments to that effect to the start of the netcdf_xxx.h files.
2019-10-28 10:29:39 -06:00
Ward Fisher
d001ec8590 Removing a problematic const causing issues on OSX. 2019-10-09 17:18:48 -06:00
Dennis Heimbigner
f1506d552e Change (again), and hopefully simplify, the file model inference algorithm.
* For URL paths, the new approach essentially centralizes all information
  in the URL into the "#mode=" fragment key and uses that value
  to determine the dispatcher for (most) URLs.

* The new approach has the following steps:

  1. canonicalize the path if it is a URL.
  2. use the mode= fragment key to determine the dispatcher
  3. if dispatcher still not determined, then use the mode flags
     argument to nc_open/nc_create to determine the dispatcher.
  4. if the path points to something readable, attempt to read the
     magic number at the front, and use that to determine the dispatcher.
     this case may override all previous cases.

* Misc changes.

  1. Update documentation
  2. Moved some unit tests from libdispatch to unit_test directory.
  3. Fixed use of wrong #ifdef macro in test_filter_reg.c
     [I think this may fix an previously reported esupport query].
2019-09-29 12:59:28 -06:00
John Buonagurio
fbc4d1a53d Add missing DLL exports for parallel functions. 2019-09-21 21:55:11 -04:00
Ward Fisher
e7cc899264 Merge branch 'ejh_try2' of https://github.com/NetCDF-World-Domination-Council/netcdf-c into gh1487.wif 2019-09-20 14:04:56 -06:00
Greg Sjaardema
56c0d5cf8a Spelling fixes 2019-09-18 08:03:01 -06:00
edwardhartnett
2cd228bcd4 porting changes from other PR 2019-09-16 11:28:18 -06:00
edwardhartnett
5d1e3b4593 more doxygen documentation for nc4internal.h 2019-08-21 18:31:37 -06:00
edwardhartnett
5b3891a253 more unit tests, starting on doxygen docs for nc4internal.h 2019-08-21 16:04:59 -06:00
edwardhartnett
100e000d43 more unit tests for nc4internal.c 2019-08-21 13:34:43 -06:00
edwardhartnett
e4ef7b1a65 more unit tests, this time for nc4internal.c 2019-08-21 04:46:00 -06:00
edwardhartnett
fb371d3c3e added 2019 to netcdf.h copyright notice 2019-08-16 04:38:26 -06:00
Ed Hartnett
bce3fa6169
Merge branch 'master' into ejh_next 2019-08-16 03:42:32 -06:00
edwardhartnett
94f1a89a40 final removal 2019-08-15 07:05:10 -06:00
edwardhartnett
2077729abc removed base_pe functions from dispatch table 2019-08-15 06:51:06 -06:00
edwardhartnett
c7e022f7e5 removed rest of LOCKNUMREC and _CRAYMPP obsolete macros 2019-08-14 06:53:33 -06:00
edwardhartnett
60f436e7ee starting to remove obsolete _CRAYMPP macros 2019-08-14 06:13:45 -06:00