Commit Graph

649 Commits

Author SHA1 Message Date
Dennis Heimbigner
a5a34f6aba
Merge branch 'master' into nc_mpiio_nc_mpiposix 2018-10-06 13:33:55 -06:00
Dennis Heimbigner
d07c05b58f Fix memory problems when using HDF5 version 1.10.x and later.
re: issue https://github.com/Unidata/netcdf-c/issues/1156

Starting with HDF5 version 1.10.x, the plugin code MUST be
careful when using the standard *malloc()*, *realloc()*, and
*free()* function.

In the event that the code is allocating, reallocating, or
free'ing memory that either came from -- or will be exported to --
the calling HDF5 library, then one MUST use the corresponding
HDF5 functions *H5allocate_memory()*, *H5resize_memory()*,
*H5free_memory()* [5] to avoid memory failures.

Additionally, if your filter code leaks memory, then the HDF5 library
generates a failure something like this.
````
H5MM.c:232: H5MM_final_sanity_check: Assertion `0 == H5MM_curr_alloc_bytes_s' failed.
````

This PR modifies the code in the plugins directory to
conform to these new requirements.

This raises a question about the libhdf5 code where this
same problem may occur. We need to scan especially nc4hdf.c
to look for this problem.
2018-10-04 11:37:21 -06:00
Dennis Heimbigner
dd654a272e Remove debug from tst_filter.sh 2018-10-01 15:54:25 -06:00
Dennis Heimbigner
8072d1f6bb Modify DAP2 and DAP4 to optionally allow Fillvalue/Variable mismatch
re: issue https://github.com/Unidata/netcdf-c/issues/1151

Modify DAP2 and DAP4 code to handle case when _FillValue type is not
same as the parent variable type.

Specifically:
1. Define a parameter [fillmismatch] to allow this mismatch;
   default is to disallow.
2. If allowed, forcibly change the type of the _FillValue to match
   the parent variable.
3. If allowed Convert the values to match new type
4. Generate a log message
5. if not allowed, then fail

Implementing this required some changes to ncdap_test/dapcvt.c
Also added test cases.

Minor Unrelated Changes:
1. There were a number of warnings about e.g.
   assigning a const char* to a char*. Fix these
2. In nccopy.1, replace .NP with .IP "n"
   (re PR https://github.com/Unidata/netcdf-c/pull/1144)
3. fix minor error in ncdump/ocprint
2018-10-01 15:51:43 -06:00
Wei-keng Liao
0ed70756cc Ignore flags NC_MPIIO and NC_MPIPOSIX. 2018-09-22 20:22:34 -05:00
Ward Fisher
5259b4b1e8
Merge branch 'master' into ejh_hdf5_sep_next_2 2018-09-17 14:53:55 -06:00
Dennis Heimbigner
108dc0f01d Fix szip filter handling code and correspondingtests
re: https://github.com/Unidata/netcdf-c/issues/972

The current szip plugin code in the HDF5 library has some
unexpected behaviors that require some changes to how
nc_inq_var_szip is implemented and to the corresponding tests:
nc_test4/{test_szip,tst_vars3}.

Specifically, the following can happen:

1. The number of parameters provided by the user will be two,
   but the number of parameters returned by nc_inq_var_filter
   will be four because the HDF5 code (H5Zszip) will add two
   extra parameters for internal use. It turns out that the two
   parameters provided when calling nc_def_var_filter correspond
   to the first two parameters of the four parameters returned
   by nc_inq_var_filter.

2. The nc_inq_var_szip values corresponding to the ones provided
   by the caller may be different than those provided by
   nc_def_var_filter.  The value of the options_mask argument is
   known to add additional flag bits, and the pixels_per_block
   parameter may be modified.
2018-09-15 15:21:51 -06:00
Ed Hartnett
86e002d794 merged master 2018-09-06 14:01:59 -06:00
Ed Hartnett
80dc5bc0f7 merged master 2018-09-06 12:24:29 -06:00
Ward Fisher
fbe0a18b1c
Merge branch 'master' into ejh_loop_cleanup_2 2018-09-05 11:22:55 -06:00
Ward Fisher
ecffb53f40
Merge branch 'master' into NC_ENOTINDEFINE 2018-09-04 11:23:19 -06:00
Ed Hartnett
9fd74d2c83 uncommented test code 2018-08-28 10:09:39 -06:00
Wei-keng Liao
116f303182 correct error code for operation not allowed in data mode is NC_ENOTINDEFINE 2018-08-24 21:26:25 -05:00
Ed Hartnett
8847c843fb further cleanup for benchmark builds 2018-08-24 12:48:42 -06:00
Ed Hartnett
e1bd6f2c20 fixing cmake benchmarks, also removing unneeded run_bm.sh 2018-08-24 09:04:01 -06:00
Ed Hartnett
1310066494 getting parallel benchmarks to work, cleanup of benchmark scripts 2018-08-24 08:59:58 -06:00
Ed Hartnett
77d3a6db22 removed tst_ar5 from Makefile.am and cmake build 2018-08-24 07:11:51 -06:00
Ed Hartnett
933d91a556 fixed many benchmark warnings 2018-08-23 08:21:55 -06:00
Ed Hartnett
c665a23e86 fixed warnings in tst_chunks3.c 2018-08-23 07:05:33 -06:00
Ed Hartnett
077a96471e got tst_chunks3 benchmark working 2018-08-23 07:02:44 -06:00
Ed Hartnett
c2b6d36779 comments 2018-08-23 06:34:10 -06:00
Ed Hartnett
a102f1c55d got knmi benchmark test working with parallel builds 2018-08-23 06:32:04 -06:00
Ed Hartnett
cb51061861 got tst_files2 2018-08-22 10:09:32 -06:00
Ed Hartnett
d8c1a6209a removed unneeded lookup function 2018-08-21 11:54:06 -06:00
Ed Hartnett
30f03bc155 removed unneeded test data file 2018-08-21 07:09:49 -06:00
Ed Hartnett
563052d4b8 removed unneeded enum test 2018-08-21 06:42:10 -06:00
Ed Hartnett
b57b87c971 uncommented test code 2018-08-21 06:36:40 -06:00
Ed Hartnett
c02f976d84 more tests 2018-08-20 17:05:29 -06:00
Ed Hartnett
fda87a8d5f now check for max of 32 dims for a var in HDF5 file 2018-08-20 09:53:08 -06:00
Ed Hartnett
f77350c66d testing error for HDF5 file with circular groups 2018-08-20 09:26:50 -06:00
Ed Hartnett
72805a5eed tracking enum issue 2018-08-20 05:45:30 -06:00
Ed Hartnett
5b2f6ecebc changed literal in netcdf.h 2018-08-18 05:22:07 -06:00
Ed Hartnett
9cf272f24e merged ejh_vars_null_count_issue_2 2018-08-18 04:21:21 -06:00
Ed Hartnett
99c758c649 cleaned up configure.ac, fixed parallel test warning 2018-08-16 11:43:01 -06:00
Ed Hartnett
1b318a01fb getting automake build working 2018-08-16 10:55:11 -06:00
Ed Hartnett
59a25093d9 added setting of mpiexec command to CMake build 2018-08-16 07:33:04 -06:00
Ed Hartnett
afb7e5ab43 set mpiexec from configure 2018-08-16 07:23:41 -06:00
Ed Hartnett
b3842197e0 merged master 2018-08-16 05:49:41 -06:00
Ed Hartnett
d59446ccd5
Merge branch 'master' into ejh_func_cleanup3 2018-08-15 18:08:34 -06:00
Ed Hartnett
a0bea92919 fixed problem of not calling NC_ versions of vars and varm functions for extra error checking 2018-08-14 05:04:09 -06:00
Ed Hartnett
5c39fa115a return error for NULL start for varm functions 2018-08-14 04:35:38 -06:00
Ed Hartnett
a271ebb5f2 testing NULL starts 2018-08-14 04:17:34 -06:00
Ed Hartnett
97c2541a37 put_vars checking for NULL start for non-scalar vars and test 2018-08-13 13:05:32 -06:00
Ward Fisher
a996ed554e Swapped /bin/bash for /bin/sh to test on osx. 2018-08-12 23:01:08 -06:00
Ed Hartnett
ad25807bd1
Merge branch 'master' into ejh_global_att_read 2018-08-06 16:30:26 -06:00
Ward Fisher
5f89204bdb
Merge branch 'master' into ejh_func_cleanup3 2018-08-06 11:32:45 -06:00
Ed Hartnett
56a57a57f0 bring in changes from lazy vars branch 2018-08-06 10:57:19 -06:00
Dennis Heimbigner
4ec592648a Force chunking when testing filter 2018-08-04 18:57:02 -06:00
Ed Hartnett
b9afdee6cd merged master 2018-07-31 19:44:28 -06:00
Ed Hartnett
7ab6d7bdda
Merge branch 'master' into ejh_global_att_read 2018-07-31 19:32:36 -06:00