Commit Graph

5951 Commits

Author SHA1 Message Date
Ward Fisher
ea66842bea
Merge branch 'master' into nc_mpiio_nc_mpiposix 2018-10-09 15:21:50 -06:00
Ward Fisher
6b0217f38c
Merge pull request #1149 from Unidata/netcdf_mem_fix.dmh
Remove unneeded include of netcdf.h from netcdf_mem.h
2018-10-09 13:50:57 -06:00
Dennis Heimbigner
534364b069 This Pre removed #include <netcdf.h> from netcdf_mem.h.
In turn that caused some problems elsewhere because some files
did not include netcdf.h preceding netcdf_mem.h. Specifically,
building under visual studio fails.
So fix those cases that I can find. May need to
add more later.
2018-10-07 12:05:26 -06:00
Dennis Heimbigner
a5a34f6aba
Merge branch 'master' into nc_mpiio_nc_mpiposix 2018-10-06 13:33:55 -06:00
Dennis Heimbigner
77886deff2
Merge branch 'master' into netcdf_mem_fix.dmh 2018-10-06 13:32:57 -06:00
Ward Fisher
266d3c5997
Merge pull request #1157 from Unidata/filtermem.dmh
Fix memory problems when using HDF5 version 1.10.x and later.
2018-10-04 16:26:29 -06:00
Ward Fisher
cbcd6c616d Added new generated files to .gitignore file. 2018-10-04 15:02:02 -06:00
Ward Fisher
35ea9da7f6
Merge branch 'master' into filtermem.dmh 2018-10-04 15:01:29 -06:00
Ward Fisher
debb149a4f
Merge pull request #1141 from NetCDF-World-Domination-Council/ejh_cache_sep
Separate the HDF5 cache code
2018-10-04 12:05:39 -06:00
Dennis Heimbigner
d07c05b58f Fix memory problems when using HDF5 version 1.10.x and later.
re: issue https://github.com/Unidata/netcdf-c/issues/1156

Starting with HDF5 version 1.10.x, the plugin code MUST be
careful when using the standard *malloc()*, *realloc()*, and
*free()* function.

In the event that the code is allocating, reallocating, or
free'ing memory that either came from -- or will be exported to --
the calling HDF5 library, then one MUST use the corresponding
HDF5 functions *H5allocate_memory()*, *H5resize_memory()*,
*H5free_memory()* [5] to avoid memory failures.

Additionally, if your filter code leaks memory, then the HDF5 library
generates a failure something like this.
````
H5MM.c:232: H5MM_final_sanity_check: Assertion `0 == H5MM_curr_alloc_bytes_s' failed.
````

This PR modifies the code in the plugins directory to
conform to these new requirements.

This raises a question about the libhdf5 code where this
same problem may occur. We need to scan especially nc4hdf.c
to look for this problem.
2018-10-04 11:37:21 -06:00
Wei-keng Liao
65ca7d09b9 add NC_WRITE to LEGAL_CREATE_FLAGS for PnetCDF dispatcher 2018-10-04 11:53:39 -05:00
Ward Fisher
de3b0902fc
Merge branch 'master' into ejh_cache_sep 2018-10-03 14:14:06 -06:00
Ward Fisher
0381e1a4e6
Merge pull request #1155 from Unidata/fillmismatch.dmh
Modify DAP2 and DAP4 to optionally allow Fillvalue/Variable mismatch
2018-10-03 14:08:52 -06:00
Ward Fisher
9b02df9472 Updated travis.yml to enable filter testing as part of the travis CI tests. 2018-10-02 17:41:52 -06:00
Dennis Heimbigner
26da2fae01 Remove debug output 2018-10-02 11:43:11 -06:00
Wei-keng Liao
cfcf93eda3 add missing option --has_parallel4 2018-10-02 01:58:21 -05:00
Wei-keng Liao
82d7ae2a93 revise comments about collective I/O mode 2018-10-02 01:19:44 -05:00
Wei-keng Liao
08629267e4 add default setting in configure help strings 2018-10-02 01:18:51 -05:00
Dennis Heimbigner
dd654a272e Remove debug from tst_filter.sh 2018-10-01 15:54:25 -06:00
Dennis Heimbigner
8072d1f6bb Modify DAP2 and DAP4 to optionally allow Fillvalue/Variable mismatch
re: issue https://github.com/Unidata/netcdf-c/issues/1151

Modify DAP2 and DAP4 code to handle case when _FillValue type is not
same as the parent variable type.

Specifically:
1. Define a parameter [fillmismatch] to allow this mismatch;
   default is to disallow.
2. If allowed, forcibly change the type of the _FillValue to match
   the parent variable.
3. If allowed Convert the values to match new type
4. Generate a log message
5. if not allowed, then fail

Implementing this required some changes to ncdap_test/dapcvt.c
Also added test cases.

Minor Unrelated Changes:
1. There were a number of warnings about e.g.
   assigning a const char* to a char*. Fix these
2. In nccopy.1, replace .NP with .IP "n"
   (re PR https://github.com/Unidata/netcdf-c/pull/1144)
3. fix minor error in ncdump/ocprint
2018-10-01 15:51:43 -06:00
Ed
8cd4f89aa4 fixing curl.h problem a bit 2018-09-25 04:17:56 -06:00
Ed
faac874399 now ensure that --disable-netcdf was used when building hdf4, also check for jpeg library with HDF4 builds 2018-09-24 23:53:05 -06:00
Dennis Heimbigner
bba2a676a7
Remove unneeded include of netcdf.h from netcdf_mem.h 2018-09-23 14:16:45 -06:00
Wei-keng Liao
a7c08c3a49 add a missed update to NC_HDF4_open 2018-09-23 01:36:50 -05:00
Wei-keng Liao
0ed70756cc Ignore flags NC_MPIIO and NC_MPIPOSIX. 2018-09-22 20:22:34 -05:00
Wei-keng Liao
50e5b4fe8b remove unused files, test_get_p5.m4 is covered by test_get.m4 2018-09-22 18:21:55 -05:00
Wei-keng Liao
01da14da6f if cmode does not contain NC_NETCDF4, then it is for PnetCDF 2018-09-22 18:21:45 -05:00
Wei-keng Liao
bc63c08f41 add a check for MPI_Info_f2c 2018-09-21 17:32:36 -05:00
Wei-keng Liao
740a50d1f2 add more descriptions for parallel APIs 2018-09-20 12:59:19 -05:00
Wei-keng Liao
0853b3c9ee Merge branch 'pnetcdf_driver' of github.com:wkliao/netcdf-c into pnetcdf_driver 2018-09-20 11:50:27 -05:00
Wei-keng Liao
5ed6c239e1 More replacement of "parallel-netcdf" with "PnetCDF". 2018-09-20 11:45:25 -05:00
Ed Hartnett
cc62140f1d
Merge branch 'master' into ejh_cache_sep 2018-09-18 16:59:49 -06:00
Ward Fisher
5b5311ef80
Merge branch 'master' into pnetcdf_driver 2018-09-18 12:21:11 -06:00
Ward Fisher
18a05d0ed6
Merge pull request #1129 from NetCDF-World-Domination-Council/ejh_hdf5_sep_next_2
Fix benchmark programs
2018-09-18 12:20:59 -06:00
Wei-keng Liao
28512f77f0 remove unused variables 2018-09-18 01:27:57 -05:00
Wei-keng Liao
83608143ee more checks of NC_ENOTBUILT 2018-09-17 17:18:48 -05:00
Wei-keng Liao
5d297238d5 fix nc_test/tst_addvar.c. This program reads output file
tst_pnetcdf.nc generated from tst_pnetcdf.c.
2018-09-17 17:18:48 -05:00
Wei-keng Liao
48da78e133 Use PnetCDF instead of parallel-netcdf to avoid confusion with
parallel netcdf4. Also, update PnetCDF web page.
2018-09-17 17:18:48 -05:00
Wei-keng Liao
64e2982d0f adjust test logics for NC_ENOTBUILT 2018-09-17 17:18:48 -05:00
Wei-keng Liao
d24f292077 return NC_ENOTBUILT when CDF5 is not enabled but NC_64BIT_DATA is used 2018-09-17 17:18:48 -05:00
Wei-keng Liao
ec2b3250a3 Use cmode for nc_create() and omode for nc_open() 2018-09-17 17:18:48 -05:00
Wei-keng Liao
668e3021a8 Check if NC_ENOTBUILT is returend when PnetCDF feature is not
enabled but NC_PNETCDF is used in cmode or omode.
2018-09-17 17:18:48 -05:00
Wei-keng Liao
2e85316dbb Improve parallel create/open mode logic.
1. When model is detected as NC_FORMATX_NC3 and is called from
   nc_create_par, change the model to NC_FORMATX_PNETCDF.
2. When called from nc_create() or nc_open(), using NC_MPIIO or
   NC_MPIPOSIX is considered invalid.
3. Handle the case when NETCDF4 is not enabled but cmode/omode
   contains NC_NETCDF4.
4. Handle the case when PNETCDF is not enabled but cmode/omode
   contains NC_PNETCDF.
5. Correct comments about PnetCDF only handles CDF-5 files.
6. Add a check for MPI_ERR_NO_SUCH_FILE error class.

Make NC_check_file_type() static, as it is used in dfile.c only.
2018-09-17 17:18:48 -05:00
Wei-keng Liao
a537312895 nc_create_par() and nc_open_par() implies parallel I/O, so
using NC_MPIIO or NC_MPIPOSIX explicitly is not necessary.
These 2 subroutines call NC_create() and NC_open() with argument
useparallel set to 1, which should be used to tell which I/O
driver to use underneath.
2018-09-17 17:18:48 -05:00
Wei-keng Liao
329a44f911 fix varm API when stride == NULL and imap != NULL
fix bug when NC_check_id() failed, nc should not be used.
2018-09-17 17:18:48 -05:00
Ward Fisher
5259b4b1e8
Merge branch 'master' into ejh_hdf5_sep_next_2 2018-09-17 14:53:55 -06:00
Ward Fisher
58d382ce45
Merge pull request #1142 from Unidata/issue972.dmh
Fix szip filter handling code and corresponding tests
2018-09-17 14:53:37 -06:00
Dennis Heimbigner
108dc0f01d Fix szip filter handling code and correspondingtests
re: https://github.com/Unidata/netcdf-c/issues/972

The current szip plugin code in the HDF5 library has some
unexpected behaviors that require some changes to how
nc_inq_var_szip is implemented and to the corresponding tests:
nc_test4/{test_szip,tst_vars3}.

Specifically, the following can happen:

1. The number of parameters provided by the user will be two,
   but the number of parameters returned by nc_inq_var_filter
   will be four because the HDF5 code (H5Zszip) will add two
   extra parameters for internal use. It turns out that the two
   parameters provided when calling nc_def_var_filter correspond
   to the first two parameters of the four parameters returned
   by nc_inq_var_filter.

2. The nc_inq_var_szip values corresponding to the ones provided
   by the caller may be different than those provided by
   nc_def_var_filter.  The value of the options_mask argument is
   known to add additional flag bits, and the pixels_per_block
   parameter may be modified.
2018-09-15 15:21:51 -06:00
Ward Fisher
26b02ae362 Corrected a few things for old experimental bootstrap fortran option. Still not working tho. 2018-09-14 16:08:11 -06:00
Ward Fisher
a1dd3d2a2a Bumped experimental fortran autoinstall script to 4.4.4 2018-09-14 15:56:28 -06:00