Commit Graph

118 Commits

Author SHA1 Message Date
Dennis Heimbigner
d07c05b58f Fix memory problems when using HDF5 version 1.10.x and later.
re: issue https://github.com/Unidata/netcdf-c/issues/1156

Starting with HDF5 version 1.10.x, the plugin code MUST be
careful when using the standard *malloc()*, *realloc()*, and
*free()* function.

In the event that the code is allocating, reallocating, or
free'ing memory that either came from -- or will be exported to --
the calling HDF5 library, then one MUST use the corresponding
HDF5 functions *H5allocate_memory()*, *H5resize_memory()*,
*H5free_memory()* [5] to avoid memory failures.

Additionally, if your filter code leaks memory, then the HDF5 library
generates a failure something like this.
````
H5MM.c:232: H5MM_final_sanity_check: Assertion `0 == H5MM_curr_alloc_bytes_s' failed.
````

This PR modifies the code in the plugins directory to
conform to these new requirements.

This raises a question about the libhdf5 code where this
same problem may occur. We need to scan especially nc4hdf.c
to look for this problem.
2018-10-04 11:37:21 -06:00
Wei-keng Liao
0ed70756cc Ignore flags NC_MPIIO and NC_MPIPOSIX. 2018-09-22 20:22:34 -05:00
Wei-keng Liao
48da78e133 Use PnetCDF instead of parallel-netcdf to avoid confusion with
parallel netcdf4. Also, update PnetCDF web page.
2018-09-17 17:18:48 -05:00
Wei-keng Liao
739dfa2aa2 PnetCDF does not support per-variable collective/independent data mode change; modify nc_var_par_access to ignore varid 2018-07-15 16:19:21 -05:00
luz.paz
74fbacdb82 Misc. source comment typos
Some are user-facing. Found via `codespell` and through the downstream FreeCAD.
2018-04-26 23:04:01 -04:00
Thomas Braun
3af477a926 netcdf-c: Make cmake pass a windows cmd prompt
add_sh_test only creates the test if HAVE_BASH is TRUE.
Therefore we can only call SET_TESTS_PROPERTIES on the created test if
HAVE_BASH is equally true.
2018-04-26 14:43:27 +02:00
Dennis Heimbigner
d3b309722e re: gh issue https://github.com/Unidata/netcdf-c/issues/911
I took Ed's advice and moved the plugin stuff to its own
top-level directory. This is an attempt to solve the problem of
copying files that we have experienced. In any case, it will
serve as a place to stick additional plugins.
2018-04-21 20:10:47 -06:00
Ward Fisher
dad2101a04
Merge branch 'master' into index.dmh 2018-04-02 13:56:00 -06:00
Dennis Heimbigner
cdd6e00434 Try to fix file copy from nc_test4/hdf5plugins 2018-03-29 18:29:31 -06:00
Dennis Heimbigner
e671e78300 Forgot to add run_filter to EXTRA_DIST 2018-03-16 12:14:51 -06:00
Dennis Heimbigner
25f062528b This completes (for now) the refactoring of libsrc4.
The file docs/indexing.dox tries to provide design
information for the refactoring.

The primary change is to replace all walking of linked
lists with the use of the NCindex data structure.
Ncindex is a combination of a hash table (for name-based
lookup) and a vector (for walking the elements in the index).
Additionally, global vectors are added to NC_HDF5_FILE_INFO_T
to support direct mapping of an e.g. dimid to the NC_DIM_INFO_T
object. These global vectors exist for dimensions, types, and groups
because they have globally unique id numbers.

WARNING:
1. since libsrc4 and libsrchdf4 share code, there are also
   changes in libsrchdf4.
2. Any outstanding pull requests that change libsrc4 or libhdf4
   are likely to cause conflicts with this code.
3. The original reason for doing this was for performance improvements,
   but as noted elsewhere, this may not be significant because
   the meta-data read performance apparently is being dominated
   by the hdf5 library because we do bulk meta-data reading rather
   than lazy reading.
2018-03-16 11:46:18 -06:00
Dennis Heimbigner
7b324657d4
Merge branch 'master' into ncpyfilter.dmh 2018-03-02 19:49:43 -07:00
Dennis Heimbigner
dd4e71c150 1. Solve issue raised on netcdfgroup mailing list:
https://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2018/msg00020.html
See docs/filters.md for a detailed description of the solution.
Basically we add nccopy flags "-F none" and "-F <varname>:none"
to suppress output compression as desired.

2. Add tests to nc_test4/tst_filter.sh to test out the solution.
2018-03-02 16:55:58 -07:00
Ed Hartnett
45329862a8 removed valgrind stuff 2018-02-27 08:36:12 -07:00
Dennis Heimbigner
727b613459 This is the initial step in moving to the new higher performance
(I hope) metadata mechanism. This mostly just adds new pieces of
code (e.g. nclistmap) and does some minor fixes.

It should be transparent to everything else.
The next set of changes will be the big step.
2018-02-08 19:53:40 -07:00
Ed Hartnett
383a19161a fixed warnings 2018-02-01 07:36:34 -07:00
Ward Fisher
b7de06a982 Removed some confounding code in CmakeLists.txt, added a .gitignore for files that are copied dynamically. 2018-01-30 16:27:06 -06:00
Dennis Heimbigner
d388f0e7f3 - Rename target logical name bzip2 in nc_test4/hdf5plugins in order to avoid
cmake error CMP0002: duplicate name
- Assuming that all compilers treat floating point constants as double,
  remove d suffix from DBLVAL in nc_test4 and examples/C
2018-01-25 19:33:23 -07:00
Ward Fisher
1d1dc317a0 Another tweak 2018-01-24 18:07:21 -07:00
Ward Fisher
7f1f02c96d Additional tweaks for make dist. 2018-01-24 18:05:30 -07:00
Ward Fisher
6e3eca4fc0 Added hdf5plugins example code to EXTRA_DIST 2018-01-24 17:44:01 -07:00
Dennis Heimbigner
3e47f5f300 Rebuilt the filter parameter handling code to use a common
parser everywhere.
2018-01-23 16:00:11 -07:00
Dennis Heimbigner
22ebecab23 Use an explicit can_apply function for filters 2018-01-19 18:45:56 -07:00
Dennis Heimbigner
991c4c6d16 Add some extra nccopy filter debug output 2018-01-19 12:06:02 -07:00
Ward Fisher
5507d81523 Updated example file for OSX 2018-01-16 17:27:28 -07:00
Dennis Heimbigner
99fccab359 1. Keep up to date by merging master
2. Fixed plugin building (nc_test4/hdf5plugins)
   to be done properly by cmake and automake.
4. Duplicated part of the nc_test4 filter test code
   in examples/C

An incomplete and untested set of hooks exist
for OS-X in nc_test4/findplugins.in. They need testing.
2018-01-16 11:00:09 -07:00
Ward Fisher
69a418c27d Merge branch 'cmake-fix' of https://github.com/nehaljwani/netcdf-c into gh692 2018-01-02 13:24:13 -07:00
Ward Fisher
c42bbd0ea2 Removed old file, updated travis file to allow for tests to run in parallel. 2017-12-05 12:17:40 -07:00
Ward Fisher
cc2189293d Merging create_sample_files script into do_comps.sh, as cmake dependencies between scripts is proving a bit problematic. 2017-12-01 14:02:18 -07:00
Ward Fisher
4837b1382f Fixing for cmake parallel tests. 2017-12-01 14:01:07 -07:00
Nehal J Wani
03dd4e260f
Make certain tests conditional on dependencies
tst_h_rdc0 depends on ncdump_tst_netcdf4, which isn't build if HAVE_BASH
isn't true. But CMake errors out with:

-- Bash shell not found; disabling shell script tests.
CMake Error at CMakeLists.txt:1521 (SET_PROPERTY):
  SET_PROPERTY given TEST names that do not exist:

    tst_h_rdc0

There are also some tests which cdl_do_maps depends on, but it is
also not defined if HAVE_BASH isn't true. CMake errors out with:

CMake Error at examples/CDL/CMakeLists.txt:10 (SET_TESTS_PROPERTIES):
  SET_TESTS_PROPERTIES Can not find test to add properties to:
cdl_do_comps

This patch makes these tests conditional, making CMake happy.
2017-11-26 10:17:58 +05:30
Ward Fisher
394ee6a1e0 Corrected typo, more enhancements in CMakeLists.txt 2017-11-22 13:34:56 -07:00
Ward Fisher
1ccdf0219f Work towards parallel testing via cmake. 2017-11-22 12:10:58 -07:00
Ward Fisher
136a717d26 Getting tests to run in parallel under cmake 2017-11-22 11:37:00 -07:00
Ed Hartnett
812c2fd4d1 got make -j check working in ncdump and examples directories 2017-11-17 10:16:17 -07:00
Ward Fisher
5ef5ffc2bc Corrected an inconsistency between the documentation and the example. 2016-01-08 15:29:32 -07:00
Ward Fisher
612b35a84c Merge branch 'master' into cdf-5, in preparation for merging the CDF-5 functionality into the master branch. This will be the key new feature for netcdf 4.4.0. 2015-11-05 13:40:35 -07:00
tbeu
e2820e4d8a Fix common typos
Detected by https://github.com/vlajos/misspell_fixer
2015-08-20 11:42:05 +02:00
dmh
859f105005 merge-squash 2015-08-15 16:26:35 -06:00
Ward Fisher
7f812b367e Manual merge of pull request https://github.com/Unidata/netcdf-c/pull/64 contributed by nschloe. Assorted CMake improvements. 2014-06-11 15:51:31 -06:00
Ward Fisher
44fae42214 Cleaned up indentation, white space in multiple CMakeLists.txt files. 2014-04-21 11:15:33 -06:00
Ward Fisher
33d3d06971 Added initial 'make dist', 'make distcheck' support to
CMake-based builds.
2013-06-03 16:42:04 +00:00
Ward Fisher
ddf3c31bb0 Corrected a handful of syntax issues in CMake config files,
probably introduced more.  

Added CMake-related files to Makefile.am files for inclusion
when creating a distribution package.
2013-02-20 23:28:28 +00:00
Ward Fisher
7a226dd3f1 Merging the win_netcdf branch into the trunk. 2012-09-27 22:50:41 +00:00
Ward Fisher
31cfdd5910 Added 'examples' directory to CMake 2012-08-28 22:31:07 +00:00
Ward Fisher
d1b159bfa1 Modified Makefile.am scripts to run tests when working with
Windows DLLs.
2012-05-23 20:27:34 +00:00
Ed Hartnett
922ed6a303 removed fortran and cxx 2011-08-15 11:13:48 +00:00
Ed Hartnett
9212be6606 took out fortran API, made some documentation changes 2011-08-11 17:54:34 +00:00
Ed Hartnett
2b2bcded42 removed cxx and cxx4, bumped version to 4.2, and fixed up documentation some 2011-08-10 18:00:32 +00:00
Dennis Heimbigner
53e8d22d24 Fixed NCF-104 2011-07-27 20:48:58 +00:00
Ed Hartnett
b377906536 beginning of conversion to doxygen 2011-07-11 16:04:49 +00:00
Ed Hartnett
ec91981a4a fixed mistake in f90 example 2011-06-03 09:52:56 +00:00
Ed Hartnett
be2813aa5c fixed parallel example to use appropriate parallel flag when opening file 2011-06-03 09:52:13 +00:00
Ed Hartnett
e2685a1475 changes to remove EXTERN_LDFLAG from build, also to fix netCDF-4 sync bug 2011-05-17 19:14:35 +00:00
Ed Hartnett
8d99e0b43a more changes for mingw/DLL builds 2011-05-11 19:51:11 +00:00
Ed Hartnett
2f4cf5382e took out ldflags in Makefile.am files 2011-05-11 17:09:12 +00:00
Ed Hartnett
29e8619787 simplified build system 2011-04-26 20:57:24 +00:00
Ed Hartnett
d0d5e7c8c3 changes to make make check work on dll builds (by turning off shell tests for dll builds) 2011-03-21 20:31:56 +00:00
Russ Rew
7db57da22a Fix group bug in cxx4, remove extraneous include from cxx4 examples. 2010-11-30 17:39:21 +00:00
Ed Hartnett
1d25a39a8f changed permissions of shell scripts so they would actually work. 2010-10-19 17:52:48 +00:00
Ed Hartnett
a658960034 added more valgrind testing 2010-10-13 22:53:25 +00:00
Ed Hartnett
2ce4c3e368 removed unneeded files from repository 2010-06-29 14:22:30 +00:00
Ed Hartnett
cbb36d7710 more changes to makefiles 2010-06-22 19:48:55 +00:00
Ed Hartnett
bb57cf2e4b moved functions out of m4 into c files in libdispatch/netcdf.m4 2010-06-18 14:01:51 +00:00
Ed Hartnett
f039f511c8 more changes to get distcheck working 2010-06-07 16:13:40 +00:00
Ed Hartnett
bb0035c95d fixed Makefile.am includes to find new netcdf.h location 2010-06-07 15:40:31 +00:00
Ed Hartnett
92ccf1c5fa moved headers to include directory 2010-06-03 20:22:55 +00:00
Ed Hartnett
6e22b23b01 begin creating trunk subdir 2010-06-03 13:23:50 +00:00