Commit Graph

29 Commits

Author SHA1 Message Date
Ed Hartnett
cc18944fa7 moved lazy atts handling for nc_inq_attid() 2018-11-26 09:58:31 -07:00
Ed Hartnett
d97824d3e6 moved lazy atts handling for nc_inq_att() 2018-11-26 09:51:28 -07:00
Ed Hartnett
1f64c66cdf rename HDF5 dispatch functions to start with NC4_HDF5 2018-11-26 08:13:57 -07:00
Ed Hartnett
8aa4577bef merged ejh_next_7 2018-11-15 16:40:14 -07:00
Dennis Heimbigner
751300ec59 Fix more memory leaks in netcdf-c library
This is a follow up to PR https://github.com/Unidata/netcdf-c/pull/1173

Sorry that it is so big, but leak suppression can be complex.

This PR fixes all remaining memory leaks -- as determined by
-fsanitize=address, and with the exceptions noted below.

Unfortunately. there remains a significant leak that I cannot
solve. It involves vlens, and it is unclear if the leak is
occurring in the netcdf-c library or the HDF5 library.

I have added a check_PROGRAM to the ncdump directory to show the
problem.  The program is called tst_vlen_demo.c To exercise it,
build the netcdf library with -fsanitize=address enabled. Then
go into ncdump and do a "make clean check".  This should build
tst_vlen_demo without actually executing it.  Then do the
command "./tst_vlen_demo" to see the output of the memory
checker.  Note the the lost malloc is deep in the HDF5 library
(in H5Tvlen.c).

I am temporarily working around this error in the following way.
1. I modified several test scripts to not execute known vlen tests
   that fail as described above.
2. Added an environment variable called NC_VLEN_NOTEST.
   If set, then those specific tests are suppressed.

This should mean that the --disable-utilities option to
./configure should not need to be set to get a memory leak clean
build.  This should allow for detection of any new leaks.

Note: I used an environment variable rather than a ./configure
option to control the vlen tests. This is because it is
temporary (I hope) and because it is a bit tricky for shell
scripts to access ./configure options.

Finally, as before, this only been tested with netcdf-4 and hdf5 support.
2018-11-15 10:00:38 -07:00
Ed Hartnett
67317bf5a0 more var work 2018-11-13 11:30:23 -07:00
Ed Hartnett
902ee64ff0 hdf5-specific group stuff for hdf5attr.c 2018-11-12 13:38:11 -07:00
Ed Hartnett
11d725facc allocating and freeing memory for hdf5-specific attribute info 2018-11-07 13:45:51 -07:00
Ward Fisher
784d777bff Merge branch 'master' into provenance.dmh 2018-09-06 15:13:09 -06:00
Dennis Heimbigner
2ea1cf5f1b There was a request to extend the provenance information
stored in the _NCProperties attribute to allow two things:
1. capture of additional library dependencies (over and above
   hdf5)
2. Recognition of non-netcdf libraries that create netcdf-4 format
   files.

To this end, the _NCProperties format has been extended to be
and arbitrary set of key=value pairs separated by commas.
This new format has version = 2, and uses commas as the pair separator.
Thus the general form is:
    _NCProperties = "version=2,key1=value,key2=value2..." ;

This new version is accompanied by a new ./configure option of the form
    --with-ncproperties="key1=value1,key2=value2..."
that specifies pairs to add to the _NCProperties attribute for all
files created with that netcdf library.

At this point, what is missing is some programmatic way to
specify either all the pairs or additional pairs
to the _NCProperties attribute. Not sure of the best way
to do this.

Builders using non-netcdf libraries can specify
whatever they want in the key value pairs (as long
as the version=2 is specified first).

By convention, the primary library is expected to be the
the first pair after the leading version=2 pair, but this
is convention only and is neither required nor enforced.

Related changes:
1. Fixed the tests that check _NCProperties to properly operate with version=2.
2. When reading a version 1 _NCProperties attribute, convert it to look
   like a version 2 attribute.
2. Added some version 2 tests to ncdump/tst_fileinfo.c and
   ncdump/tst_fileinfo.sh

Misc Changes:
1. Fix minor problem in ncdap_test/testurl.sh where a parameter to
   buildurl needed to be quoted.
2. Minor fix to ncgen to swap switches -H and -h to be consistent
   with other utilities.
3. Document the -M flag in nccopy usage() and the nccopy man page.
4. Modify a test case to use the nccopy -M flag.
2018-08-25 21:44:41 -06:00
Wei-keng Liao
116f303182 correct error code for operation not allowed in data mode is NC_ENOTINDEFINE 2018-08-24 21:26:25 -05:00
Ed Hartnett
5a452c5815 bring in changes from lazy vars branch 2018-08-06 10:49:31 -06:00
Ed Hartnett
5631fa512a merged master 2018-07-24 01:51:51 -06:00
Ed Hartnett
5a52f28bb7 further condensing code 2018-07-21 10:43:36 -06:00
Ed Hartnett
afa7c0a87d further condensing code 2018-07-21 09:26:16 -06:00
Ed Hartnett
43f094ca50 changing the way global atts are read 2018-07-21 06:46:29 -06:00
Ed Hartnett
2dd0ac6319 clean up 2018-07-16 04:50:15 -06:00
Ed Hartnett
786c5a8f2e moved hdf5 specific header stuff to hdf5internal.h 2018-07-12 07:05:21 -06:00
Ed Hartnett
697f033823 renamed NC_HDF5_FILE_INFO to NC_FILE_INFO 2018-06-22 07:08:09 -06:00
Ed Hartnett
d4967405b4 cleaned up 2018-06-21 19:49:30 -06:00
Ed Hartnett
eafe151f13 added test 2018-06-19 14:59:07 -06:00
Ed Hartnett
77936f2884 more progress towards lazy read of atts for vars 2018-06-19 08:25:19 -06:00
Ed Hartnett
81add527f2 lazy read of global atts now working 2018-06-19 07:39:42 -06:00
Ed Hartnett
19ae8b47d1 took out src_long and dest_long again. Getting good at it! ;-) 2018-06-16 05:33:04 -06:00
Ed Hartnett
037a3cb58c reverting 2018-06-09 06:17:52 -06:00
Ed Hartnett
09366bf43b removed longs from conver_type again 2018-06-09 06:14:14 -06:00
Ward Fisher
1d789d9d39 Additional reconciliation 2018-06-08 15:50:39 -06:00
Ed Hartnett
9a2782b56c got long working with master 2018-06-05 14:40:49 -06:00
Ed Hartnett
8e7a617c98 added files 2018-05-15 06:51:44 -06:00