Commit Graph

231 Commits

Author SHA1 Message Date
Ed Hartnett
3ec8b34bfb removed unneeded HDF5 fields 2018-11-19 09:23:43 -07:00
Ward Fisher
53dc9022be Merge remote-tracking branch 'origin/ncgenmem.dmh' into tmptmp 2018-11-15 10:20:08 -07:00
Dennis Heimbigner
751300ec59 Fix more memory leaks in netcdf-c library
This is a follow up to PR https://github.com/Unidata/netcdf-c/pull/1173

Sorry that it is so big, but leak suppression can be complex.

This PR fixes all remaining memory leaks -- as determined by
-fsanitize=address, and with the exceptions noted below.

Unfortunately. there remains a significant leak that I cannot
solve. It involves vlens, and it is unclear if the leak is
occurring in the netcdf-c library or the HDF5 library.

I have added a check_PROGRAM to the ncdump directory to show the
problem.  The program is called tst_vlen_demo.c To exercise it,
build the netcdf library with -fsanitize=address enabled. Then
go into ncdump and do a "make clean check".  This should build
tst_vlen_demo without actually executing it.  Then do the
command "./tst_vlen_demo" to see the output of the memory
checker.  Note the the lost malloc is deep in the HDF5 library
(in H5Tvlen.c).

I am temporarily working around this error in the following way.
1. I modified several test scripts to not execute known vlen tests
   that fail as described above.
2. Added an environment variable called NC_VLEN_NOTEST.
   If set, then those specific tests are suppressed.

This should mean that the --disable-utilities option to
./configure should not need to be set to get a memory leak clean
build.  This should allow for detection of any new leaks.

Note: I used an environment variable rather than a ./configure
option to control the vlen tests. This is because it is
temporary (I hope) and because it is a bit tricky for shell
scripts to access ./configure options.

Finally, as before, this only been tested with netcdf-4 and hdf5 support.
2018-11-15 10:00:38 -07:00
Greg Sjaardema
cd011cd1bb Eliminate compiler warnings (and code error) 2018-11-14 11:00:47 -07:00
Ed Hartnett
825047b8f6 rest of moving HDF5 specific group info to libhdf5 2018-11-12 14:04:17 -07:00
Ed Hartnett
79c2f843d8 rest of hdf5 specific group changes 2018-11-12 13:57:04 -07:00
Ed Hartnett
971b16cdc8 hdf5 specific changes for hdf5open.c 2018-11-12 13:51:08 -07:00
Ed Hartnett
766f1d0195 hdf5-specific changes for hdf5grp.c 2018-11-12 13:43:43 -07:00
Ed Hartnett
902ee64ff0 hdf5-specific group stuff for hdf5attr.c 2018-11-12 13:38:11 -07:00
Ed Hartnett
261c7fb8d6 hdf5-specific group info for nc4info.c 2018-11-12 13:06:15 -07:00
Ed Hartnett
9c3a8cb2c5 hdf5-specific group stuff in hdf5var.c 2018-11-12 13:01:54 -07:00
Ed Hartnett
5c7fc1fb55 HDF5-specific group changes 2018-11-12 12:52:24 -07:00
Ed Hartnett
e74009c708 HDF5-specific group changes 2018-11-12 12:32:21 -07:00
Ed Hartnett
c59d6930f0 HDF5-specific group changes 2018-11-12 12:12:56 -07:00
Ed Hartnett
65bde4878b HDF5-specific group changes 2018-11-12 11:03:00 -07:00
Ed Hartnett
d6c5e1f127 HDF5-specific group changes 2018-11-12 10:48:24 -07:00
Ed Hartnett
17552b7eb3 HDF5-specific group changes 2018-11-12 10:37:06 -07:00
Ed Hartnett
6b75bb483b allocating HDF5-specific group info struct for root group 2018-11-12 09:02:46 -07:00
Ed Hartnett
9364157a9b allocating and freeing memory for HDF5-specific group info 2018-11-12 08:11:06 -07:00
Ed Hartnett
7534cb24fe cleanup 2018-11-08 11:12:38 -07:00
Ed Hartnett
e88b98daaa moving hdf5_objid field to hdf5-specific dim info 2018-11-08 11:05:20 -07:00
Ed Hartnett
856e4ead03 moved hdf_dimscaleid to hdf5-specific dim info 2018-11-08 10:55:21 -07:00
Ed Hartnett
cc02ec3aa3 more use of HDF5-specific dim info 2018-11-08 10:48:02 -07:00
Ed Hartnett
1dd9004a11 more use of HDF5-specific dim info 2018-11-08 10:43:36 -07:00
Ed Hartnett
7ab71a69bb more use of HDF5-specific dim info 2018-11-08 10:40:39 -07:00
Ed Hartnett
30da28edff using hdf5-specific dim info 2018-11-08 10:37:32 -07:00
Ed Hartnett
0ef97a84f1 cleanup 2018-11-08 10:27:34 -07:00
Ed Hartnett
bff06f2c4d starting to use HDF5-specific dim info 2018-11-08 10:22:37 -07:00
Ed Hartnett
dab52a97ee more looking up of HDF5 dim info 2018-11-08 10:16:55 -07:00
Ed Hartnett
92dc85b802 continuing to set hdf5-specific dim info struct 2018-11-08 10:11:46 -07:00
Ed Hartnett
b85eec1bb8 starting to set hdf5-specific dim info struct 2018-11-08 10:07:59 -07:00
Ed Hartnett
e89ad03b24 more looking up HDF5-specific dim info 2018-11-08 10:02:17 -07:00
Ed Hartnett
3e80723525 continuing to find HDF5 specific dim info 2018-11-08 09:20:01 -07:00
Ed Hartnett
008d4ee796 starting to find HDF5 specific dim info 2018-11-08 08:57:44 -07:00
Ed Hartnett
9373989931 allocating and freeing space for HDF5-specific dim info 2018-11-08 08:51:46 -07:00
Ed Hartnett
8d31f5b806 clean up 2018-11-07 14:23:55 -07:00
Ed Hartnett
6f4b4ac80d moving attribute HDF5 stuff to libhdf5 2018-11-07 14:21:57 -07:00
Ed Hartnett
11d725facc allocating and freeing memory for hdf5-specific attribute info 2018-11-07 13:45:51 -07:00
Ed Hartnett
5f36a3b425 merged master 2018-11-02 10:00:53 -06:00
Dennis Heimbigner
245961de00 re: github issues
https://github.com/Unidata/netcdf-c/issues/1168
    https://github.com/Unidata/netcdf-c/issues/1163
    https://github.com/Unidata/netcdf-c/issues/1162

This PR partially fixes memory leaks in the netcdf-c library,
in the ncdump utility, and in some test cases.

The netcdf-c library now runs memory clean with the assumption
that the --disable-utilities option is used. The primary remaining
problem is ncgen. Once that is fixed, I believe the netcdf-c library
will run memory clean with no limitations.

Notes
-----------
1. Memory checking was performed using gcc -fsanitize=address.
   Valgrind-based testing has yet to be performed.
2. The pnetcdf, hdf4, and examples code has not been tested.

Misc. Non-leak changes
1. Make tst_diskless2 only run when netcdf4 is enabled (issue 1162)
2. Fix CmakeLists.txt to turn off logging if ENABLE_NETCDF_4 is OFF
3. Isolated all my debug scripts into a single top-level directory
   called debug
4. Fix some USE_NETCDF4 dependencies in nc_test and nc_test4 Makefile.am
2018-10-30 20:48:12 -06:00
Ed Hartnett
c2cb3379f4 doing HDF5 closing in libhdf5 2018-10-23 06:59:22 -06:00
Ed Hartnett
35cfaefc0c closing HDF5 objects separately 2018-10-23 05:39:00 -06:00
Ed Hartnett
452f75fadd commented out some tests 2018-10-22 15:04:44 -06:00
Ed Hartnett
958826d0af merged ejh_mem_check 2018-10-22 13:29:46 -06:00
Dennis Heimbigner
e40eb2e950 switch 2018-10-19 11:11:36 -06:00
Ed Hartnett
5cbde66781 now closing att HDF5 typeid in HDF5 close code 2018-10-18 03:43:53 -06:00
Ed Hartnett
695e295734 now calling recursive function to close HDF5 objects in file 2018-10-18 03:29:21 -06:00
Ed Hartnett
fa86d3c488 continuing to separate hdf5/libsrc4 file close code 2018-10-18 03:20:39 -06:00
Ed Hartnett
c90ab24b48 moving towards separating HDF5 file close from netcdf4 file close 2018-10-18 03:17:38 -06:00
Dennis Heimbigner
979873f81d Fix provenance memory leak 2018-10-17 11:43:14 -06:00