Commit Graph

97 Commits

Author SHA1 Message Date
edwardhartnett
8083b3596e fixed problem of unlim dim and var sharing the same name but not being related 2019-11-15 09:18:42 -07:00
Constantine Khrulev
6abbf8d429 Whitespace changes 2019-11-13 10:07:10 -09:00
Constantine Khrulev
098f2c1056 Modify the condition used to check if an attribute can be re-used
This should make the code a bit cleaner.
2019-11-13 08:38:12 -09:00
Constantine Khrulev
dd181deca9 Improve the fix for #350 included in #1119
1) We have to use H5Tequal() to compare HDF5 type IDs.
2) When checking if we can re-use an NC_CHAR attribute it is enough to
   compare data types (H5Tequal() takes care of the size comparison).
3) This commit adds missing code (reuse_att was set but not used).

Now an attribute in a NetCDF-4 file can be modified as many times as
necessary, as long its type and length remain the same.

Modifications changing either type or length of an attribute require
deleting and re-creating an attribute which increments the attribute
order creation index. Once this index reaches 65535 all attribute
modifications (for a particular group or variable) will fail.

For reference:

Issue 350 title: NetCDF-4 limits the number of times an attribute can
be modified

Pull request 1119 title: Fix checking for HDF5 max dims, no longer
re-create atts if not needed, confirm behavior for HDF5 cyclical
files, allow user to set mpiexec
2019-11-12 21:45:47 -09:00
Greg Sjaardema
56c0d5cf8a Spelling fixes 2019-09-18 08:03:01 -06:00
Dennis Heimbigner
8d0bced60d Allow in-line definition of filters
Priority: Low

re: issue https://github.com/Unidata/netcdf-c/issues/1329

HDF5 has the ability to programmatically define new filters,
as opposed to using HDF5_PLUGIN_PATH env variable.
This PR adds support for that feature.
Not clear how useful this is, though.
See docs/filters.md for details.
2019-03-21 11:33:27 -06:00
Ed Hartnett
b1d30a0f67 cleanup of whitespace in HDF5 directory 2019-02-19 05:18:53 -07:00
Ed Hartnett
8e6f38b099 detecting conditions for mandatory rename of vars with varid > renamed var 2019-02-03 06:35:29 -07:00
Ed Hartnett
1dd76c996e added tst_rename3.c for more rename testing 2019-02-02 05:53:45 -07:00
Ed Hartnett
e74ec6f2a0 made function give_var_secret_name() not static, fixed warning 2019-01-27 11:10:41 -07:00
Ed Hartnett
627a55cf78 added function give_var_secret_name() 2019-01-27 11:06:02 -07:00
Ed Hartnett
42c64598dc created function create_dim_wo_var() 2019-01-27 10:51:25 -07:00
Ed Hartnett
60132a0ed7 made function static, removed unneeded if statement 2019-01-20 09:53:00 -07:00
Ed Hartnett
9d40e0a2af made function static, removed unneeded if statement 2019-01-20 09:52:42 -07:00
Ed Hartnett
029aa5f626 now using hidden coordinates att to speed file opens 2018-12-20 05:59:31 -07:00
Ed Hartnett
6e3d284fcc write coordinates hidden attribute for all variables 2018-12-19 09:16:21 -07:00
Ed Hartnett
8ca5a1ac17
Merge branch 'master' into ejh_fast_var_prep 2018-12-12 07:05:45 -07:00
Ward Fisher
30ea33435c Merge remote-tracking branch 'origin/license_update.wif' into pr-aggregation.wif 2018-12-11 17:08:21 -05:00
Ed Hartnett
28aa22f8cf cleanup and some comments for rec_write_metadata() 2018-12-11 09:33:46 -07:00
Ed Hartnett
8e1d781ea3 fixed error handling in write_dim(), also converted dimscale creation to H5Dcreate2() 2018-12-11 09:28:23 -07:00
Ed Hartnett
1a0cfb729e fixed error handling in write_var() 2018-12-11 08:10:08 -07:00
Ed Hartnett
26239a0897 cleaned up loop reattaching dimscales 2018-12-11 07:21:09 -07:00
Ed Hartnett
7b72d0b832 fixed error handling in attach_dimscales() 2018-12-11 07:15:46 -07:00
Ed Hartnett
39a0f822fa changed loop to use ncindexlookup() instead of looping through the names 2018-12-11 07:07:00 -07:00
Ed Hartnett
d7b657af4e cleaned up comments, changed loop to use ncindexlookup() instead of looping through the names 2018-12-11 06:43:27 -07:00
Ed Hartnett
53bdb74869 fixed error handling of write_netcdf4_dimid() 2018-12-11 06:38:23 -07:00
Ed Hartnett
3c9a141ee3 moved function detect_preserve_dimids and made it static 2018-12-11 06:15:47 -07:00
Ward Fisher
87110139f6 More synchronization of copyright stanzas. 2018-12-06 14:34:40 -07:00
Ed Hartnett
c824ad9ec9 further removal of libhdf5 code from libsrc4 2018-11-28 15:48:08 -07:00
Ed Hartnett
64b45b5e75 changes in support of separating HDF5 and libsrc4 2018-11-26 04:40:33 -07:00
Ed Hartnett
f6c3093fb3 moved convert function to libsrc4 2018-11-21 14:49:52 -07:00
Ed Hartnett
ab963e3d41 removing HDF5 type info from libsrc4 2018-11-20 14:24:40 -07:00
Ed Hartnett
52d58ecd2e changing over native_hdf_typeid 2018-11-20 10:49:26 -07:00
Ed Hartnett
5104262f6b changing types 2018-11-20 08:00:48 -07:00
Ed Hartnett
1d5307b600 merged in ejh_next_10 2018-11-19 09:25:04 -07:00
Ed Hartnett
3ec8b34bfb removed unneeded HDF5 fields 2018-11-19 09:23:43 -07:00
Ed Hartnett
262763c254 clean up 2018-11-16 10:01:31 -07:00
Ed Hartnett
8aa4577bef merged ejh_next_7 2018-11-15 16:40:14 -07:00
Dennis Heimbigner
751300ec59 Fix more memory leaks in netcdf-c library
This is a follow up to PR https://github.com/Unidata/netcdf-c/pull/1173

Sorry that it is so big, but leak suppression can be complex.

This PR fixes all remaining memory leaks -- as determined by
-fsanitize=address, and with the exceptions noted below.

Unfortunately. there remains a significant leak that I cannot
solve. It involves vlens, and it is unclear if the leak is
occurring in the netcdf-c library or the HDF5 library.

I have added a check_PROGRAM to the ncdump directory to show the
problem.  The program is called tst_vlen_demo.c To exercise it,
build the netcdf library with -fsanitize=address enabled. Then
go into ncdump and do a "make clean check".  This should build
tst_vlen_demo without actually executing it.  Then do the
command "./tst_vlen_demo" to see the output of the memory
checker.  Note the the lost malloc is deep in the HDF5 library
(in H5Tvlen.c).

I am temporarily working around this error in the following way.
1. I modified several test scripts to not execute known vlen tests
   that fail as described above.
2. Added an environment variable called NC_VLEN_NOTEST.
   If set, then those specific tests are suppressed.

This should mean that the --disable-utilities option to
./configure should not need to be set to get a memory leak clean
build.  This should allow for detection of any new leaks.

Note: I used an environment variable rather than a ./configure
option to control the vlen tests. This is because it is
temporary (I hope) and because it is a bit tricky for shell
scripts to access ./configure options.

Finally, as before, this only been tested with netcdf-4 and hdf5 support.
2018-11-15 10:00:38 -07:00
Ed Hartnett
d7fe095066 moving rest of var stuff 2018-11-13 16:59:07 -07:00
Ed Hartnett
4045588516 removing hid_t from NC_VAR_INFO_T 2018-11-13 16:10:34 -07:00
Ed Hartnett
9fee4db86b more var changes 2018-11-13 12:37:53 -07:00
Ed Hartnett
e1eeb309a1 more var changes to libhdf5 from libsrc4 2018-11-13 11:08:10 -07:00
Ed Hartnett
192a9b1b9e moving hdf5-specific var stuff 2018-11-13 10:18:29 -07:00
Ed Hartnett
49e0c1dc87 setting values in libhdf5 version of var hdf5-specific info 2018-11-13 09:00:15 -07:00
Ed Hartnett
87ddebac4b setting values for var hdf5 info 2018-11-13 07:31:59 -07:00
Ed Hartnett
825047b8f6 rest of moving HDF5 specific group info to libhdf5 2018-11-12 14:04:17 -07:00
Ed Hartnett
5c7fc1fb55 HDF5-specific group changes 2018-11-12 12:52:24 -07:00
Ed Hartnett
e74009c708 HDF5-specific group changes 2018-11-12 12:32:21 -07:00
Ed Hartnett
c59d6930f0 HDF5-specific group changes 2018-11-12 12:12:56 -07:00