Commit Graph

62 Commits

Author SHA1 Message Date
Greg Sjaardema
56c0d5cf8a Spelling fixes 2019-09-18 08:03:01 -06:00
Even Rouault
77ffbce43b
NC4_get_vars(): fix out-of-bounds write with unlimited dimension
This fixes an issue hit by GDAL, and that is found in netcdf 4.6.3
and 4.7.0

git bisect pointed the problem to have started with

```
77ab979c5f is the first bad commit
commit 77ab979c5f
Author: Ed Hartnett <edwardjameshartnett@gmail.com>
Date:   Sat Jun 16 09:58:48 2018 -0600

    using get_vars but not put_vars

:040000 040000 8611e77aae fc9ffd1d13 M	libsrc4
```

where nc_get_vara_double() started using nc4_get_vars() underneath.

It turns out that nc4_get_vars() was buggy in the situation exercised by GDAL.

This can be reproduced with the following simple test case:

```

int main()
{
    int status;
    int cdfid = -1;
    int first_dim;
    int varid;
    int other_var;
    size_t anStart[NC_MAX_DIMS];
    size_t anCount[NC_MAX_DIMS];
    double* val = (double*)calloc(3, sizeof(double));

    status = nc_create("foo.nc", NC_NETCDF4, &cdfid);
    assert( status == NC_NOERR );

    status = nc_def_dim(cdfid, "unlimited_dim", NC_UNLIMITED, &first_dim);
    assert( status == NC_NOERR );

    status = nc_def_var(cdfid, "my_var", NC_DOUBLE, 1, &first_dim, &varid);
    assert( status == NC_NOERR );

    status = nc_def_var(cdfid, "other_var", NC_DOUBLE, 1, &first_dim, &other_var);
    assert( status == NC_NOERR );

    status = nc_enddef(cdfid);
    assert( status == NC_NOERR );

    /* Write 3 elements to set the size of the unlimited dim to 3 */
    anStart[0] = 0;
    anCount[0] = 3;
    status = nc_put_vara_double(cdfid, other_var, anStart, anCount, val);
    assert( status == NC_NOERR );

    /* Read 2 elements starting with index=1 */
    anStart[0] = 1;
    anCount[0] = 2;
    status = nc_get_vara_double(cdfid, varid, anStart, anCount, val);
    assert( status == NC_NOERR );

    status = nc_close(cdfid);
    assert( status == NC_NOERR );

    free(val);

    return 0;
}
```

Running it under Valgrind without this patch leads to
```
==19637==
==19637== Invalid write of size 8
==19637==    at 0x4C326CB: memcpy@@GLIBC_2.14 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19637==    by 0x4EDBE3D: NC4_get_vars (hdf5var.c:2131)
==19637==    by 0x4EDA24C: NC4_get_vara (hdf5var.c:1342)
==19637==    by 0x4E68878: NC_get_vara (dvarget.c:104)
==19637==    by 0x4E69FDB: nc_get_vara_double (dvarget.c:815)
==19637==    by 0x400C08: main (in /home/even/netcdf-c/build/test)
==19637==  Address 0xb70e3e8 is 8 bytes before a block of size 24 alloc'd
==19637==    at 0x4C2FB55: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19637==    by 0x4009E8: main (in /home/even/netcdf-c/build/test)
==19637==
```
2019-07-18 01:25:21 +02:00
Ward Fisher
d6a3944199
Merge pull request #1409 from Unidata/nccopydefault.dmh
Nccopy was overriding default chunking when it should not.
2019-05-29 15:26:09 -06:00
Dennis Heimbigner
112b2cc5e2 Convert to use LOGGING 2019-05-25 12:35:52 -06:00
Dennis Heimbigner
7901353cf5 Restore nc_perf/CMakeLists.txt 2019-05-25 12:15:56 -06:00
Dennis Heimbigner
06498ff16a various fixes 2019-05-23 16:35:03 -06:00
Ed Hartnett
150662dd0b changes to support build of libsrc4 without libhdf5 2019-05-22 07:50:12 -06:00
Dennis Heimbigner
6ebc108f00 Nccopy was overriding default chunking when it should not.
re: issue https://github.com/Unidata/netcdf-c/issues/1398
re: esupport NDY-294972

The new chunking code added to nccopy missed one case.
In the event that there are no chunking specifications
of any kind, and the input is not netcdf-4, and the output
is netcdf-4 and must be chunked, then use the default chunking
that the library computes as part of the nc_def_var() function.

Misc. changes:
1. add some chunking debug code to hdf5var.c
2019-05-21 15:59:27 -06:00
Ed Hartnett
b1d30a0f67 cleanup of whitespace in HDF5 directory 2019-02-19 05:18:53 -07:00
Ed Hartnett
70201adb51 comment cleanup 2019-02-03 07:43:56 -07:00
Ed Hartnett
5d908a0bbb now preserve order of varids after a var rename 2019-02-03 06:56:03 -07:00
Ed Hartnett
8e6f38b099 detecting conditions for mandatory rename of vars with varid > renamed var 2019-02-03 06:35:29 -07:00
Ed Hartnett
e30a2bf208 added come comments 2019-02-02 07:32:31 -07:00
Ed Hartnett
f25f050be8 converted hdf5var to use H5Lmove instead of deprecated H5Gmove 2019-02-02 07:20:14 -07:00
Ed Hartnett
828304ed41 now using secret hdf5 var name during renames if needed 2019-01-27 11:33:06 -07:00
Ed Hartnett
1b3f397c4c added name parameter to give_var_secret_name to base secret name on 2019-01-27 11:29:49 -07:00
Ed Hartnett
784dc0e0ad now creating secret hdf5 name on var rename, if needed 2019-01-27 11:17:57 -07:00
Ed Hartnett
660bda1be3 made function give_var_secret_name() static again 2019-01-27 11:14:29 -07:00
Ed Hartnett
e74ec6f2a0 made function give_var_secret_name() not static, fixed warning 2019-01-27 11:10:41 -07:00
Ed Hartnett
c95887cc53 removed name param from function give_var_secret_name() 2019-01-27 11:07:57 -07:00
Ed Hartnett
627a55cf78 added function give_var_secret_name() 2019-01-27 11:06:02 -07:00
Ed Hartnett
243cef8fa5 changed var atts_not_read to atts_read 2019-01-21 08:40:04 -07:00
Ed Hartnett
281f67da6e removed unneeded vars, fixed and added comments 2019-01-20 09:46:15 -07:00
Ed Hartnett
15e6a782db removed unneeded variable, shortened function name 2019-01-20 09:25:04 -07:00
Ed Hartnett
1b38d9aef8 lazy read of some var metadata 2018-12-18 07:48:22 -07:00
Ed Hartnett
d0587c9536 more name normalization 2018-11-30 09:14:53 -07:00
Ed Hartnett
77d0922d49 starting to deal with normalized name in HDF5 attribute code 2018-11-30 08:59:58 -07:00
Ed Hartnett
104b4b50fe clean up 2018-11-29 06:25:34 -07:00
Ed Hartnett
d51b221c62 move setting of var chunk cache to libhdf5 2018-11-29 06:10:39 -07:00
Ed Hartnett
016f675fad more separation of lazy att reads from libsrc4 2018-11-26 10:44:48 -07:00
Ed Hartnett
1f64c66cdf rename HDF5 dispatch functions to start with NC4_HDF5 2018-11-26 08:13:57 -07:00
Ed Hartnett
ab963e3d41 removing HDF5 type info from libsrc4 2018-11-20 14:24:40 -07:00
Ed Hartnett
52d58ecd2e changing over native_hdf_typeid 2018-11-20 10:49:26 -07:00
Ed Hartnett
d29feb53de have switched location of hdf_native_typeid 2018-11-20 10:29:02 -07:00
Ed Hartnett
c072c89357 more type work 2018-11-20 09:57:47 -07:00
Ed Hartnett
e11e9b7bfd more type work 2018-11-20 09:06:40 -07:00
Ed Hartnett
a431c61573 more type work 2018-11-20 08:21:02 -07:00
Ed Hartnett
4c1b3a225b clean up 2018-11-16 10:07:54 -07:00
Ed Hartnett
262763c254 clean up 2018-11-16 10:01:31 -07:00
Ed Hartnett
4045588516 removing hid_t from NC_VAR_INFO_T 2018-11-13 16:10:34 -07:00
Ed Hartnett
1d951f6d04 more var changes to libhdf5 from libsrc4 2018-11-13 11:14:36 -07:00
Ed Hartnett
49e0c1dc87 setting values in libhdf5 version of var hdf5-specific info 2018-11-13 09:00:15 -07:00
Ed Hartnett
d994a28139 allocating storage for var hdf5 info 2018-11-13 06:26:36 -07:00
Ed Hartnett
9c3a8cb2c5 hdf5-specific group stuff in hdf5var.c 2018-11-12 13:01:54 -07:00
Ed Hartnett
856e4ead03 moved hdf_dimscaleid to hdf5-specific dim info 2018-11-08 10:55:21 -07:00
Ed Hartnett
1dd9004a11 more use of HDF5-specific dim info 2018-11-08 10:43:36 -07:00
Ed Hartnett
bff06f2c4d starting to use HDF5-specific dim info 2018-11-08 10:22:37 -07:00
Ed Hartnett
e89ad03b24 more looking up HDF5-specific dim info 2018-11-08 10:02:17 -07:00
Wei-keng Liao
0ed70756cc Ignore flags NC_MPIIO and NC_MPIPOSIX. 2018-09-22 20:22:34 -05:00
Ed Hartnett
90d5783eea removed commented-out code 2018-09-12 06:08:02 -06:00