were added to provide a path name converter from e.g. cygwin
paths to e.g. windows paths. This is necessary because
the shell scripts may produce cygwin paths, but the code
may have been compiled with Visual Studio. Similar issues
arise with Mingw.
At appropriate places, and if using Visual Studio or Mingw,
I added calls to the path conversion code.
Apparently I forgot to find all the places where this
conversion was needed. So this pr does the following:
1. Push the calls to the converter to the various libXXX
directories and out of libdispatch/dfile.c.
2. Add conversion calls to other parts of the code like oc2.
I also turns out that conversion code in dapcvt.c
had a bug when handling DAP Byte type under visual studio.
Notes:
1. there may still be places I missed that need to do path conversion.
2. need to make sure that calls to e.g. H5open also use converted path.
The problem was that for opendap, it is possible to use keywords
as identifiers
when there is no ambiguity. However, the DAP2
parser lost the case of the identifier used the lower case version.
Fix is to use the actual text of the symbol when it is used as an identifier.
Also added a test case for this (kwcase.*).
Additionally cleaned up some misc. dap2 testing problems.
1. ncdap_test/tst_ncdap3.sh was using an empty test set.
restored the testing of datasets.
2. as a consequence of #1, some tests needed to be updated with minor
tweeks.
3. fix dapmerge to handle multiple DODS_EXTRAS attributes.
4. modify buildattribute to suppress nul characters and terminate
the name at the first nul.
5. clean up various test scripts to remove residual, unused
references to obsolete netcdf-4 translation.
6. export e.g. NCDUMP from test_common.in so that non-top-level
shell scripts can access it.
Getting the value of the x variable on the file corresponding to the below ncdump output
with -fsanitize=undefined raises
ncx.c:1034:14: runtime error: left shift of 128 by 24 places cannot be represented in type 'int'
This is due to *cp being promoted to int before doing the left shift, instead
of the intended unsigned. So do the cast to unsigned internally rather than
externally
ncdump file to reproduce:
netcdf temp {
dimensions:
x = 2 ;
y = 2 ;
v = 2 ;
variables:
int x(v) ;
byte y(y, x) ;
data:
x = _, _ ;
y =
-127, -127,
-127, -127 ;
}
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2356
Credit to OSS Fuzz
This is a follow-on in that the old utf8 code was still being
used in ncgen to convert utf8->utf16 when converting cdl to Java
(see genj.c).
The new code apparently has no utf16 support, but it does have
utf32 support. Converting utf32 -> utf16 can be approximated by
truncating the 32bits to 16 bits, unless the top 16 bits are
not zero. This latter condition is unlikely to be common because
it implies use of some rather obscure characters.
So solution is to convert to utf32 and truncate to 16 bits to
get utf16. An error is reported if the high-order truncated 16
bits are not zero. If we get complaints, then I will figure out
how to convert full utf32 to a utf16 pair.
Other changes:
1. removed the old code from ncgen.
2. changed UTF8PROC_DLLEXPORT (in utf8proc) to EXTERNL
and added appropriate includes. This should fix
issue https://github.com/Unidata/netcdf-c/issues/404,
but since we cannot duplicate the failure, I am not quite
sure.
Running a build on the .nc file corresponding to the below ncdump output
with -fsanitize=undefined raises
libsrc/ncx.c:4722:26: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'
This is due to *cp being promoted to int before doing the left shift, instead
of the intended unsigned. So do the cast to unsigned internally rather than
externally
ncdump file to reproduce:
netcdf temp {
dimensions:
y = UNLIMITED ; // (0 currently)
x = 109067 ;
variables:
byte t(y, x, x) ;
data:
}
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2265
Credit to OSS Fuzz
Current code only frees char* text in error cases. It should
also free it in success case.
Otherwise Valgrind reports a leak:
==28536== 64 bytes in 1 blocks are definitely lost in loss record 4 of 13
==28536== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28536== by 0xE673496: NC4_buildpropinfo (nc4info.c:239)
==28536== by 0xE67313B: NC4_put_propattr (nc4info.c:162)
==28536== by 0xE65BF35: nc4_create_file (nc4file.c:468)
==28536== by 0xE65C0AC: NC4_create (nc4file.c:564)
==28536== by 0xE608A08: NC_create (dfile.c:1773)
==28536== by 0xE607E6A: nc__create (dfile.c:511)
==28536== by 0xE607E23: nc_create (dfile.c:440)
Credit to OSS Fuzz
This is a follow-on in that the old utf8 code was still being
used in ncgen to convert utf8->utf16 when converting cdl to Java
(see genj.c).
The new code apparently has no utf16 support, but it does have
utf32 support. Converting utf32 -> utf16 can be approximated by
truncating the 32bits to 16 bits, unless the top 16 bits are
not zero. This latter condition is unlikely to be common because
it implies use of some rather obscure characters.
So solution is to convert to utf32 and truncate to 16 bits to
get utf16. An error is reported if the high-order truncated 16
bits are not zero. If we get complaints, then I will figure out
how to convert full utf32 to a utf16 pair.
Also removed the old code from ncgen.
The documentation for `H5Tget_member_name` states that the memory returned should be freed by `H5free_memory` instead of `free`. I was getting test failure until I changed this to call H5free_memory on a Mac OS Sierra system with hdf5-1.9.236
