2. modified ncdap_tests to remove <cr>
from generated output before comparison
to expected. This is a hack because
my other attempts to force windows to use
binary mode have not worked.
were added to provide a path name converter from e.g. cygwin
paths to e.g. windows paths. This is necessary because
the shell scripts may produce cygwin paths, but the code
may have been compiled with Visual Studio. Similar issues
arise with Mingw.
At appropriate places, and if using Visual Studio or Mingw,
I added calls to the path conversion code.
Apparently I forgot to find all the places where this
conversion was needed. So this pr does the following:
1. Push the calls to the converter to the various libXXX
directories and out of libdispatch/dfile.c.
2. Add conversion calls to other parts of the code like oc2.
I also turns out that conversion code in dapcvt.c
had a bug when handling DAP Byte type under visual studio.
Notes:
1. there may still be places I missed that need to do path conversion.
2. need to make sure that calls to e.g. H5open also use converted path.
Some temporary files are being left in a tempdir (e.g. /tmp
under *nix*).
The situation is described tersely in
netcdf-c/docs/auth.html#REDIR Basically, when a url is used that
requires redirection, a physical cookiejar file is required
to exist in the file system in order for this to work.
Since it was difficult to figure out when redirection was
being used (it was internal to libcurl) I needed to be prepared for that
eventuality. The result was that I always created a cookiejar file if one
was not specified in the rc file. This actually occurs in two places:
one inside oc2 and one inside libdap4.
The solution was two-fold:
1. do not use a cookiejar directory -- create cookiejar file directly
2. ensure that all cookiejar related files are reclaimed by nc_close().
Note that if nc_close (or nc_abort) is not called for whatever reason,
then reclamation will not occur.
The problem was that for opendap, it is possible to use keywords
as identifiers
when there is no ambiguity. However, the DAP2
parser lost the case of the identifier used the lower case version.
Fix is to use the actual text of the symbol when it is used as an identifier.
Also added a test case for this (kwcase.*).
Additionally cleaned up some misc. dap2 testing problems.
1. ncdap_test/tst_ncdap3.sh was using an empty test set.
restored the testing of datasets.
2. as a consequence of #1, some tests needed to be updated with minor
tweeks.
3. fix dapmerge to handle multiple DODS_EXTRAS attributes.
4. modify buildattribute to suppress nul characters and terminate
the name at the first nul.
5. clean up various test scripts to remove residual, unused
references to obsolete netcdf-4 translation.
6. export e.g. NCDUMP from test_common.in so that non-top-level
shell scripts can access it.
Getting the value of the x variable on the file corresponding to the below ncdump output
with -fsanitize=undefined raises
ncx.c:1034:14: runtime error: left shift of 128 by 24 places cannot be represented in type 'int'
This is due to *cp being promoted to int before doing the left shift, instead
of the intended unsigned. So do the cast to unsigned internally rather than
externally
ncdump file to reproduce:
netcdf temp {
dimensions:
x = 2 ;
y = 2 ;
v = 2 ;
variables:
int x(v) ;
byte y(y, x) ;
data:
x = _, _ ;
y =
-127, -127,
-127, -127 ;
}
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2356
Credit to OSS Fuzz
This is a follow-on in that the old utf8 code was still being
used in ncgen to convert utf8->utf16 when converting cdl to Java
(see genj.c).
The new code apparently has no utf16 support, but it does have
utf32 support. Converting utf32 -> utf16 can be approximated by
truncating the 32bits to 16 bits, unless the top 16 bits are
not zero. This latter condition is unlikely to be common because
it implies use of some rather obscure characters.
So solution is to convert to utf32 and truncate to 16 bits to
get utf16. An error is reported if the high-order truncated 16
bits are not zero. If we get complaints, then I will figure out
how to convert full utf32 to a utf16 pair.
Other changes:
1. removed the old code from ncgen.
2. changed UTF8PROC_DLLEXPORT (in utf8proc) to EXTERNL
and added appropriate includes. This should fix
issue https://github.com/Unidata/netcdf-c/issues/404,
but since we cannot duplicate the failure, I am not quite
sure.
Running a build on the .nc file corresponding to the below ncdump output
with -fsanitize=undefined raises
libsrc/ncx.c:4722:26: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'
This is due to *cp being promoted to int before doing the left shift, instead
of the intended unsigned. So do the cast to unsigned internally rather than
externally
ncdump file to reproduce:
netcdf temp {
dimensions:
y = UNLIMITED ; // (0 currently)
x = 109067 ;
variables:
byte t(y, x, x) ;
data:
}
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2265
Credit to OSS Fuzz
