were added to provide a path name converter from e.g. cygwin
paths to e.g. windows paths. This is necessary because
the shell scripts may produce cygwin paths, but the code
may have been compiled with Visual Studio. Similar issues
arise with Mingw.
At appropriate places, and if using Visual Studio or Mingw,
I added calls to the path conversion code.
Apparently I forgot to find all the places where this
conversion was needed. So this pr does the following:
1. Push the calls to the converter to the various libXXX
directories and out of libdispatch/dfile.c.
2. Add conversion calls to other parts of the code like oc2.
I also turns out that conversion code in dapcvt.c
had a bug when handling DAP Byte type under visual studio.
Notes:
1. there may still be places I missed that need to do path conversion.
2. need to make sure that calls to e.g. H5open also use converted path.
Getting the value of the x variable on the file corresponding to the below ncdump output
with -fsanitize=undefined raises
ncx.c:1034:14: runtime error: left shift of 128 by 24 places cannot be represented in type 'int'
This is due to *cp being promoted to int before doing the left shift, instead
of the intended unsigned. So do the cast to unsigned internally rather than
externally
ncdump file to reproduce:
netcdf temp {
dimensions:
x = 2 ;
y = 2 ;
v = 2 ;
variables:
int x(v) ;
byte y(y, x) ;
data:
x = _, _ ;
y =
-127, -127,
-127, -127 ;
}
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2356
Credit to OSS Fuzz
Running a build on the .nc file corresponding to the below ncdump output
with -fsanitize=undefined raises
libsrc/ncx.c:4722:26: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'
This is due to *cp being promoted to int before doing the left shift, instead
of the intended unsigned. So do the cast to unsigned internally rather than
externally
ncdump file to reproduce:
netcdf temp {
dimensions:
y = UNLIMITED ; // (0 currently)
x = 109067 ;
variables:
byte t(y, x, x) ;
data:
}
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2265
Credit to OSS Fuzz
It turns out that the chunksize used in the
ncio-related code must be a multiple of eight in
size. Both memio.c and mmapio.c were
potentially violating this constraint.
See also pr https://github.com/Unidata/netcdf-c/pull/400
Specific changes:
1. Add dap4 code: libdap4 and dap4_test.
Note that until the d4ts server problem is solved, dap4 is turned off.
2. Modify various files to support dap4 flags:
configure.ac, Makefile.am, CMakeLists.txt, etc.
3. Add nc_test/test_common.sh. This centralizes
the handling of the locations of various
things in the build tree: e.g. where is
ncgen.exe located. See nc_test/test_common.sh
for details.
4. Modify .sh files to use test_common.sh
5. Obsolete separate oc2 by moving it to be part of
netcdf-c. This means replacing code with netcdf-c
equivalents.
5. Add --with-testserver to configure.ac to allow
override of the servers to be used for --enable-dap-remote-tests.
6. There were multiple versions of nctypealignment code. Try to
centralize in libdispatch/doffset.c and include/ncoffsets.h
7. Add a unit test for the ncuri code because of its complexity.
8. Move the findserver code out of libdispatch and into
a separate, self contained program in ncdap_test and dap4_test.
9. Move the dispatch header files (nc{3,4}dispatch.h) to
.../include because they are now shared by modules.
10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts.
11. Make use of MREMAP if available
12. Misc. minor changes e.g.
- #include <config.h> -> #include "config.h"
- Add some no-install headers to /include
- extern -> EXTERNL and vice versa as needed
- misc header cleanup
- clean up checking for misc. unix vs microsoft functions
13. Change copyright decls in some files to point to LICENSE file.
14. Add notes to RELEASENOTES.md
Update utf8proc.[ch] to use the version now
maintained by the Julia Language project
(https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md).
The license for the previous version was
unacceptable for the Debian and Ubuntu release
systems. The new version both updates the code
and addresses the license issue.
It turns out that the utf8proc software we are using
was turned over to the Julia Language developers
and the license terms changed to allow modification.
(https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md).
So the fix here is as follows:
1. Wrap the library with a fixed interface: libdispatch/dutf8.c
and include/ncutf8.h.
2. Replace the existing utf8proc code with the new version
from https://github.com/JuliaLang/utf8proc.
3. Add a couple more test cases: nc_test/tst_utf8_validate.c
and nc_test_utf8_phrases.c. If/when I can find a usable
normalization test, I will incorporate that later.
