Commit Graph

7 Commits

Author SHA1 Message Date
Ward Fisher
bb65fe5ea5 More potential memory leaks squashed. 2018-06-07 15:58:47 -06:00
Dennis Heimbigner
8f2f9b74e3 re: pull request https://github.com/Unidata/netcdf-c/pull/364
This is a follow-on in that the old utf8 code was still being
used in ncgen to convert utf8->utf16 when converting cdl to Java
(see genj.c).

The new code apparently has no utf16 support, but it does have
utf32 support. Converting utf32 -> utf16 can be approximated by
truncating the 32bits to 16 bits, unless the top 16 bits are
not zero. This latter condition is unlikely to be common because
it implies use of some rather obscure characters.

So solution is to convert to utf32 and truncate to 16 bits to
get utf16. An error is reported if the high-order truncated 16
bits are not zero. If we get complaints, then I will figure out
how to convert full utf32 to a utf16 pair.

Also removed the old code from ncgen.
2017-06-06 15:23:59 -06:00
Ward Fisher
6bf5a38d90 Corrected an issue with autotools and OSX and utf8. 2017-04-05 11:14:42 -06:00
Ward Fisher
773fe80727 Refactored some data types due to collision. 2017-02-28 11:12:00 -07:00
Ward Fisher
5883b6587f Additional utf8 refactoring to avoid name collisions. 2017-02-28 11:05:45 -07:00
Ward Fisher
f20228def5 Refactoring utf8 2017-02-28 10:36:46 -07:00
Dennis Heimbigner
47daf33074 Resolves Github issue https://github.com/Unidata/netcdf-c/issues/349.
Update utf8proc.[ch] to use the version now
maintained by the Julia Language project
(https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md).
The license for the previous version was
unacceptable for the Debian and Ubuntu release
systems. The new version both updates the code
and addresses the license issue.

It turns out that the utf8proc software we are using
was turned over to the Julia Language developers
and the license terms changed to allow modification.
(https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md).

So the fix here is as follows:
1. Wrap the library with a fixed interface: libdispatch/dutf8.c
   and include/ncutf8.h.
2. Replace the existing utf8proc code with the new version
   from https://github.com/JuliaLang/utf8proc.
3. Add a couple more test cases: nc_test/tst_utf8_validate.c
   and nc_test_utf8_phrases.c.  If/when I can find a usable
   normalization test, I will incorporate that later.
2017-02-16 14:27:54 -07:00