mirror of
https://github.com/Unidata/netcdf-c.git
synced 2024-11-21 03:13:42 +08:00
9cde91687a
This is a follow-on in that the old utf8 code was still being used in ncgen to convert utf8->utf16 when converting cdl to Java (see genj.c). The new code apparently has no utf16 support, but it does have utf32 support. Converting utf32 -> utf16 can be approximated by truncating the 32bits to 16 bits, unless the top 16 bits are not zero. This latter condition is unlikely to be common because it implies use of some rather obscure characters. So solution is to convert to utf32 and truncate to 16 bits to get utf16. An error is reported if the high-order truncated 16 bits are not zero. If we get complaints, then I will figure out how to convert full utf32 to a utf16 pair. Other changes: 1. removed the old code from ncgen. 2. changed UTF8PROC_DLLEXPORT (in utf8proc) to EXTERNL and added appropriate includes. This should fix issue https://github.com/Unidata/netcdf-c/issues/404, but since we cannot duplicate the failure, I am not quite sure.
49 lines
1.4 KiB
C
49 lines
1.4 KiB
C
/*
|
|
* Copyright 2017, University Corporation for Atmospheric Research
|
|
* See netcdf/COPYRIGHT file for copying and redistribution conditions.
|
|
*/
|
|
|
|
#ifndef NCUTF8_H
|
|
#define NCUTF8_H 1
|
|
|
|
#include "ncexternl.h"
|
|
|
|
/* Provide a wrapper around whatever utf8 library we use. */
|
|
|
|
/*
|
|
* Check validity of a UTF8 encoded null-terminated byte string.
|
|
* Return codes:
|
|
* NC_NOERR -- string is valid utf8
|
|
* NC_ENOMEM -- out of memory
|
|
* NC_EBADNAME-- not valid utf8
|
|
*/
|
|
EXTERNL int nc_utf8_validate(const unsigned char * name);
|
|
|
|
/*
|
|
* Apply NFC normalization to a string.
|
|
* Returns a pointer to newly allocated memory of an NFC
|
|
* normalized version of the null-terminated string 'str'.
|
|
* Pointer to normalized string is returned in normalp argument;
|
|
* caller must free.
|
|
* Return codes:
|
|
* NC_NOERR -- success
|
|
* NC_ENOMEM -- out of memory
|
|
* NC_EBADNAME -- other failure
|
|
*/
|
|
EXTERNL int nc_utf8_normalize(const unsigned char* str, unsigned char** normalp);
|
|
|
|
/*
|
|
* Convert a normalized utf8 string to utf16. This is approximate
|
|
* because it just does the truncation version of conversion for
|
|
* each 32-bit codepoint to get the corresponding utf16.
|
|
* Return codes:
|
|
* NC_NOERR -- success
|
|
* NC_ENOMEM -- out of memory
|
|
* NC_EINVAL -- invalid argument or internal error
|
|
* NC_EBADNAME-- not valid utf16
|
|
*/
|
|
|
|
EXTERNL int nc_utf8_to_utf16(const unsigned char* s8, unsigned short** utf16p, size_t* lenp);
|
|
|
|
#endif /*NCUTF8_H*/
|