diff --git a/docs/filters.md b/docs/filters.md index adc5ddd99..c39741c6d 100644 --- a/docs/filters.md +++ b/docs/filters.md @@ -2,13 +2,12 @@ NetCDF-4 Filter Support ============================ -NetCDF-4 Filter Support {#compress} -================================= +NetCDF-4 Filter Support {#filters} +============================ [TOC] -Introduction {#compress_intro} -================== +# Introduction {#filters_intro} The HDF5 library (1.8.11 and later) supports a general filter mechanism to apply various @@ -37,8 +36,7 @@ can locate, load, and utilize the compressor. These libraries are expected to installed in a specific directory. -Enabling A Compression Filter {#Enable} -============================= +# Enabling A Compression Filter {#filters_enable} In order to compress a variable, the netcdf-c library must be given three pieces of information: @@ -66,8 +64,7 @@ using __ncgen__, via an API call, or via command line parameters to __nccopy__. In any case, remember that filtering also requires setting chunking, so the variable must also be marked with chunking information. -Using The API {#API} -------------- +## Using The API {#filters_API} The necessary API methods are included in __netcdf.h__ by default. One API method is for setting the filter to be used when writing a variable. The relevant signature is @@ -90,8 +87,7 @@ __params__. As is usual with the netcdf API, one is expected to call this function twice. The first time to get __nparams__ and the second to get the parameters in client-allocated memory. -Using ncgen {#NCGEN} -------------- +## Using ncgen {#filters_NCGEN} In a CDL file, compression of a variable can be specified by annotating it with the following attribute: @@ -107,8 +103,8 @@ This is a "special" attribute, which means that it will normally be invisible when using __ncdump__ unless the -s flag is specified. -Example CDL File (Data elided) ------------------------------- +### Example CDL File (Data elided) + ```` netcdf bzip2 { dimensions: @@ -123,8 +119,8 @@ data: } ```` -Using nccopy {#NCCOPY} -------------- +## Using nccopy {#filters_NCCOPY} + When copying a netcdf file using __nccopy__ it is possible to specify filter information for any output variable by using the "-F" option on the command line; for example: @@ -171,8 +167,7 @@ by this table. false-Fvar,...NAuse output filter -Parameter Encoding {#ParamEncode} -========== +# Parameter Encode/Decode {#filters_paramcoding} The parameters passed to a filter are encoded internally as a vector of 32-bit unsigned integers. It may be that the parameters @@ -198,57 +193,83 @@ them between the local machine byte order and network byte order. Parameters whose size is larger than 32-bits present a byte order problem. -This typically includes double precision floats and (signed or unsigned) -64-bit integers. For these cases, the machine byte order must be -handled by the compression code. This is because HDF5 will treat, +This specifically includes double precision floats and (signed or unsigned) +64-bit integers. For these cases, the machine byte order issue must be +handled, in part, by the compression code. This is because HDF5 will treat, for example, an unsigned long long as two 32-bit unsigned integers and will convert each to network order separately. This means that on a machine whose byte order is different than the machine in which -the parameters were initially created, the two integers are out of order -and must be swapped to get the correct unsigned long long value. -Consider this example. Suppose we have this little endian unsigned long long. +the parameters were initially created, the two integers will be separately +endian converted. But this will be incorrect for 64-bit values. - 1000000230000004 +So, we have this situation: -In network byte order, it will be stored as two 32-bit integers. +1. the 8 bytes come in as native machine order for the machine + doing the call to *nc_def_var_filter*. +2. HDF5 divides the 8 bytes into 2 four byte pieces and ensures that each piece + is in network (big) endian order. +3. When the filter is called, the two pieces are returned in the same order + but with the bytes in each piece consistent with the native machine order + for the machine executing the filter. - 20000001 40000003 +## Encoding Algorithms -On a big endian machine, this will be given to the filter in that form. +In order to properly extract the correct 8-byte value, we need to ensure +that the values stored in the HDF5 file have a known format independent of +the native format of the creating machine. - 2000000140000003 +The idea is to do sufficient manipulation so that HDF5 +will store the 8-byte value as a little endian value +divided into two 4-byte integers. +Note that little-endian is used as the standard +because it is the most common machine format. +When read, the filter code needs to be aware of this convention +and do the appropriate conversions. -But note that the proper big endian unsigned long long form is this. +This leads to the following set of rules. -4000000320000001 +### Encoding -So, the two words need to be swapped. +1. Encode on little endian (LE) machine: no special action is required. + The 8-byte value is passed to HDF5 as two 4-byte integers. HDF5 byte + swaps each integer and stores it in the file. +2. Encode on a big endian (BE) machine: several steps are required: -But consider the case when both original and final machines are big endian. + 1. Do an 8-byte byte swap to convert the original value to little-endian + format. + 2. Since the encoding machine is BE, HDF5 will just store the value. + So it is necessary to simulate little endian encoding by byte-swapping + each 4-byte integer separately. + 3. This doubly swapped pair of integers is then passed to HDF5 and is stored + unchanged. -1. 4000000320000001 -2. 40000003 20000001 -3. 40000003 20000001 +### Decoding -where #1 is the original number, #2 is the network order and -#3 is the what is given to the filter. In this case we do not -want to swap words. +1. Decode on LE machine: no special action is required. + HDF5 will get the two 4-bytes values from the file and byte-swap each + separately. The concatenation of those two integers will be the expected + LE value. +2. Decode on a big endian (BE) machine: the inverse of the encode case must + be implemented. -The solution is to forcibly encode the original number using some -specified endianness so that the filter always assumes it is getting -its parameters in that order and will always do swapping as needed. -This is irritating, but one needs to be aware of it. Since most -machines are little-endian. We choose to use that as the endianness -for handling 64 bit entities. + 1. HDF5 sends the two 4-byte values to the filter. + 2. The filter must then byte-swap each 4-byte value independently. + 3. The filter then must concatenate the two 4-byte values into a single + 8-byte value. Because of the encoding rules, this 8-byte value will + be in LE format. + 4. The filter must finally do an 8-byte byte-swap on that 8-byte value + to convert it to desired BE format. -Filter Specification Syntax {#Syntax} -========== +To support these rules, some utility programs exist and are discussed in +Appendix A. + +# Filter Specification Syntax {#filters_syntax} Both of the utilities __ncgen__ and __nccopy__ -allow the specification of filter parameters. +allow the specification of filter parameters in text format. These specifications consist of a sequence of comma separated constants. The constants are converted within the utility to a proper set of unsigned int @@ -257,7 +278,7 @@ constants (see the parameter encoding section). To simplify things, various kinds of constants can be specified rather than just simple unsigned integers. The utilities will encode them properly using the rules specified in -the parameter encoding section. +the section on parameter encode/decode. The currently supported constants are as follows. @@ -270,9 +291,9 @@ The currently supported constants are as follows.
77implicit unsigned 32-bit integerNo tag
93Uexplicit unsigned 32-bit integeru|U
789f32-bit floatf|F -
12345678.12345678d64-bit doubled|DNetwork byte order -
-9223372036854775807L64-bit signed long longl|LNetwork byte order -
18446744073709551615UL64-bit unsigned long longu|U l|LNetwork byte order +
12345678.12345678d64-bit doubled|DLE encoding +
-9223372036854775807L64-bit signed long longl|LLE encoding +
18446744073709551615UL64-bit unsigned long longu|U l|LLE encoding
Some things to note. @@ -283,10 +304,10 @@ Some things to note. 2. For signed byte and short, the value is sign extended to 32 bits and then treated as an unsigned int value. 3. For double, and signed|unsigned long long, they are converted - to network byte order and then treated as two unsigned int values. - This is consistent with the parameter encoding. + as specified in the section on + parameter encode/decode. -Dynamic Loading Process {#Process} +Dynamic Loading Process {#filters_Process} ========== The documentation[1,2] for the HDF5 dynamic loading was (at the time @@ -294,7 +315,7 @@ this was written) out-of-date with respect to the actual HDF5 code (see HDF5PL.c). So, the following discussion is largely derived from looking at the actual code. This means that it is subject to change. -Plugin directory {#Plugindir} +Plugin directory {#filters_Plugindir} ---------------- The HDF5 loader expects plugins to be in a specified plugin directory. @@ -306,7 +327,7 @@ The default directory is: The default may be overridden using the environment variable __HDF5_PLUGIN_PATH__. -Plugin Library Naming {#Pluginlib} +Plugin Library Naming {#filters_Pluginlib} --------------------- Given a plugin directory, HDF5 examines every file in that @@ -320,7 +341,7 @@ as determined by the platform on which the library is being executed. Windows*.dll -Plugin Verification {#Pluginverify} +Plugin Verification {#filters_Pluginverify} ------------------- For each dynamic library located using the previous patterns, HDF5 attempts to load the library and attempts to obtain information @@ -340,7 +361,7 @@ specified for the variable in __nc_def_var_filter__ in order to be used. If plugin verification fails, then that plugin is ignored and the search continues for another, matching plugin. -Debugging {#Debug} +Debugging {#filters_Debug} ------- Debugging plugins can be very difficult. You will probably need to use the old printf approach for debugging the filter itself. @@ -356,7 +377,7 @@ Since ncdump is not being asked to access the data (the -h flag), it can obtain the filter information without failures. Then it can print out the filter id and the parameters (the -s flag). -Test Case {#TestCase} +Test Case {#filters_TestCase} ------- Within the netcdf-c source tree, the directory __netcdf-c/nc_test4__ contains a test case (__test_filter.c__) for @@ -365,7 +386,7 @@ bzip2. Another test (__test_filter_misc.c__) validates parameter passing. These tests are disabled if __--enable-shared__ is not set or if __--enable-netcdf-4__ is not set. -Example {#Example} +Example {#filters_Example} ------- A slightly simplified version of the filter test case is also available as an example within the netcdf-c source tree @@ -444,45 +465,35 @@ has been known to work. gcc -g -O0 -shared -o libbzip2.so -L${HDF5LIBDIR} -lhdf5_hl -lhdf5 -L${ZLIBDIR} -lz ```` -Appendix A. Byte Swap Code {#AppendixA} +Appendix A. Support Utilities {#filters_AppendixA} ========== -Since in some cases, it is necessary for a filter to -byte swap from little-endian to big-endian, This appendix -provides sample code for doing this. It also provides -a code snippet for testing if the machine the -endianness of a machine. -Byte swap an 8-byte chunk of memory -------- -```` -static void -byteswap8(unsigned char* mem) -{ - register unsigned char c; - c = mem[0]; - mem[0] = mem[7]; - mem[7] = c; - c = mem[1]; - mem[1] = mem[6]; - mem[6] = c; - c = mem[2]; - mem[2] = mem[5]; - mem[5] = c; - c = mem[3]; - mem[3] = mem[4]; - mem[4] = c; -} +Two functions are exported from the netcdf-c library +for use by client programs and by filter implementations. -```` +1. ````int NC_parsefilterspec(const char* spec, unsigned int* idp, size_t* nparamsp, unsigned int** paramsp);```` + * idp will contain the filter id value from the spec. + * nparamsp will contain the number of 4-byte parameters + * paramsp will contain a pointer to the parsed parameters -- the caller + must free. + This function can parse filter spec strings as defined in + the section on Filter Specification Syntax. + This function parses the first argument and returns several values. -Test for Machine Endianness -------- -```` -static const unsigned char b[4] = {0x0,0x0,0x0,0x1}; /* value 1 in big-endian*/ -int endianness = (1 == *(unsigned int*)b); /* 1=>big 0=>little endian -```` -References {#References} -======================== +2. ````int NC_filterfix8(unsigned char* mem8, int decode);```` + * mem8 is a pointer to the 8-byte value either to fix. + * decode is 1 if the function should apply the 8-byte decoding algorithm + else apply the encoding algorithm. + This function implements the 8-byte conversion algorithms. + Before calling *nc_def_var_filter* (unless *NC_parsefilterspec* was used), + the client must call this function with the decode argument set to 0. + Inside the filter code, this function should be called with the decode + argument set to 1. + +Examples of the use of these functions can be seen in the test program +*nc_test4/tst_filterparser.c*. + +# References {#filters_References} 1. https://support.hdfgroup.org/HDF5/doc/Advanced/DynamicallyLoadedFilters/HDF5DynamicallyLoadedFilters.pdf 2. https://support.hdfgroup.org/HDF5/doc/TechNotes/TechNote-HDF5-CompressionTroubleshooting.pdf @@ -490,8 +501,7 @@ References {#References} 4. https://support.hdfgroup.org/services/contributions.html#filters 5. https://support.hdfgroup.org/HDF5/doc/RM/RM_H5.html -Point of Contact -================ +# Point of Contact __Author__: Dennis Heimbigner
__Email__: dmh at ucar dot edu diff --git a/include/Makefile.am b/include/Makefile.am index 7ce2654c4..be73a2cb7 100644 --- a/include/Makefile.am +++ b/include/Makefile.am @@ -6,7 +6,7 @@ # Ed Hartnett, Dennis Heimbigner, Ward Fisher -include_HEADERS = netcdf.h netcdf_meta.h netcdf_mem.h netcdf_aux.h +include_HEADERS = netcdf.h netcdf_meta.h netcdf_mem.h netcdf_aux.h netcdf_filter.h if BUILD_PARALLEL include_HEADERS += netcdf_par.h @@ -17,7 +17,7 @@ ncuri.h ncutf8.h ncdispatch.h ncdimscale.h netcdf_f.h err_macros.h \ ncbytes.h nchashmap.h ceconstraints.h rnd.h nclog.h ncconfigure.h \ nc4internal.h nctime.h nc3internal.h onstack.h ncrc.h ncauth.h \ ncoffsets.h nctestserver.h nc4dispatch.h nc3dispatch.h ncexternl.h \ -ncwinpath.h ncfilter.h ncindex.h hdf4dispatch.h hdf5internal.h \ +ncwinpath.h ncindex.h hdf4dispatch.h hdf5internal.h \ nc_provenance.h hdf5dispatch.h if USE_DAP diff --git a/include/ncfilter.h b/include/netcdf_filter.h similarity index 77% rename from include/ncfilter.h rename to include/netcdf_filter.h index 3c0078463..13f811925 100644 --- a/include/ncfilter.h +++ b/include/netcdf_filter.h @@ -1,8 +1,8 @@ /* Copyright 2018, UCAR/Unidata and OPeNDAP, Inc. See the COPYRIGHT file for more information. */ -#ifndef NCFILTER_H -#define NCFILTER_H 1 +#ifndef NETCDF_FILTER_H +#define NETCDF_FILTER_H 1 /* API for libdispatch/dfilter.c */ @@ -18,10 +18,10 @@ extern "C" { /* Provide consistent filter spec parser */ EXTERNL int NC_parsefilterspec(const char* spec, unsigned int* idp, size_t* nparamsp, unsigned int** paramsp); -EXTERNL void NC_byteswap8(unsigned char* mem); +EXTERNL void NC_filterfix8(unsigned char* mem, int decode); #if defined(__cplusplus) } #endif -#endif /* NCFILTER_H */ +#endif /* NETCDF_FILTER_H */ diff --git a/libdap4/d4data.c b/libdap4/d4data.c index 51ea6e28b..16ad52ed9 100644 --- a/libdap4/d4data.c +++ b/libdap4/d4data.c @@ -76,15 +76,8 @@ NCD4_processdata(NCD4meta* meta) FAIL(ret,"delimit failure"); } - /* Swap the data for each top level variable, - including the checksum (if any) - */ - if(meta->swap) { - if((ret=NCD4_swapdata(meta,toplevel))) - FAIL(ret,"byte swapping failed"); - } - /* Compute the checksums of the top variables */ + /* must occur before any byte swapping */ if(meta->localchecksumming) { for(i=0;iswap) { + if((ret=NCD4_swapdata(meta,toplevel))) + FAIL(ret,"byte swapping failed"); + } + done: if(toplevel) nclistfree(toplevel); return THROW(ret); diff --git a/libdap4/d4debug.h b/libdap4/d4debug.h index cbba56bea..90d4b4957 100644 --- a/libdap4/d4debug.h +++ b/libdap4/d4debug.h @@ -27,7 +27,7 @@ #endif -#define D4CATCH /* Warning: significant performance impact */ +#undef D4CATCH /* Warning: significant performance impact */ #define PANIC(msg) assert(d4panic(msg)); #define PANIC1(msg,arg) assert(d4panic(msg,arg)); diff --git a/libdap4/d4swap.c b/libdap4/d4swap.c index 3fc0aa58f..a87ee994a 100644 --- a/libdap4/d4swap.c +++ b/libdap4/d4swap.c @@ -229,10 +229,10 @@ walkSeq(NCD4meta* compiler, NCD4node* topvar, NCD4node* vlentype, void** offsetp offset = *offsetp; /* process the record count */ + if(compiler->swap) + swapinline64(offset); recordcount = GETCOUNTER(offset); SKIPCOUNTER(offset); - if(compiler->swap) - swapinline64(&recordcount); basetype = vlentype->basetype; /* This may be of any type potentially */ assert(basetype->sort == NCD4_TYPE); diff --git a/libdispatch/dfilter.c b/libdispatch/dfilter.c index e48eddfcb..aae6ce6b9 100644 --- a/libdispatch/dfilter.c +++ b/libdispatch/dfilter.c @@ -12,6 +12,7 @@ #endif #include "netcdf.h" +#include "netcdf_filter.h" /* Common utilities related to filters. @@ -47,7 +48,7 @@ NC_parsefilterspec(const char* spec, unsigned int* idp, size_t* nparamsp, unsign size_t len; int i; unsigned int* ulist = NULL; - unsigned char mem[8]; /* to convert to network byte order */ + unsigned char mem[8]; if(spec == NULL || strlen(spec) == 0) goto fail; sdata = strdup(spec); @@ -135,14 +136,13 @@ NC_parsefilterspec(const char* spec, unsigned int* idp, size_t* nparamsp, unsign ulist[nparams++] = *(unsigned int*)&valf; break; + /* The following are 8-byte values, so we must swap pieces if this + is a little endian machine */ case 'd': sstat = sscanf(p,"%lf",&vald); if(sstat != 1) goto fail; - /* convert to network byte order */ memcpy(mem,&vald,sizeof(mem)); -#ifdef WORDS_BIGENDIAN - NC_byteswap8(mem); /* convert big endian to little endian */ -#endif + NC_filterfix8(mem,0); vector = (unsigned int*)mem; ulist[nparams++] = vector[0]; ulist[nparams++] = vector[1]; @@ -153,12 +153,9 @@ NC_parsefilterspec(const char* spec, unsigned int* idp, size_t* nparamsp, unsign else sstat = sscanf(p,"%lld",(long long*)&val64u); if(sstat != 1) goto fail; - /* convert to network byte order */ memcpy(mem,&val64u,sizeof(mem)); -#ifdef WORDS_BIGENDIAN - NC_byteswap8(mem); /* convert big endian to little endian */ -#endif - vector = (unsigned int*)mem; + NC_filterfix8(mem,0); + vector = (unsigned int*)&mem; ulist[nparams++] = vector[0]; ulist[nparams++] = vector[1]; break; @@ -216,9 +213,8 @@ gettype(const int q0, const int q1, int* isunsignedp) #ifdef WORDS_BIGENDIAN /* Byte swap an 8-byte integer in place */ -EXTERNL -void -NC_byteswap8(unsigned char* mem) +static void +byteswap8(unsigned char* mem) { unsigned char c; c = mem[0]; @@ -234,4 +230,35 @@ NC_byteswap8(unsigned char* mem) mem[3] = mem[4]; mem[4] = c; } + +/* Byte swap an 8-byte integer in place */ +static void +byteswap4(unsigned char* mem) +{ + unsigned char c; + c = mem[0]; + mem[0] = mem[3]; + mem[3] = c; + c = mem[1]; + mem[1] = mem[2]; + mem[2] = c; +} #endif + +EXTERNL void +NC_filterfix8(unsigned char* mem, int decode) +{ +#ifdef WORDS_BIGENDIAN + if(decode) { /* Apply inverse of the encode case */ + byteswap4(mem); /* step 1: byte-swap each piece */ + byteswap4(mem+4); + byteswap8(mem); /* step 2: convert to little endian format */ + } else { /* encode */ + byteswap8(mem); /* step 1: convert to little endian format */ + byteswap4(mem); /* step 2: byte-swap each piece */ + byteswap4(mem+4); + } +#else /* Little endian */ + /* No action is necessary */ +#endif +} diff --git a/nc_test4/test_filter_misc.c b/nc_test4/test_filter_misc.c index 086cebedd..465e89668 100644 --- a/nc_test4/test_filter_misc.c +++ b/nc_test4/test_filter_misc.c @@ -10,6 +10,7 @@ #include #include "netcdf.h" +#include "netcdf_filter.h" #undef DEBUG @@ -160,7 +161,7 @@ verifyparams(void) static int openfile(void) { - unsigned int* params; + unsigned int* params = NULL; /* Open the file and check it. */ CHECK(nc_open(TESTFILE, NC_NOWRITE, &ncid)); @@ -191,6 +192,8 @@ openfile(void) } if(nerrs > 0) return NC_EFILTER; + if(params) free(params); + /* Verify chunking */ if(!verifychunks()) return 0; @@ -287,7 +290,13 @@ showparameters(void) static void insert(int index, void* src, size_t size) { + unsigned char src8[8]; void* dst = &baseline[index]; + if(size == 8) { + memcpy(src8,src,size); + NC_filterfix8(src8,0); + src = src8; + } memcpy(dst,src,size); } diff --git a/nc_test4/tst_filterparser.c b/nc_test4/tst_filterparser.c index 333b49f2f..4554a4172 100644 --- a/nc_test4/tst_filterparser.c +++ b/nc_test4/tst_filterparser.c @@ -9,7 +9,7 @@ #include #include "netcdf.h" -#include "ncfilter.h" +#include "netcdf_filter.h" #define PARAMS_ID 32768 @@ -19,19 +19,14 @@ */ #define DBLVAL 12345678.12345678 +#define FLTVAL 789.0 + +#define LONGLONGVAL -9223372036854775807LL +#define ULONGLONGVAL 18446744073709551615ULL + #define MAXPARAMS 32 #define NPARAMS 16 /* # of unsigned ints in params */ -static unsigned int baseline[NPARAMS]; - -/* Expected contents of baseline: -id = 32768 -params = 4294967279, 23, 4294967271, 27, 77, 93, 1145389056, 3287505826, 1097305129, 1, 2147483647, 4294967295U, 4294967295U -*/ - -static const char* spec = -"32768, -17b, 23ub, -25S, 27US, 77, 93U, 789f, 12345678.12345678d, -9223372036854775807L, 18446744073709551615UL, 2147483647, -2147483648, 4294967295U"; - /* Test support for the conversions */ /* Not sure if this kind of casting via union is legal C99 */ static union { @@ -50,6 +45,79 @@ static union { long long ll; } ul; +/* Expected contents of baseline: +id = 32768 +index spec item Value +---------------------------------------- +0 -17b 4294967279 +1 23ub 23 +2 -25S 42949677271 +3 27US 27 +4 77 77 +5 93U 93 +6 2147483647 ? +7 -2147483648 ? +8 4294967295U ? +9 789f 1145389056 +<8-bytes start here > +10 -9223372036854775807L 1, 2147483647 +12 18446744073709551615UL 4294967295U, 4294967295U +14 12345678.12345678d 3287505826, 1097305129 + +expected (LE): +{239, 23, 65511, 27, 77, 93, 2147483647, 2147483648, 4294967295, 1145389056, + 1, 2147483648, 4294967295, 4294967295, 3287505826, 1097305129} + +params (LE): +0x000000ef +0x00000017 +0x0000ffe7 +0x0000001b +0x0000004d +0x0000005d +0x7fffffff +0x80000000 +0xffffffff +0x44454000 +0x000000001 .ll +0x80000000 +0xffffffff .ull +0xffffffff +0xc3f35ba2 .d +0x41678c29 + +expected (BE): +{239, 23, 65511, 27, 77, 93, 2147483647, 2147483648, 4294967295, 1145389056, + 16777216, 128, 4294967295, 4294967295, 2723935171, 697067329} + +params (BE): +0x000000ef +0x00000017 +0x0000ffe7 +0x0000001b +0x0000004d +0x0000005d +0x7fffffff +0x80000000 +0xffffffff +0x44454000 +0x01000000 .ll +0x00000080 +0xffffffff .ull +0xffffffff +0xa25bf3c3 .d +0x298c6741 + +*/ + +static unsigned int baseline[MAXPARAMS]; /* Expected */ + +static const char* spec = +"32768, -17b, 23ub, -25S, 27US, 77, 93U, 2147483647, -2147483648, 4294967295U, 789f, -9223372036854775807L, 18446744073709551615UL, 12345678.12345678d"; + +/* Define the type strings for each spec entry */ +static const char* spectype[] = {"i", "b", "ub", "s", "us", "i", "ui", "i", "i", "ui", "f", "ll", "ull", "d"}; + static int nerrs = 0; static void @@ -85,31 +153,31 @@ buildbaseline(void) double float8; val4 = ((unsigned int)-17) & 0xff; - insert(0,&val4,sizeof(val4)); /* 0 signed int*/ + insert(0,&val4,sizeof(val4)); /* signed int*/ val4 = (unsigned int)23; - insert(1,&val4,sizeof(val4)); /* 1 unsigned int*/ + insert(1,&val4,sizeof(val4)); /* unsigned int*/ val4 = ((unsigned int)-25) & 0xffff; - insert(2,&val4,sizeof(val4)); /* 3 signed int*/ + insert(2,&val4,sizeof(val4)); /* signed int*/ val4 = (unsigned int)27; - insert(3,&val4,sizeof(val4)); /* 4 unsigned int*/ + insert(3,&val4,sizeof(val4)); /* unsigned int*/ val4 = (unsigned int)77; - insert(4,&val4,sizeof(val4)); /* 4 signed int*/ + insert(4,&val4,sizeof(val4)); /* signed int*/ val4 = (unsigned int)93; - insert(5,&val4,sizeof(val4)); /* 5 unsigned int*/ + insert(5,&val4,sizeof(val4)); /* unsigned int*/ + val4 = 2147483647; /*0x7fffffff*/ + insert(6,&val4,sizeof(val4)); /* signed int */ + val4 = (-2147483647)-1; /*0x80000000*/ + insert(7,&val4,sizeof(val4)); /* signed int */ + val4 = 4294967295U; /*0xffffffff*/ + insert(8,&val4,sizeof(val4)); /* unsigned int */ float4 = 789.0f; - insert(6,&float4,sizeof(float4)); /* 6 float */ - float8 = DBLVAL; - insert(7,&float8,sizeof(float8)); /* 7 double */ + insert(9,&float4,sizeof(float4)); /*float */ val8 = -9223372036854775807L; - insert(9,&val8,sizeof(val8)); /* 9 signed long long */ + insert(10,&val8,sizeof(val8)); /* signed long long */ val8 = 18446744073709551615UL; - insert(11,&val8,sizeof(val8)); /* 11 unsigned long long */ - val4 = 2147483647; - insert(13,&val4,sizeof(val4)); /* 13 signed int */ - val4 = (-2147483647)-1; - insert(14,&val4,sizeof(val4)); /* 14 signed int */ - val4 = 4294967295U; - insert(15,&val4,sizeof(val4)); /* 15 unsigned int */ + insert(12,&val8,sizeof(val8)); /* unsigned long long */ + float8 = DBLVAL; + insert(114,&float8,sizeof(float8)); /* double */ } /**************************************************/ @@ -132,46 +200,224 @@ main(int argc, char **argv) } if(id != PARAMS_ID) fprintf(stderr,"mismatch: id: expected=%u actual=%u\n",PARAMS_ID,id); - for(i=0;i 0 ? 1 : 0); } -#ifdef WORD_BIGENDIAN +#if 0 +/* Look at q0 and q1) to determine type */ +static int +gettype(const int q0, const int q1, int* isunsignedp) +{ + int type = 0; + int isunsigned = 0; + char typechar; + + isunsigned = (q0 == 'u' || q0 == 'U'); + if(q1 == '\0') + typechar = q0; /* we were given only a single char */ + else if(isunsigned) + typechar = q1; /* we have something like Ux as the tag */ + else + typechar = q1; /* look at last char for tag */ + switch (typechar) { + case 'f': case 'F': case '.': type = 'f'; break; /* float */ + case 'd': case 'D': type = 'd'; break; /* double */ + case 'b': case 'B': type = 'b'; break; /* byte */ + case 's': case 'S': type = 's'; break; /* short */ + case 'l': case 'L': type = 'l'; break; /* long long */ + case '0': case '1': case '2': case '3': case '4': + case '5': case '6': case '7': case '8': case '9': type = 'i'; break; + case 'u': case 'U': type = 'i'; isunsigned = 1; break; /* unsigned int */ + case '\0': type = 'i'; break; + default: break; + } + if(isunsignedp) *isunsignedp = isunsigned; + return type; +} + +static int +parsefilterspec(const char* spec, unsigned int* idp, size_t* nparamsp, unsigned int** paramsp) +{ + int stat = NC_NOERR; + int sstat; /* for scanf */ + char* p; + char* sdata = NULL; + unsigned int id; + size_t count; /* no. of comma delimited params */ + size_t nparams; /* final no. of unsigned ints */ + size_t len; + int i; + unsigned int* ulist = NULL; + unsigned char mem[8]; + + if(spec == NULL || strlen(spec) == 0) goto fail; + sdata = strdup(spec); + + /* Count number of parameters + id and delimit */ + p=sdata; + for(count=0;;count++) { + char* q = strchr(p,','); + if(q == NULL) break; + *q++ = '\0'; + p = q; + } + count++; /* for final piece */ + + if(count == 0) + goto fail; /* no id and no parameters */ + + /* Extract the filter id */ + p = sdata; + sstat = sscanf(p,"%u",&id); + if(sstat != 1) goto fail; + /* skip past the filter id */ + p = p + strlen(p) + 1; + count--; + + /* Allocate the max needed space; *2 in case the params are all doubles */ + ulist = (unsigned int*)malloc(sizeof(unsigned int)*(count)*2); + if(ulist == NULL) goto fail; + + /* walk and convert */ + nparams = 0; /* actual count */ + for(i=0;i 2 => we might have a two letter tag */ + q = (p + len) - 2; + type = gettype(*q,*(q+1),&isunsigned); + break; + } + + /* Now parse */ + switch (type) { + case 'b': + case 's': + case 'i': + /* special case for a positive integer;for back compatibility.*/ + if(!isnegative) + sstat = sscanf(p,"%u",&val32u); + else + sstat = sscanf(p,"%d",(int*)&val32u); + if(sstat != 1) goto fail; + switch(type) { + case 'b': val32u = (val32u & 0xFF); break; + case 's': val32u = (val32u & 0xFFFF); break; + } + ulist[nparams++] = val32u; + break; + + case 'f': + sstat = sscanf(p,"%lf",&vald); + if(sstat != 1) goto fail; + valf = (float)vald; + ulist[nparams++] = *(unsigned int*)&valf; + break; + + /* The following are 8-byte values, so we must swap pieces if this + is a little endian machine */ + case 'd': + sstat = sscanf(p,"%lf",&vald); + if(sstat != 1) goto fail; + memcpy(mem,&vald,sizeof(mem)); + NC_filterfix8(mem,0); + vector = (unsigned int*)mem; + ulist[nparams++] = vector[0]; + ulist[nparams++] = vector[1]; + break; + case 'l': /* long long */ + if(isunsigned) + sstat = sscanf(p,"%llu",&val64u); + else + sstat = sscanf(p,"%lld",(long long*)&val64u); + if(sstat != 1) goto fail; + memcpy(mem,&val64u,sizeof(mem)); + NC_filterfix8(mem,0); + vector = (unsigned int*)&mem; + ulist[nparams++] = vector[0]; + ulist[nparams++] = vector[1]; + break; + default: + goto fail; + } + p = p + strlen(p) + 1; /* move to next param */ + } + /* Now return results */ + if(idp) *idp = id; + if(nparamsp) *nparamsp = nparams; + if(paramsp) { + *paramsp = ulist; + ulist = NULL; /* avoid duplicate free */ + } +done: + if(sdata) free(sdata); + if(ulist) free(ulist); + return stat; +fail: + stat = NC_EFILTER; + goto done; +} + +#ifdef WORDS_BIGENDIAN /* Byte swap an 8-byte integer in place */ static void byteswap8(unsigned char* mem) @@ -190,4 +436,38 @@ byteswap8(unsigned char* mem) mem[3] = mem[4]; mem[4] = c; } + +/* Byte swap an 8-byte integer in place */ +static void +byteswap4(unsigned char* mem) +{ + unsigned char c; + c = mem[0]; + mem[0] = mem[3]; + mem[3] = c; + c = mem[1]; + mem[1] = mem[2]; + mem[2] = c; +} + #endif + +static void +NC_filterfix8(unsigned char* mem, int decode) +{ +#ifdef WORDS_BIGENDIAN + if(decode) { /* Apply inverse of the encode case */ + byteswap4(mem); /* step 1: byte-swap each piece */ + byteswap4(mem+4); + byteswap8(mem); /* step 2: convert to little endian format */ + } else { /* encode */ + byteswap8(mem); /* step 1: convert to little endian format */ + byteswap4(mem); /* step 2: byte-swap each piece */ + byteswap4(mem+4); + } +#else /* Little endian */ + /* No action is necessary */ +#endif +} + +#endif /*0*/ diff --git a/ncdump/nccopy.c b/ncdump/nccopy.c index ec0e4b649..1608fb15e 100644 --- a/ncdump/nccopy.c +++ b/ncdump/nccopy.c @@ -17,12 +17,12 @@ #endif #include #include "netcdf.h" +#include "netcdf_filter.h" #include "nciter.h" #include "utils.h" #include "chunkspec.h" #include "dimmap.h" #include "nccomps.h" -#include "ncfilter.h" #include "list.h" #undef DEBUGFILTER diff --git a/ncdump/ncdump.c b/ncdump/ncdump.c index c13f41ab4..5489308d1 100644 --- a/ncdump/ncdump.c +++ b/ncdump/ncdump.c @@ -1072,6 +1072,7 @@ pr_att_specials( } printf("\" ;\n"); } + if(params) free(params); } { int no_fill = 0; diff --git a/ncgen/ncgen.y b/ncgen/ncgen.y index c119ce2dc..5170fc7a4 100644 --- a/ncgen/ncgen.y +++ b/ncgen/ncgen.y @@ -17,7 +17,7 @@ static char SccsId[] = "$Id: ncgen.y,v 1.42 2010/05/18 21:32:46 dmh Exp $"; #include "ncgeny.h" #include "ncgen.h" #ifdef USE_NETCDF4 -#include "ncfilter.h" +#include "netcdf_filter.h" #endif /* Following are in ncdump (for now)*/ diff --git a/ncgen/ncgenl.c b/ncgen/ncgenl.c index 43dfc31da..25ebfa29b 100644 --- a/ncgen/ncgenl.c +++ b/ncgen/ncgenl.c @@ -1,5 +1,5 @@ -#line 3 "ncgenl.c" +#line 2 "ncgenl.c" #define YY_INT_ALIGNED short int @@ -7,11 +7,17 @@ #define yy_create_buffer ncg_create_buffer #define yy_delete_buffer ncg_delete_buffer -#define yy_flex_debug ncg_flex_debug +#define yy_scan_buffer ncg_scan_buffer +#define yy_scan_string ncg_scan_string +#define yy_scan_bytes ncg_scan_bytes #define yy_init_buffer ncg_init_buffer #define yy_flush_buffer ncg_flush_buffer #define yy_load_buffer_state ncg_load_buffer_state #define yy_switch_to_buffer ncg_switch_to_buffer +#define yypush_buffer_state ncgpush_buffer_state +#define yypop_buffer_state ncgpop_buffer_state +#define yyensure_buffer_stack ncgensure_buffer_stack +#define yy_flex_debug ncg_flex_debug #define yyin ncgin #define yyleng ncgleng #define yylex ncglex @@ -27,11 +33,245 @@ #define FLEX_SCANNER #define YY_FLEX_MAJOR_VERSION 2 #define YY_FLEX_MINOR_VERSION 6 -#define YY_FLEX_SUBMINOR_VERSION 0 +#define YY_FLEX_SUBMINOR_VERSION 4 #if YY_FLEX_SUBMINOR_VERSION > 0 #define FLEX_BETA #endif +#ifdef yy_create_buffer +#define ncg_create_buffer_ALREADY_DEFINED +#else +#define yy_create_buffer ncg_create_buffer +#endif + +#ifdef yy_delete_buffer +#define ncg_delete_buffer_ALREADY_DEFINED +#else +#define yy_delete_buffer ncg_delete_buffer +#endif + +#ifdef yy_scan_buffer +#define ncg_scan_buffer_ALREADY_DEFINED +#else +#define yy_scan_buffer ncg_scan_buffer +#endif + +#ifdef yy_scan_string +#define ncg_scan_string_ALREADY_DEFINED +#else +#define yy_scan_string ncg_scan_string +#endif + +#ifdef yy_scan_bytes +#define ncg_scan_bytes_ALREADY_DEFINED +#else +#define yy_scan_bytes ncg_scan_bytes +#endif + +#ifdef yy_init_buffer +#define ncg_init_buffer_ALREADY_DEFINED +#else +#define yy_init_buffer ncg_init_buffer +#endif + +#ifdef yy_flush_buffer +#define ncg_flush_buffer_ALREADY_DEFINED +#else +#define yy_flush_buffer ncg_flush_buffer +#endif + +#ifdef yy_load_buffer_state +#define ncg_load_buffer_state_ALREADY_DEFINED +#else +#define yy_load_buffer_state ncg_load_buffer_state +#endif + +#ifdef yy_switch_to_buffer +#define ncg_switch_to_buffer_ALREADY_DEFINED +#else +#define yy_switch_to_buffer ncg_switch_to_buffer +#endif + +#ifdef yypush_buffer_state +#define ncgpush_buffer_state_ALREADY_DEFINED +#else +#define yypush_buffer_state ncgpush_buffer_state +#endif + +#ifdef yypop_buffer_state +#define ncgpop_buffer_state_ALREADY_DEFINED +#else +#define yypop_buffer_state ncgpop_buffer_state +#endif + +#ifdef yyensure_buffer_stack +#define ncgensure_buffer_stack_ALREADY_DEFINED +#else +#define yyensure_buffer_stack ncgensure_buffer_stack +#endif + +#ifdef yylex +#define ncglex_ALREADY_DEFINED +#else +#define yylex ncglex +#endif + +#ifdef yyrestart +#define ncgrestart_ALREADY_DEFINED +#else +#define yyrestart ncgrestart +#endif + +#ifdef yylex_init +#define ncglex_init_ALREADY_DEFINED +#else +#define yylex_init ncglex_init +#endif + +#ifdef yylex_init_extra +#define ncglex_init_extra_ALREADY_DEFINED +#else +#define yylex_init_extra ncglex_init_extra +#endif + +#ifdef yylex_destroy +#define ncglex_destroy_ALREADY_DEFINED +#else +#define yylex_destroy ncglex_destroy +#endif + +#ifdef yyget_debug +#define ncgget_debug_ALREADY_DEFINED +#else +#define yyget_debug ncgget_debug +#endif + +#ifdef yyset_debug +#define ncgset_debug_ALREADY_DEFINED +#else +#define yyset_debug ncgset_debug +#endif + +#ifdef yyget_extra +#define ncgget_extra_ALREADY_DEFINED +#else +#define yyget_extra ncgget_extra +#endif + +#ifdef yyset_extra +#define ncgset_extra_ALREADY_DEFINED +#else +#define yyset_extra ncgset_extra +#endif + +#ifdef yyget_in +#define ncgget_in_ALREADY_DEFINED +#else +#define yyget_in ncgget_in +#endif + +#ifdef yyset_in +#define ncgset_in_ALREADY_DEFINED +#else +#define yyset_in ncgset_in +#endif + +#ifdef yyget_out +#define ncgget_out_ALREADY_DEFINED +#else +#define yyget_out ncgget_out +#endif + +#ifdef yyset_out +#define ncgset_out_ALREADY_DEFINED +#else +#define yyset_out ncgset_out +#endif + +#ifdef yyget_leng +#define ncgget_leng_ALREADY_DEFINED +#else +#define yyget_leng ncgget_leng +#endif + +#ifdef yyget_text +#define ncgget_text_ALREADY_DEFINED +#else +#define yyget_text ncgget_text +#endif + +#ifdef yyget_lineno +#define ncgget_lineno_ALREADY_DEFINED +#else +#define yyget_lineno ncgget_lineno +#endif + +#ifdef yyset_lineno +#define ncgset_lineno_ALREADY_DEFINED +#else +#define yyset_lineno ncgset_lineno +#endif + +#ifdef yywrap +#define ncgwrap_ALREADY_DEFINED +#else +#define yywrap ncgwrap +#endif + +#ifdef yyalloc +#define ncgalloc_ALREADY_DEFINED +#else +#define yyalloc ncgalloc +#endif + +#ifdef yyrealloc +#define ncgrealloc_ALREADY_DEFINED +#else +#define yyrealloc ncgrealloc +#endif + +#ifdef yyfree +#define ncgfree_ALREADY_DEFINED +#else +#define yyfree ncgfree +#endif + +#ifdef yytext +#define ncgtext_ALREADY_DEFINED +#else +#define yytext ncgtext +#endif + +#ifdef yyleng +#define ncgleng_ALREADY_DEFINED +#else +#define yyleng ncgleng +#endif + +#ifdef yyin +#define ncgin_ALREADY_DEFINED +#else +#define yyin ncgin +#endif + +#ifdef yyout +#define ncgout_ALREADY_DEFINED +#else +#define yyout ncgout +#endif + +#ifdef yy_flex_debug +#define ncg_flex_debug_ALREADY_DEFINED +#else +#define yy_flex_debug ncg_flex_debug +#endif + +#ifdef yylineno +#define ncglineno_ALREADY_DEFINED +#else +#define yylineno ncglineno +#endif + /* First, we deal with platform-specific or compiler-specific issues. */ /* begin standard C headers. */ @@ -52,7 +292,7 @@ #if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 says to define __STDC_LIMIT_MACROS before including stdint.h, - * if you want the limit (max/min) macros for int types. + * if you want the limit (max/min) macros for int types. */ #ifndef __STDC_LIMIT_MACROS #define __STDC_LIMIT_MACROS 1 @@ -69,7 +309,7 @@ typedef uint32_t flex_uint32_t; typedef signed char flex_int8_t; typedef short int flex_int16_t; typedef int flex_int32_t; -typedef unsigned char flex_uint8_t; +typedef unsigned char flex_uint8_t; typedef unsigned short int flex_uint16_t; typedef unsigned int flex_uint32_t; @@ -102,60 +342,48 @@ typedef unsigned int flex_uint32_t; #define UINT32_MAX (4294967295U) #endif +#ifndef SIZE_MAX +#define SIZE_MAX (~(size_t)0) +#endif + #endif /* ! C99 */ #endif /* ! FLEXINT_H */ -#ifdef __cplusplus +/* begin standard C++ headers. */ -/* The "const" storage-class-modifier is valid. */ -#define YY_USE_CONST - -#else /* ! __cplusplus */ - -/* C99 requires __STDC__ to be defined as 1. */ -#if defined (__STDC__) - -#define YY_USE_CONST - -#endif /* defined (__STDC__) */ -#endif /* ! __cplusplus */ - -#ifdef YY_USE_CONST +/* TODO: this is always defined, so inline it */ #define yyconst const + +#if defined(__GNUC__) && __GNUC__ >= 3 +#define yynoreturn __attribute__((__noreturn__)) #else -#define yyconst +#define yynoreturn #endif /* Returned upon end-of-file. */ #define YY_NULL 0 -/* Promotes a possibly negative, possibly signed char to an unsigned - * integer for use as an array index. If the signed char is negative, - * we want to instead treat it as an 8-bit unsigned char, hence the - * double cast. +/* Promotes a possibly negative, possibly signed char to an + * integer in range [0..255] for use as an array index. */ -#define YY_SC_TO_UI(c) ((unsigned int) (unsigned char) c) +#define YY_SC_TO_UI(c) ((YY_CHAR) (c)) /* Enter a start condition. This macro really ought to take a parameter, * but we do it the disgusting crufty way forced on us by the ()-less * definition of BEGIN. */ #define BEGIN (yy_start) = 1 + 2 * - /* Translate the current start state into a value that can be later handed * to BEGIN to return to the state. The YYSTATE alias is for lex * compatibility. */ #define YY_START (((yy_start) - 1) / 2) #define YYSTATE YY_START - /* Action number for EOF rule of a given start state. */ #define YY_STATE_EOF(state) (YY_END_OF_BUFFER + state + 1) - /* Special action meaning "start processing a new file". */ -#define YY_NEW_FILE ncgrestart(ncgin ) - +#define YY_NEW_FILE yyrestart( yyin ) #define YY_END_OF_BUFFER_CHAR 0 /* Size of default input buffer. */ @@ -185,31 +413,30 @@ typedef struct yy_buffer_state *YY_BUFFER_STATE; typedef size_t yy_size_t; #endif -extern yy_size_t ncgleng; +extern int yyleng; -extern FILE *ncgin, *ncgout; +extern FILE *yyin, *yyout; #define EOB_ACT_CONTINUE_SCAN 0 #define EOB_ACT_END_OF_FILE 1 #define EOB_ACT_LAST_MATCH 2 - + #define YY_LESS_LINENO(n) #define YY_LINENO_REWIND_TO(ptr) - + /* Return all but the first "n" matched characters back to the input stream. */ #define yyless(n) \ do \ { \ - /* Undo effects of setting up ncgtext. */ \ + /* Undo effects of setting up yytext. */ \ int yyless_macro_arg = (n); \ YY_LESS_LINENO(yyless_macro_arg);\ *yy_cp = (yy_hold_char); \ YY_RESTORE_YY_MORE_OFFSET \ (yy_c_buf_p) = yy_cp = yy_bp + yyless_macro_arg - YY_MORE_ADJ; \ - YY_DO_BEFORE_ACTION; /* set up ncgtext again */ \ + YY_DO_BEFORE_ACTION; /* set up yytext again */ \ } \ while ( 0 ) - #define unput(c) yyunput( c, (yytext_ptr) ) #ifndef YY_STRUCT_YY_BUFFER_STATE @@ -224,7 +451,7 @@ struct yy_buffer_state /* Size of input buffer in bytes, not including room for EOB * characters. */ - yy_size_t yy_buf_size; + int yy_buf_size; /* Number of characters read into yy_ch_buf, not including EOB * characters. @@ -269,8 +496,8 @@ struct yy_buffer_state * possible backing-up. * * When we actually see the EOF, we change the status to "new" - * (via ncgrestart()), so that the user can continue scanning by - * just pointing ncgin at a new input file. + * (via yyrestart()), so that the user can continue scanning by + * just pointing yyin at a new input file. */ #define YY_BUFFER_EOF_PENDING 2 @@ -280,7 +507,7 @@ struct yy_buffer_state /* Stack of input buffers. */ static size_t yy_buffer_stack_top = 0; /**< index of top of stack. */ static size_t yy_buffer_stack_max = 0; /**< capacity of stack. */ -static YY_BUFFER_STATE * yy_buffer_stack = 0; /**< Stack as an array. */ +static YY_BUFFER_STATE * yy_buffer_stack = NULL; /**< Stack as an array. */ /* We provide macros for accessing buffer states in case in the * future we want to put the buffer states in a more general @@ -291,109 +518,98 @@ static YY_BUFFER_STATE * yy_buffer_stack = 0; /**< Stack as an array. */ #define YY_CURRENT_BUFFER ( (yy_buffer_stack) \ ? (yy_buffer_stack)[(yy_buffer_stack_top)] \ : NULL) - /* Same as previous macro, but useful when we know that the buffer stack is not * NULL or when we need an lvalue. For internal use only. */ #define YY_CURRENT_BUFFER_LVALUE (yy_buffer_stack)[(yy_buffer_stack_top)] -/* yy_hold_char holds the character lost when ncgtext is formed. */ +/* yy_hold_char holds the character lost when yytext is formed. */ static char yy_hold_char; static int yy_n_chars; /* number of characters read into yy_ch_buf */ -yy_size_t ncgleng; +int yyleng; /* Points to current character in buffer. */ -static char *yy_c_buf_p = (char *) 0; +static char *yy_c_buf_p = NULL; static int yy_init = 0; /* whether we need to initialize */ static int yy_start = 0; /* start state number */ -/* Flag which is used to allow ncgwrap()'s to do buffer switches - * instead of setting up a fresh ncgin. A bit of a hack ... +/* Flag which is used to allow yywrap()'s to do buffer switches + * instead of setting up a fresh yyin. A bit of a hack ... */ static int yy_did_buffer_switch_on_eof; -void ncgrestart (FILE *input_file ); -void ncg_switch_to_buffer (YY_BUFFER_STATE new_buffer ); -YY_BUFFER_STATE ncg_create_buffer (FILE *file,int size ); -void ncg_delete_buffer (YY_BUFFER_STATE b ); -void ncg_flush_buffer (YY_BUFFER_STATE b ); -void ncgpush_buffer_state (YY_BUFFER_STATE new_buffer ); -void ncgpop_buffer_state (void ); +void yyrestart ( FILE *input_file ); +void yy_switch_to_buffer ( YY_BUFFER_STATE new_buffer ); +YY_BUFFER_STATE yy_create_buffer ( FILE *file, int size ); +void yy_delete_buffer ( YY_BUFFER_STATE b ); +void yy_flush_buffer ( YY_BUFFER_STATE b ); +void yypush_buffer_state ( YY_BUFFER_STATE new_buffer ); +void yypop_buffer_state ( void ); -static void ncgensure_buffer_stack (void ); -static void ncg_load_buffer_state (void ); -static void ncg_init_buffer (YY_BUFFER_STATE b,FILE *file ); +static void yyensure_buffer_stack ( void ); +static void yy_load_buffer_state ( void ); +static void yy_init_buffer ( YY_BUFFER_STATE b, FILE *file ); +#define YY_FLUSH_BUFFER yy_flush_buffer( YY_CURRENT_BUFFER ) -#define YY_FLUSH_BUFFER ncg_flush_buffer(YY_CURRENT_BUFFER ) +YY_BUFFER_STATE yy_scan_buffer ( char *base, yy_size_t size ); +YY_BUFFER_STATE yy_scan_string ( const char *yy_str ); +YY_BUFFER_STATE yy_scan_bytes ( const char *bytes, int len ); -YY_BUFFER_STATE ncg_scan_buffer (char *base,yy_size_t size ); -YY_BUFFER_STATE ncg_scan_string (yyconst char *yy_str ); -YY_BUFFER_STATE ncg_scan_bytes (yyconst char *bytes,yy_size_t len ); - -void *ncgalloc (yy_size_t ); -void *ncgrealloc (void *,yy_size_t ); -void ncgfree (void * ); - -#define yy_new_buffer ncg_create_buffer +void *yyalloc ( yy_size_t ); +void *yyrealloc ( void *, yy_size_t ); +void yyfree ( void * ); +#define yy_new_buffer yy_create_buffer #define yy_set_interactive(is_interactive) \ { \ if ( ! YY_CURRENT_BUFFER ){ \ - ncgensure_buffer_stack (); \ + yyensure_buffer_stack (); \ YY_CURRENT_BUFFER_LVALUE = \ - ncg_create_buffer(ncgin,YY_BUF_SIZE ); \ + yy_create_buffer( yyin, YY_BUF_SIZE ); \ } \ YY_CURRENT_BUFFER_LVALUE->yy_is_interactive = is_interactive; \ } - #define yy_set_bol(at_bol) \ { \ if ( ! YY_CURRENT_BUFFER ){\ - ncgensure_buffer_stack (); \ + yyensure_buffer_stack (); \ YY_CURRENT_BUFFER_LVALUE = \ - ncg_create_buffer(ncgin,YY_BUF_SIZE ); \ + yy_create_buffer( yyin, YY_BUF_SIZE ); \ } \ YY_CURRENT_BUFFER_LVALUE->yy_at_bol = at_bol; \ } - #define YY_AT_BOL() (YY_CURRENT_BUFFER_LVALUE->yy_at_bol) /* Begin user sect3 */ +typedef flex_uint8_t YY_CHAR; -typedef unsigned char YY_CHAR; - -FILE *ncgin = (FILE *) 0, *ncgout = (FILE *) 0; +FILE *yyin = NULL, *yyout = NULL; typedef int yy_state_type; -extern int ncglineno; +extern int yylineno; +int yylineno = 1; -int ncglineno = 1; - -extern char *ncgtext; +extern char *yytext; #ifdef yytext_ptr #undef yytext_ptr #endif -#define yytext_ptr ncgtext +#define yytext_ptr yytext -static yy_state_type yy_get_previous_state (void ); -static yy_state_type yy_try_NUL_trans (yy_state_type current_state ); -static int yy_get_next_buffer (void ); -#if defined(__GNUC__) && __GNUC__ >= 3 -__attribute__((__noreturn__)) -#endif -static void yy_fatal_error (yyconst char msg[] ); +static yy_state_type yy_get_previous_state ( void ); +static yy_state_type yy_try_NUL_trans ( yy_state_type current_state ); +static int yy_get_next_buffer ( void ); +static void yynoreturn yy_fatal_error ( const char* msg ); /* Done after the current pattern has been matched and before the - * corresponding action - sets up ncgtext. + * corresponding action - sets up yytext. */ #define YY_DO_BEFORE_ACTION \ (yytext_ptr) = yy_bp; \ - ncgleng = (size_t) (yy_cp - yy_bp); \ + yyleng = (int) (yy_cp - yy_bp); \ (yy_hold_char) = *yy_cp; \ *yy_cp = '\0'; \ (yy_c_buf_p) = yy_cp; - #define YY_NUM_RULES 49 #define YY_END_OF_BUFFER 50 /* This struct is not used in this scanner, @@ -403,7 +619,7 @@ struct yy_trans_info flex_int32_t yy_verify; flex_int32_t yy_nxt; }; -static yyconst flex_int16_t yy_accept[422] = +static const flex_int16_t yy_accept[422] = { 0, 0, 0, 46, 46, 0, 0, 50, 48, 1, 44, 48, 48, 48, 48, 38, 32, 36, 36, 35, 35, @@ -454,7 +670,7 @@ static yyconst flex_int16_t yy_accept[422] = 0 } ; -static yyconst YY_CHAR yy_ec[256] = +static const YY_CHAR yy_ec[256] = { 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 3, 1, 2, 2, 1, 1, 1, 1, 1, 1, 1, @@ -486,7 +702,7 @@ static yyconst YY_CHAR yy_ec[256] = 1, 1, 1, 1, 1 } ; -static yyconst YY_CHAR yy_meta[69] = +static const YY_CHAR yy_meta[69] = { 0, 1, 1, 2, 1, 1, 1, 3, 4, 5, 5, 6, 7, 8, 8, 8, 8, 8, 8, 8, 1, @@ -497,7 +713,7 @@ static yyconst YY_CHAR yy_meta[69] = 11, 11, 11, 14, 1, 11, 11, 11 } ; -static yyconst flex_uint16_t yy_base[440] = +static const flex_int16_t yy_base[440] = { 0, 0, 0, 325, 321, 264, 255, 318, 2387, 67, 2387, 64, 269, 61, 62, 95, 77, 136, 259, 51, 61, @@ -549,7 +765,7 @@ static yyconst flex_uint16_t yy_base[440] = 2315, 2329, 2339, 2345, 2353, 2355, 2361, 2367, 2373 } ; -static yyconst flex_int16_t yy_def[440] = +static const flex_int16_t yy_def[440] = { 0, 421, 1, 422, 422, 423, 423, 421, 421, 421, 421, 424, 425, 421, 426, 421, 427, 421, 17, 428, 428, @@ -601,7 +817,7 @@ static yyconst flex_int16_t yy_def[440] = 421, 421, 421, 421, 421, 421, 421, 421, 421 } ; -static yyconst flex_uint16_t yy_nxt[2456] = +static const flex_int16_t yy_nxt[2456] = { 0, 8, 9, 10, 9, 8, 11, 12, 8, 13, 14, 15, 16, 17, 18, 18, 18, 18, 18, 18, 8, @@ -875,7 +1091,7 @@ static yyconst flex_uint16_t yy_nxt[2456] = 421, 421, 421, 421, 421 } ; -static yyconst flex_int16_t yy_chk[2456] = +static const flex_int16_t yy_chk[2456] = { 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, @@ -1152,8 +1368,8 @@ static yyconst flex_int16_t yy_chk[2456] = static yy_state_type yy_last_accepting_state; static char *yy_last_accepting_cpos; -extern int ncg_flex_debug; -int ncg_flex_debug = 0; +extern int yy_flex_debug; +int yy_flex_debug = 0; /* The intent behind this definition is that it'll catch * any uses of REJECT which flex missed. @@ -1162,11 +1378,11 @@ int ncg_flex_debug = 0; #define yymore() yymore_used_but_not_detected #define YY_MORE_ADJ 0 #define YY_RESTORE_YY_MORE_OFFSET -char *ncgtext; +char *yytext; #line 1 "ncgen.l" #line 2 "ncgen.l" /********************************************************************* - * Copyright 2018, UCAR/Unidata + * Copyright 1993, UCAR/Unidata * See netcdf/COPYRIGHT file for copying and redistribution conditions. * $Id: ncgen.l,v 1.24 2009/09/25 18:22:35 dmh Exp $ *********************************************************************/ @@ -1304,7 +1520,7 @@ struct Specialtoken specials[] = { {NULL,0} /* null terminate */ }; - +#line 1523 "ncgenl.c" /* The most correct (validating) version of UTF8 character set (Taken from: http://www.w3.org/2005/03/23-lex-U) @@ -1347,7 +1563,7 @@ ID ([A-Za-z_]|{UTF8})([A-Z.@#\[\]a-z_0-9+-]|{UTF8})* /* Note: this definition of string will work for utf8 as well, although it is a very relaxed definition */ -#line 1351 "ncgenl.c" +#line 1566 "ncgenl.c" #define INITIAL 0 #define ST_C_COMMENT 1 @@ -1365,36 +1581,36 @@ ID ([A-Za-z_]|{UTF8})([A-Z.@#\[\]a-z_0-9+-]|{UTF8})* #define YY_EXTRA_TYPE void * #endif -static int yy_init_globals (void ); +static int yy_init_globals ( void ); /* Accessor methods to globals. These are made visible to non-reentrant scanners for convenience. */ -int ncglex_destroy (void ); +int yylex_destroy ( void ); -int ncgget_debug (void ); +int yyget_debug ( void ); -void ncgset_debug (int debug_flag ); +void yyset_debug ( int debug_flag ); -YY_EXTRA_TYPE ncgget_extra (void ); +YY_EXTRA_TYPE yyget_extra ( void ); -void ncgset_extra (YY_EXTRA_TYPE user_defined ); +void yyset_extra ( YY_EXTRA_TYPE user_defined ); -FILE *ncgget_in (void ); +FILE *yyget_in ( void ); -void ncgset_in (FILE * _in_str ); +void yyset_in ( FILE * _in_str ); -FILE *ncgget_out (void ); +FILE *yyget_out ( void ); -void ncgset_out (FILE * _out_str ); +void yyset_out ( FILE * _out_str ); -yy_size_t ncgget_leng (void ); + int yyget_leng ( void ); -char *ncgget_text (void ); +char *yyget_text ( void ); -int ncgget_lineno (void ); +int yyget_lineno ( void ); -void ncgset_lineno (int _line_number ); +void yyset_lineno ( int _line_number ); /* Macros after this point can all be overridden by user definitions in * section 1. @@ -1402,32 +1618,31 @@ void ncgset_lineno (int _line_number ); #ifndef YY_SKIP_YYWRAP #ifdef __cplusplus -extern "C" int ncgwrap (void ); +extern "C" int yywrap ( void ); #else -extern int ncgwrap (void ); +extern int yywrap ( void ); #endif #endif #ifndef YY_NO_UNPUT - - static void yyunput (int c,char *buf_ptr ); - + + static void yyunput ( int c, char *buf_ptr ); + #endif #ifndef yytext_ptr -static void yy_flex_strncpy (char *,yyconst char *,int ); +static void yy_flex_strncpy ( char *, const char *, int ); #endif #ifdef YY_NEED_STRLEN -static int yy_flex_strlen (yyconst char * ); +static int yy_flex_strlen ( const char * ); #endif #ifndef YY_NO_INPUT - #ifdef __cplusplus -static int yyinput (void ); +static int yyinput ( void ); #else -static int input (void ); +static int input ( void ); #endif #endif @@ -1447,7 +1662,7 @@ static int input (void ); /* This used to be an fputs(), but since the string might contain NUL's, * we now use fwrite(). */ -#define ECHO do { if (fwrite( ncgtext, ncgleng, 1, ncgout )) {} } while (0) +#define ECHO do { if (fwrite( yytext, (size_t) yyleng, 1, yyout )) {} } while (0) #endif /* Gets input and stuffs it into "buf". number of characters read, or YY_NULL, @@ -1458,20 +1673,20 @@ static int input (void ); if ( YY_CURRENT_BUFFER_LVALUE->yy_is_interactive ) \ { \ int c = '*'; \ - size_t n; \ + int n; \ for ( n = 0; n < max_size && \ - (c = getc( ncgin )) != EOF && c != '\n'; ++n ) \ + (c = getc( yyin )) != EOF && c != '\n'; ++n ) \ buf[n] = (char) c; \ if ( c == '\n' ) \ buf[n++] = (char) c; \ - if ( c == EOF && ferror( ncgin ) ) \ + if ( c == EOF && ferror( yyin ) ) \ YY_FATAL_ERROR( "input in flex scanner failed" ); \ result = n; \ } \ else \ { \ errno=0; \ - while ( (result = fread(buf, 1, max_size, ncgin))==0 && ferror(ncgin)) \ + while ( (result = (int) fread(buf, 1, (yy_size_t) max_size, yyin)) == 0 && ferror(yyin)) \ { \ if( errno != EINTR) \ { \ @@ -1479,7 +1694,7 @@ static int input (void ); break; \ } \ errno=0; \ - clearerr(ncgin); \ + clearerr(yyin); \ } \ }\ \ @@ -1512,12 +1727,12 @@ static int input (void ); #ifndef YY_DECL #define YY_DECL_IS_OURS 1 -extern int ncglex (void); +extern int yylex (void); -#define YY_DECL int ncglex (void) +#define YY_DECL int yylex (void) #endif /* !YY_DECL */ -/* Code executed at the beginning of each rule, after ncgtext and ncgleng +/* Code executed at the beginning of each rule, after yytext and yyleng * have been set up. */ #ifndef YY_USER_ACTION @@ -1539,7 +1754,7 @@ YY_DECL yy_state_type yy_current_state; char *yy_cp, *yy_bp; int yy_act; - + if ( !(yy_init) ) { (yy_init) = 1; @@ -1551,31 +1766,31 @@ YY_DECL if ( ! (yy_start) ) (yy_start) = 1; /* first start state */ - if ( ! ncgin ) - ncgin = stdin; + if ( ! yyin ) + yyin = stdin; - if ( ! ncgout ) - ncgout = stdout; + if ( ! yyout ) + yyout = stdout; if ( ! YY_CURRENT_BUFFER ) { - ncgensure_buffer_stack (); + yyensure_buffer_stack (); YY_CURRENT_BUFFER_LVALUE = - ncg_create_buffer(ncgin,YY_BUF_SIZE ); + yy_create_buffer( yyin, YY_BUF_SIZE ); } - ncg_load_buffer_state( ); + yy_load_buffer_state( ); } { #line 220 "ncgen.l" -#line 1573 "ncgenl.c" +#line 1787 "ncgenl.c" while ( /*CONSTCOND*/1 ) /* loops until end-of-file is reached */ { yy_cp = (yy_c_buf_p); - /* Support of ncgtext. */ + /* Support of yytext. */ *yy_cp = (yy_hold_char); /* yy_bp points to the position in yy_ch_buf of the start of @@ -1597,9 +1812,9 @@ yy_match: { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 422 ) - yy_c = yy_meta[(unsigned int) yy_c]; + yy_c = yy_meta[yy_c]; } - yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int) yy_c]; + yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; ++yy_cp; } while ( yy_base[yy_current_state] != 2387 ); @@ -1648,14 +1863,14 @@ YY_RULE_SETUP /* In netcdf4, this will be used in a variety of places, so only remove escapes */ /* -if(ncgleng > MAXTRST) { +if(yyleng > MAXTRST) { yyerror("string too long, truncated\n"); -ncgtext[MAXTRST-1] = '\0'; +yytext[MAXTRST-1] = '\0'; } */ - len = unescape((char *)ncgtext+1,ncgleng-2,!ISIDENT,&s); + len = unescape((char *)yytext+1,yyleng-2,!ISIDENT,&s); if(len < 0) { - sprintf(errstr,"Illegal character: %s",ncgtext); + sprintf(errstr,"Illegal character: %s",yytext); yyerror(errstr); } bbClear(lextext); @@ -1669,8 +1884,8 @@ case 4: YY_RULE_SETUP #line 250 "ncgen.l" { /* drop leading 0x; pad to even number of chars */ - char* p = ncgtext+2; - int len = ncgleng - 2; + char* p = yytext+2; + int len = yyleng - 2; bbClear(lextext); bbAppendn(lextext,p,len); if((len % 2) == 1) bbAppend(lextext,'0'); @@ -1795,7 +2010,7 @@ case 27: YY_RULE_SETUP #line 290 "ncgen.l" { /* missing value (pre-2.4 backward compatibility) */ - if (ncgtext[0] == '-') { + if (yytext[0] == '-') { double_val = NEGNC_INFINITE; } else { double_val = NC_INFINITE; @@ -1817,7 +2032,7 @@ case 29: YY_RULE_SETUP #line 305 "ncgen.l" {/* missing value (pre-2.4 backward compatibility)*/ - if (ncgtext[0] == '-') { + if (yytext[0] == '-') { float_val = NEGNC_INFINITEF; } else { float_val = NC_INFINITEF; @@ -1853,7 +2068,7 @@ YY_RULE_SETUP #line 330 "ncgen.l" { bbClear(lextext); - bbAppendn(lextext,(char*)ncgtext,ncgleng+1); /* include null */ + bbAppendn(lextext,(char*)yytext,yyleng+1); /* include null */ bbNull(lextext); yylval.sym = makepath(bbContents(lextext)); return lexdebug(PATH); @@ -1864,7 +2079,7 @@ YY_RULE_SETUP #line 339 "ncgen.l" {struct Specialtoken* st; bbClear(lextext); - bbAppendn(lextext,(char*)ncgtext,ncgleng+1); /* include null */ + bbAppendn(lextext,(char*)yytext,yyleng+1); /* include null */ bbNull(lextext); for(st=specials;st->name;st++) { if(strcmp(bbContents(lextext),st->name)==0) {return lexdebug(st->token);} @@ -1881,7 +2096,7 @@ YY_RULE_SETUP char* p; char* q; /* copy the trimmed name */ bbClear(lextext); - bbAppendn(lextext,(char*)ncgtext,ncgleng+1); /* include null */ + bbAppendn(lextext,(char*)yytext,yyleng+1); /* include null */ bbNull(lextext); p = bbContents(lextext); q = p; @@ -1897,8 +2112,8 @@ case 35: YY_RULE_SETUP #line 366 "ncgen.l" { char* id = NULL; int len; - len = strlen(ncgtext); - len = unescape(ncgtext,len,ISIDENT,&id); + len = strlen(yytext); + len = unescape(yytext,len,ISIDENT,&id); if(NCSTREQ(id, FILL_STRING)) { efree(id); return lexdebug(FILLMARKER); @@ -1982,21 +2197,21 @@ YY_RULE_SETUP { int c; int token = 0; - int slen = strlen(ncgtext); + int slen = strlen(yytext); char* stag = NULL; int tag = NC_NAT; - char* hex = ncgtext+2; /* point to first true hex digit */ + char* hex = yytext+2; /* point to first true hex digit */ int xlen = (slen - 3); /* true hex length */ - ncgtext[slen-1] = '\0'; + yytext[slen-1] = '\0'; /* capture the tag string */ - tag = collecttag(ncgtext,&stag); + tag = collecttag(yytext,&stag); if(tag == NC_NAT) { sprintf(errstr,"Illegal integer suffix: %s",stag); yyerror(errstr); goto done; } - ncgtext[slen - strlen(stag)] = '\0'; + yytext[slen - strlen(stag)] = '\0'; if(xlen > 16) { /* truncate hi order digits */ hex += (xlen - 16); } @@ -2018,8 +2233,8 @@ YY_RULE_SETUP token = UINT64_CONST; break; default: /* should never happen */ - if (sscanf((char*)ncgtext, "%i", &uint32_val) != 1) { - sprintf(errstr,"bad unsigned int constant: %s",(char*)ncgtext); + if (sscanf((char*)yytext, "%i", &uint32_val) != 1) { + sprintf(errstr,"bad unsigned int constant: %s",(char*)yytext); yyerror(errstr); } token = UINT_CONST; @@ -2031,8 +2246,8 @@ case 38: YY_RULE_SETUP #line 490 "ncgen.l" { - if (sscanf((char*)ncgtext, "%le", &double_val) != 1) { - sprintf(errstr,"bad long or double constant: %s",(char*)ncgtext); + if (sscanf((char*)yytext, "%le", &double_val) != 1) { + sprintf(errstr,"bad long or double constant: %s",(char*)yytext); yyerror(errstr); } return lexdebug(DOUBLE_CONST); @@ -2042,8 +2257,8 @@ case 39: YY_RULE_SETUP #line 497 "ncgen.l" { - if (sscanf((char*)ncgtext, "%e", &float_val) != 1) { - sprintf(errstr,"bad float constant: %s",(char*)ncgtext); + if (sscanf((char*)yytext, "%e", &float_val) != 1) { + sprintf(errstr,"bad float constant: %s",(char*)yytext); yyerror(errstr); } return lexdebug(FLOAT_CONST); @@ -2054,7 +2269,7 @@ case 40: YY_RULE_SETUP #line 504 "ncgen.l" { - (void) sscanf((char*)&ncgtext[1],"%c",&byte_val); + (void) sscanf((char*)&yytext[1],"%c",&byte_val); return lexdebug(BYTE_CONST); } YY_BREAK @@ -2062,9 +2277,9 @@ case 41: YY_RULE_SETUP #line 508 "ncgen.l" { - int oct = unescapeoct(&ncgtext[2]); + int oct = unescapeoct(&yytext[2]); if(oct < 0) { - sprintf(errstr,"bad octal character constant: %s",(char*)ncgtext); + sprintf(errstr,"bad octal character constant: %s",(char*)yytext); yyerror(errstr); } byte_val = (unsigned int)oct; @@ -2075,9 +2290,9 @@ case 42: YY_RULE_SETUP #line 517 "ncgen.l" { - int hex = unescapehex(&ncgtext[3]); + int hex = unescapehex(&yytext[3]); if(byte_val < 0) { - sprintf(errstr,"bad hex character constant: %s",(char*)ncgtext); + sprintf(errstr,"bad hex character constant: %s",(char*)yytext); yyerror(errstr); } byte_val = (unsigned int)hex; @@ -2088,7 +2303,7 @@ case 43: YY_RULE_SETUP #line 526 "ncgen.l" { - switch ((char)ncgtext[2]) { + switch ((char)yytext[2]) { case 'a': byte_val = '\007'; break; /* not everyone under- * stands '\a' yet */ case 'b': byte_val = '\b'; break; @@ -2100,7 +2315,7 @@ YY_RULE_SETUP case '\\': byte_val = '\\'; break; case '?': byte_val = '\177'; break; case '\'': byte_val = '\''; break; - default: byte_val = (char)ncgtext[2]; + default: byte_val = (char)yytext[2]; } return lexdebug(BYTE_CONST); } @@ -2150,7 +2365,7 @@ case 48: YY_RULE_SETUP #line 569 "ncgen.l" {/* Note: this next rule will not work for UTF8 characters */ - return lexdebug(ncgtext[0]) ; + return lexdebug(yytext[0]) ; } YY_BREAK case 49: @@ -2158,7 +2373,7 @@ YY_RULE_SETUP #line 572 "ncgen.l" ECHO; YY_BREAK -#line 2162 "ncgenl.c" +#line 2376 "ncgenl.c" case YY_STATE_EOF(INITIAL): case YY_STATE_EOF(TEXT): yyterminate(); @@ -2176,15 +2391,15 @@ case YY_STATE_EOF(TEXT): { /* We're scanning a new file or input source. It's * possible that this happened because the user - * just pointed ncgin at a new source and called - * ncglex(). If so, then we have to assure + * just pointed yyin at a new source and called + * yylex(). If so, then we have to assure * consistency between YY_CURRENT_BUFFER and our * globals. Here is the right place to do so, because * this is the first action (other than possibly a * back-up) that will match for the new input source. */ (yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; - YY_CURRENT_BUFFER_LVALUE->yy_input_file = ncgin; + YY_CURRENT_BUFFER_LVALUE->yy_input_file = yyin; YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = YY_BUFFER_NORMAL; } @@ -2237,11 +2452,11 @@ case YY_STATE_EOF(TEXT): { (yy_did_buffer_switch_on_eof) = 0; - if ( ncgwrap( ) ) + if ( yywrap( ) ) { /* Note: because we've taken care in * yy_get_next_buffer() to have set up - * ncgtext, we can now set up + * yytext, we can now set up * yy_c_buf_p so that if some total * hoser (like flex itself) wants to * call the scanner after we return the @@ -2291,7 +2506,7 @@ case YY_STATE_EOF(TEXT): } /* end of action switch */ } /* end of scanning one token */ } /* end of user's declarations */ -} /* end of ncglex */ +} /* end of yylex */ /* yy_get_next_buffer - try to read in a new buffer * @@ -2304,7 +2519,7 @@ static int yy_get_next_buffer (void) { char *dest = YY_CURRENT_BUFFER_LVALUE->yy_ch_buf; char *source = (yytext_ptr); - yy_size_t number_to_move, i; + int number_to_move, i; int ret_val; if ( (yy_c_buf_p) > &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[(yy_n_chars) + 1] ) @@ -2333,7 +2548,7 @@ static int yy_get_next_buffer (void) /* Try to read more data. */ /* First move last chars to start of buffer. */ - number_to_move = (yy_size_t) ((yy_c_buf_p) - (yytext_ptr)) - 1; + number_to_move = (int) ((yy_c_buf_p) - (yytext_ptr) - 1); for ( i = 0; i < number_to_move; ++i ) *(dest++) = *(source++); @@ -2346,7 +2561,7 @@ static int yy_get_next_buffer (void) else { - yy_size_t num_to_read = + int num_to_read = YY_CURRENT_BUFFER_LVALUE->yy_buf_size - number_to_move - 1; while ( num_to_read <= 0 ) @@ -2360,7 +2575,7 @@ static int yy_get_next_buffer (void) if ( b->yy_is_our_buffer ) { - yy_size_t new_size = b->yy_buf_size * 2; + int new_size = b->yy_buf_size * 2; if ( new_size <= 0 ) b->yy_buf_size += b->yy_buf_size / 8; @@ -2369,11 +2584,12 @@ static int yy_get_next_buffer (void) b->yy_ch_buf = (char *) /* Include room in for 2 EOB chars. */ - ncgrealloc((void *) b->yy_ch_buf,b->yy_buf_size + 2 ); + yyrealloc( (void *) b->yy_ch_buf, + (yy_size_t) (b->yy_buf_size + 2) ); } else /* Can't grow it, we don't own it. */ - b->yy_ch_buf = 0; + b->yy_ch_buf = NULL; if ( ! b->yy_ch_buf ) YY_FATAL_ERROR( @@ -2401,7 +2617,7 @@ static int yy_get_next_buffer (void) if ( number_to_move == YY_MORE_ADJ ) { ret_val = EOB_ACT_END_OF_FILE; - ncgrestart(ncgin ); + yyrestart( yyin ); } else @@ -2415,12 +2631,15 @@ static int yy_get_next_buffer (void) else ret_val = EOB_ACT_CONTINUE_SCAN; - if ((int) ((yy_n_chars) + number_to_move) > YY_CURRENT_BUFFER_LVALUE->yy_buf_size) { + if (((yy_n_chars) + number_to_move) > YY_CURRENT_BUFFER_LVALUE->yy_buf_size) { /* Extend the array by 50%, plus the number we really need. */ int new_size = (yy_n_chars) + number_to_move + ((yy_n_chars) >> 1); - YY_CURRENT_BUFFER_LVALUE->yy_ch_buf = (char *) ncgrealloc((void *) YY_CURRENT_BUFFER_LVALUE->yy_ch_buf,new_size ); + YY_CURRENT_BUFFER_LVALUE->yy_ch_buf = (char *) yyrealloc( + (void *) YY_CURRENT_BUFFER_LVALUE->yy_ch_buf, (yy_size_t) new_size ); if ( ! YY_CURRENT_BUFFER_LVALUE->yy_ch_buf ) YY_FATAL_ERROR( "out of dynamic memory in yy_get_next_buffer()" ); + /* "- 2" to take care of EOB's */ + YY_CURRENT_BUFFER_LVALUE->yy_buf_size = (int) (new_size - 2); } (yy_n_chars) += number_to_move; @@ -2438,7 +2657,7 @@ static int yy_get_next_buffer (void) { yy_state_type yy_current_state; char *yy_cp; - + yy_current_state = (yy_start); for ( yy_cp = (yytext_ptr) + YY_MORE_ADJ; yy_cp < (yy_c_buf_p); ++yy_cp ) @@ -2453,9 +2672,9 @@ static int yy_get_next_buffer (void) { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 422 ) - yy_c = yy_meta[(unsigned int) yy_c]; + yy_c = yy_meta[yy_c]; } - yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int) yy_c]; + yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; } return yy_current_state; @@ -2481,9 +2700,9 @@ static int yy_get_next_buffer (void) { yy_current_state = (int) yy_def[yy_current_state]; if ( yy_current_state >= 422 ) - yy_c = yy_meta[(unsigned int) yy_c]; + yy_c = yy_meta[yy_c]; } - yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int) yy_c]; + yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; yy_is_jam = (yy_current_state == 421); return yy_is_jam ? 0 : yy_current_state; @@ -2494,16 +2713,16 @@ static int yy_get_next_buffer (void) static void yyunput (int c, char * yy_bp ) { char *yy_cp; - + yy_cp = (yy_c_buf_p); - /* undo effects of setting up ncgtext */ + /* undo effects of setting up yytext */ *yy_cp = (yy_hold_char); if ( yy_cp < YY_CURRENT_BUFFER_LVALUE->yy_ch_buf + 2 ) { /* need to shift things up to make room */ /* +2 for EOB chars. */ - yy_size_t number_to_move = (yy_n_chars) + 2; + int number_to_move = (yy_n_chars) + 2; char *dest = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[ YY_CURRENT_BUFFER_LVALUE->yy_buf_size + 2]; char *source = @@ -2515,7 +2734,7 @@ static int yy_get_next_buffer (void) yy_cp += (int) (dest - source); yy_bp += (int) (dest - source); YY_CURRENT_BUFFER_LVALUE->yy_n_chars = - (yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_buf_size; + (yy_n_chars) = (int) YY_CURRENT_BUFFER_LVALUE->yy_buf_size; if ( yy_cp < YY_CURRENT_BUFFER_LVALUE->yy_ch_buf + 2 ) YY_FATAL_ERROR( "flex scanner push-back overflow" ); @@ -2539,7 +2758,7 @@ static int yy_get_next_buffer (void) { int c; - + *(yy_c_buf_p) = (yy_hold_char); if ( *(yy_c_buf_p) == YY_END_OF_BUFFER_CHAR ) @@ -2554,7 +2773,7 @@ static int yy_get_next_buffer (void) else { /* need more input */ - yy_size_t offset = (yy_c_buf_p) - (yytext_ptr); + int offset = (int) ((yy_c_buf_p) - (yytext_ptr)); ++(yy_c_buf_p); switch ( yy_get_next_buffer( ) ) @@ -2571,14 +2790,14 @@ static int yy_get_next_buffer (void) */ /* Reset buffer status. */ - ncgrestart(ncgin ); + yyrestart( yyin ); /*FALLTHROUGH*/ case EOB_ACT_END_OF_FILE: { - if ( ncgwrap( ) ) - return EOF; + if ( yywrap( ) ) + return 0; if ( ! (yy_did_buffer_switch_on_eof) ) YY_NEW_FILE; @@ -2597,7 +2816,7 @@ static int yy_get_next_buffer (void) } c = *(unsigned char *) (yy_c_buf_p); /* cast for 8-bit char's */ - *(yy_c_buf_p) = '\0'; /* preserve ncgtext */ + *(yy_c_buf_p) = '\0'; /* preserve yytext */ (yy_hold_char) = *++(yy_c_buf_p); return c; @@ -2606,35 +2825,35 @@ static int yy_get_next_buffer (void) /** Immediately switch to a different input stream. * @param input_file A readable stream. - * + * * @note This function does not reset the start condition to @c INITIAL . */ - void ncgrestart (FILE * input_file ) + void yyrestart (FILE * input_file ) { - + if ( ! YY_CURRENT_BUFFER ){ - ncgensure_buffer_stack (); + yyensure_buffer_stack (); YY_CURRENT_BUFFER_LVALUE = - ncg_create_buffer(ncgin,YY_BUF_SIZE ); + yy_create_buffer( yyin, YY_BUF_SIZE ); } - ncg_init_buffer(YY_CURRENT_BUFFER,input_file ); - ncg_load_buffer_state( ); + yy_init_buffer( YY_CURRENT_BUFFER, input_file ); + yy_load_buffer_state( ); } /** Switch to a different input buffer. * @param new_buffer The new input buffer. - * + * */ - void ncg_switch_to_buffer (YY_BUFFER_STATE new_buffer ) + void yy_switch_to_buffer (YY_BUFFER_STATE new_buffer ) { - + /* TODO. We should be able to replace this entire function body * with - * ncgpop_buffer_state(); - * ncgpush_buffer_state(new_buffer); + * yypop_buffer_state(); + * yypush_buffer_state(new_buffer); */ - ncgensure_buffer_stack (); + yyensure_buffer_stack (); if ( YY_CURRENT_BUFFER == new_buffer ) return; @@ -2647,61 +2866,61 @@ static int yy_get_next_buffer (void) } YY_CURRENT_BUFFER_LVALUE = new_buffer; - ncg_load_buffer_state( ); + yy_load_buffer_state( ); /* We don't actually know whether we did this switch during - * EOF (ncgwrap()) processing, but the only time this flag - * is looked at is after ncgwrap() is called, so it's safe + * EOF (yywrap()) processing, but the only time this flag + * is looked at is after yywrap() is called, so it's safe * to go ahead and always set it. */ (yy_did_buffer_switch_on_eof) = 1; } -static void ncg_load_buffer_state (void) +static void yy_load_buffer_state (void) { (yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; (yytext_ptr) = (yy_c_buf_p) = YY_CURRENT_BUFFER_LVALUE->yy_buf_pos; - ncgin = YY_CURRENT_BUFFER_LVALUE->yy_input_file; + yyin = YY_CURRENT_BUFFER_LVALUE->yy_input_file; (yy_hold_char) = *(yy_c_buf_p); } /** Allocate and initialize an input buffer state. * @param file A readable stream. * @param size The character buffer size in bytes. When in doubt, use @c YY_BUF_SIZE. - * + * * @return the allocated buffer state. */ - YY_BUFFER_STATE ncg_create_buffer (FILE * file, int size ) + YY_BUFFER_STATE yy_create_buffer (FILE * file, int size ) { YY_BUFFER_STATE b; - - b = (YY_BUFFER_STATE) ncgalloc(sizeof( struct yy_buffer_state ) ); + + b = (YY_BUFFER_STATE) yyalloc( sizeof( struct yy_buffer_state ) ); if ( ! b ) - YY_FATAL_ERROR( "out of dynamic memory in ncg_create_buffer()" ); + YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); - b->yy_buf_size = (yy_size_t)size; + b->yy_buf_size = size; /* yy_ch_buf has to be 2 characters longer than the size given because * we need to put in 2 end-of-buffer characters. */ - b->yy_ch_buf = (char *) ncgalloc(b->yy_buf_size + 2 ); + b->yy_ch_buf = (char *) yyalloc( (yy_size_t) (b->yy_buf_size + 2) ); if ( ! b->yy_ch_buf ) - YY_FATAL_ERROR( "out of dynamic memory in ncg_create_buffer()" ); + YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); b->yy_is_our_buffer = 1; - ncg_init_buffer(b,file ); + yy_init_buffer( b, file ); return b; } /** Destroy the buffer. - * @param b a buffer created with ncg_create_buffer() - * + * @param b a buffer created with yy_create_buffer() + * */ - void ncg_delete_buffer (YY_BUFFER_STATE b ) + void yy_delete_buffer (YY_BUFFER_STATE b ) { - + if ( ! b ) return; @@ -2709,27 +2928,27 @@ static void ncg_load_buffer_state (void) YY_CURRENT_BUFFER_LVALUE = (YY_BUFFER_STATE) 0; if ( b->yy_is_our_buffer ) - ncgfree((void *) b->yy_ch_buf ); + yyfree( (void *) b->yy_ch_buf ); - ncgfree((void *) b ); + yyfree( (void *) b ); } /* Initializes or reinitializes a buffer. * This function is sometimes called more than once on the same buffer, - * such as during a ncgrestart() or at EOF. + * such as during a yyrestart() or at EOF. */ - static void ncg_init_buffer (YY_BUFFER_STATE b, FILE * file ) + static void yy_init_buffer (YY_BUFFER_STATE b, FILE * file ) { int oerrno = errno; - - ncg_flush_buffer(b ); + + yy_flush_buffer( b ); b->yy_input_file = file; b->yy_fill_buffer = 1; - /* If b is the current buffer, then ncg_init_buffer was _probably_ - * called from ncgrestart() or through yy_get_next_buffer. + /* If b is the current buffer, then yy_init_buffer was _probably_ + * called from yyrestart() or through yy_get_next_buffer. * In that case, we don't want to reset the lineno or column. */ if (b != YY_CURRENT_BUFFER){ @@ -2738,15 +2957,15 @@ static void ncg_load_buffer_state (void) } b->yy_is_interactive = file ? (isatty( fileno(file) ) > 0) : 0; - + errno = oerrno; } /** Discard all buffered characters. On the next scan, YY_INPUT will be called. * @param b the buffer state to be flushed, usually @c YY_CURRENT_BUFFER. - * + * */ - void ncg_flush_buffer (YY_BUFFER_STATE b ) + void yy_flush_buffer (YY_BUFFER_STATE b ) { if ( ! b ) return; @@ -2766,23 +2985,23 @@ static void ncg_load_buffer_state (void) b->yy_buffer_status = YY_BUFFER_NEW; if ( b == YY_CURRENT_BUFFER ) - ncg_load_buffer_state( ); + yy_load_buffer_state( ); } /** Pushes the new state onto the stack. The new state becomes * the current state. This function will allocate the stack * if necessary. * @param new_buffer The new state. - * + * */ -void ncgpush_buffer_state (YY_BUFFER_STATE new_buffer ) +void yypush_buffer_state (YY_BUFFER_STATE new_buffer ) { if (new_buffer == NULL) return; - ncgensure_buffer_stack(); + yyensure_buffer_stack(); - /* This block is copied from ncg_switch_to_buffer. */ + /* This block is copied from yy_switch_to_buffer. */ if ( YY_CURRENT_BUFFER ) { /* Flush out information for old buffer. */ @@ -2796,27 +3015,27 @@ void ncgpush_buffer_state (YY_BUFFER_STATE new_buffer ) (yy_buffer_stack_top)++; YY_CURRENT_BUFFER_LVALUE = new_buffer; - /* copied from ncg_switch_to_buffer. */ - ncg_load_buffer_state( ); + /* copied from yy_switch_to_buffer. */ + yy_load_buffer_state( ); (yy_did_buffer_switch_on_eof) = 1; } /** Removes and deletes the top of the stack, if present. * The next element becomes the new top. - * + * */ -void ncgpop_buffer_state (void) +void yypop_buffer_state (void) { if (!YY_CURRENT_BUFFER) return; - ncg_delete_buffer(YY_CURRENT_BUFFER ); + yy_delete_buffer(YY_CURRENT_BUFFER ); YY_CURRENT_BUFFER_LVALUE = NULL; if ((yy_buffer_stack_top) > 0) --(yy_buffer_stack_top); if (YY_CURRENT_BUFFER) { - ncg_load_buffer_state( ); + yy_load_buffer_state( ); (yy_did_buffer_switch_on_eof) = 1; } } @@ -2824,22 +3043,22 @@ void ncgpop_buffer_state (void) /* Allocates the stack if it does not exist. * Guarantees space for at least one push. */ -static void ncgensure_buffer_stack (void) +static void yyensure_buffer_stack (void) { yy_size_t num_to_alloc; - + if (!(yy_buffer_stack)) { /* First allocation is just for 2 elements, since we don't know if this * scanner will even need a stack. We use 2 instead of 1 to avoid an * immediate realloc on the next call. */ - num_to_alloc = 1; /* After all that talk, this was set to 1 anyways... */ - (yy_buffer_stack) = (struct yy_buffer_state**)ncgalloc + num_to_alloc = 1; /* After all that talk, this was set to 1 anyways... */ + (yy_buffer_stack) = (struct yy_buffer_state**)yyalloc (num_to_alloc * sizeof(struct yy_buffer_state*) ); if ( ! (yy_buffer_stack) ) - YY_FATAL_ERROR( "out of dynamic memory in ncgensure_buffer_stack()" ); + YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); memset((yy_buffer_stack), 0, num_to_alloc * sizeof(struct yy_buffer_state*)); @@ -2854,12 +3073,12 @@ static void ncgensure_buffer_stack (void) yy_size_t grow_size = 8 /* arbitrary grow size */; num_to_alloc = (yy_buffer_stack_max) + grow_size; - (yy_buffer_stack) = (struct yy_buffer_state**)ncgrealloc + (yy_buffer_stack) = (struct yy_buffer_state**)yyrealloc ((yy_buffer_stack), num_to_alloc * sizeof(struct yy_buffer_state*) ); if ( ! (yy_buffer_stack) ) - YY_FATAL_ERROR( "out of dynamic memory in ncgensure_buffer_stack()" ); + YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); /* zero only the new slots.*/ memset((yy_buffer_stack) + (yy_buffer_stack_max), 0, grow_size * sizeof(struct yy_buffer_state*)); @@ -2870,80 +3089,80 @@ static void ncgensure_buffer_stack (void) /** Setup the input buffer state to scan directly from a user-specified character buffer. * @param base the character buffer * @param size the size in bytes of the character buffer - * + * * @return the newly allocated buffer state object. */ -YY_BUFFER_STATE ncg_scan_buffer (char * base, yy_size_t size ) +YY_BUFFER_STATE yy_scan_buffer (char * base, yy_size_t size ) { YY_BUFFER_STATE b; - + if ( size < 2 || base[size-2] != YY_END_OF_BUFFER_CHAR || base[size-1] != YY_END_OF_BUFFER_CHAR ) /* They forgot to leave room for the EOB's. */ - return 0; + return NULL; - b = (YY_BUFFER_STATE) ncgalloc(sizeof( struct yy_buffer_state ) ); + b = (YY_BUFFER_STATE) yyalloc( sizeof( struct yy_buffer_state ) ); if ( ! b ) - YY_FATAL_ERROR( "out of dynamic memory in ncg_scan_buffer()" ); + YY_FATAL_ERROR( "out of dynamic memory in yy_scan_buffer()" ); - b->yy_buf_size = size - 2; /* "- 2" to take care of EOB's */ + b->yy_buf_size = (int) (size - 2); /* "- 2" to take care of EOB's */ b->yy_buf_pos = b->yy_ch_buf = base; b->yy_is_our_buffer = 0; - b->yy_input_file = 0; + b->yy_input_file = NULL; b->yy_n_chars = b->yy_buf_size; b->yy_is_interactive = 0; b->yy_at_bol = 1; b->yy_fill_buffer = 0; b->yy_buffer_status = YY_BUFFER_NEW; - ncg_switch_to_buffer(b ); + yy_switch_to_buffer( b ); return b; } -/** Setup the input buffer state to scan a string. The next call to ncglex() will +/** Setup the input buffer state to scan a string. The next call to yylex() will * scan from a @e copy of @a str. * @param yystr a NUL-terminated string to scan - * + * * @return the newly allocated buffer state object. * @note If you want to scan bytes that may contain NUL values, then use - * ncg_scan_bytes() instead. + * yy_scan_bytes() instead. */ -YY_BUFFER_STATE ncg_scan_string (yyconst char * yystr ) +YY_BUFFER_STATE yy_scan_string (const char * yystr ) { - - return ncg_scan_bytes(yystr,strlen(yystr) ); + + return yy_scan_bytes( yystr, (int) strlen(yystr) ); } -/** Setup the input buffer state to scan the given bytes. The next call to ncglex() will +/** Setup the input buffer state to scan the given bytes. The next call to yylex() will * scan from a @e copy of @a bytes. * @param yybytes the byte buffer to scan * @param _yybytes_len the number of bytes in the buffer pointed to by @a bytes. - * + * * @return the newly allocated buffer state object. */ -YY_BUFFER_STATE ncg_scan_bytes (yyconst char * yybytes, yy_size_t _yybytes_len ) +YY_BUFFER_STATE yy_scan_bytes (const char * yybytes, int _yybytes_len ) { YY_BUFFER_STATE b; char *buf; yy_size_t n; - yy_size_t i; - + int i; + /* Get memory for full buffer, including space for trailing EOB's. */ - n = _yybytes_len + 2; - buf = (char *) ncgalloc(n ); + n = (yy_size_t) (_yybytes_len + 2); + buf = (char *) yyalloc( n ); if ( ! buf ) - YY_FATAL_ERROR( "out of dynamic memory in ncg_scan_bytes()" ); + YY_FATAL_ERROR( "out of dynamic memory in yy_scan_bytes()" ); for ( i = 0; i < _yybytes_len; ++i ) buf[i] = yybytes[i]; buf[_yybytes_len] = buf[_yybytes_len+1] = YY_END_OF_BUFFER_CHAR; - b = ncg_scan_buffer(buf,n ); + b = yy_scan_buffer( buf, n ); if ( ! b ) - YY_FATAL_ERROR( "bad buffer in ncg_scan_bytes()" ); + YY_FATAL_ERROR( "bad buffer in yy_scan_bytes()" ); /* It's okay to grow etc. this buffer, and we should throw it * away when we're done. @@ -2957,9 +3176,9 @@ YY_BUFFER_STATE ncg_scan_bytes (yyconst char * yybytes, yy_size_t _yybytes_len #define YY_EXIT_FAILURE 2 #endif -static void yy_fatal_error (yyconst char* msg ) +static void yynoreturn yy_fatal_error (const char* msg ) { - (void) fprintf( stderr, "%s\n", msg ); + fprintf( stderr, "%s\n", msg ); exit( YY_EXIT_FAILURE ); } @@ -2969,142 +3188,142 @@ static void yy_fatal_error (yyconst char* msg ) #define yyless(n) \ do \ { \ - /* Undo effects of setting up ncgtext. */ \ + /* Undo effects of setting up yytext. */ \ int yyless_macro_arg = (n); \ YY_LESS_LINENO(yyless_macro_arg);\ - ncgtext[ncgleng] = (yy_hold_char); \ - (yy_c_buf_p) = ncgtext + yyless_macro_arg; \ + yytext[yyleng] = (yy_hold_char); \ + (yy_c_buf_p) = yytext + yyless_macro_arg; \ (yy_hold_char) = *(yy_c_buf_p); \ *(yy_c_buf_p) = '\0'; \ - ncgleng = yyless_macro_arg; \ + yyleng = yyless_macro_arg; \ } \ while ( 0 ) /* Accessor methods (get/set functions) to struct members. */ /** Get the current line number. - * + * */ -int ncgget_lineno (void) +int yyget_lineno (void) { - - return ncglineno; + + return yylineno; } /** Get the input stream. - * + * */ -FILE *ncgget_in (void) +FILE *yyget_in (void) { - return ncgin; + return yyin; } /** Get the output stream. - * + * */ -FILE *ncgget_out (void) +FILE *yyget_out (void) { - return ncgout; + return yyout; } /** Get the length of the current token. - * + * */ -yy_size_t ncgget_leng (void) +int yyget_leng (void) { - return ncgleng; + return yyleng; } /** Get the current token. - * + * */ -char *ncgget_text (void) +char *yyget_text (void) { - return ncgtext; + return yytext; } /** Set the current line number. * @param _line_number line number - * + * */ -void ncgset_lineno (int _line_number ) +void yyset_lineno (int _line_number ) { - - ncglineno = _line_number; + + yylineno = _line_number; } /** Set the input stream. This does not discard the current * input buffer. * @param _in_str A readable stream. - * - * @see ncg_switch_to_buffer + * + * @see yy_switch_to_buffer */ -void ncgset_in (FILE * _in_str ) +void yyset_in (FILE * _in_str ) { - ncgin = _in_str ; + yyin = _in_str ; } -void ncgset_out (FILE * _out_str ) +void yyset_out (FILE * _out_str ) { - ncgout = _out_str ; + yyout = _out_str ; } -int ncgget_debug (void) +int yyget_debug (void) { - return ncg_flex_debug; + return yy_flex_debug; } -void ncgset_debug (int _bdebug ) +void yyset_debug (int _bdebug ) { - ncg_flex_debug = _bdebug ; + yy_flex_debug = _bdebug ; } static int yy_init_globals (void) { /* Initialization is the same as for the non-reentrant scanner. - * This function is called from ncglex_destroy(), so don't allocate here. + * This function is called from yylex_destroy(), so don't allocate here. */ - (yy_buffer_stack) = 0; + (yy_buffer_stack) = NULL; (yy_buffer_stack_top) = 0; (yy_buffer_stack_max) = 0; - (yy_c_buf_p) = (char *) 0; + (yy_c_buf_p) = NULL; (yy_init) = 0; (yy_start) = 0; /* Defined in main.c */ #ifdef YY_STDINIT - ncgin = stdin; - ncgout = stdout; + yyin = stdin; + yyout = stdout; #else - ncgin = (FILE *) 0; - ncgout = (FILE *) 0; + yyin = NULL; + yyout = NULL; #endif /* For future reference: Set errno on error, since we are called by - * ncglex_init() + * yylex_init() */ return 0; } -/* ncglex_destroy is for both reentrant and non-reentrant scanners. */ -int ncglex_destroy (void) +/* yylex_destroy is for both reentrant and non-reentrant scanners. */ +int yylex_destroy (void) { - + /* Pop the buffer stack, destroying each element. */ while(YY_CURRENT_BUFFER){ - ncg_delete_buffer(YY_CURRENT_BUFFER ); + yy_delete_buffer( YY_CURRENT_BUFFER ); YY_CURRENT_BUFFER_LVALUE = NULL; - ncgpop_buffer_state(); + yypop_buffer_state(); } /* Destroy the stack itself. */ - ncgfree((yy_buffer_stack) ); + yyfree((yy_buffer_stack) ); (yy_buffer_stack) = NULL; /* Reset the globals. This is important in a non-reentrant scanner so the next time - * ncglex() is called, initialization will occur. */ + * yylex() is called, initialization will occur. */ yy_init_globals( ); return 0; @@ -3115,9 +3334,9 @@ int ncglex_destroy (void) */ #ifndef yytext_ptr -static void yy_flex_strncpy (char* s1, yyconst char * s2, int n ) +static void yy_flex_strncpy (char* s1, const char * s2, int n ) { - + int i; for ( i = 0; i < n; ++i ) s1[i] = s2[i]; @@ -3125,7 +3344,7 @@ static void yy_flex_strncpy (char* s1, yyconst char * s2, int n ) #endif #ifdef YY_NEED_STRLEN -static int yy_flex_strlen (yyconst char * s ) +static int yy_flex_strlen (const char * s ) { int n; for ( n = 0; s[n]; ++n ) @@ -3135,14 +3354,14 @@ static int yy_flex_strlen (yyconst char * s ) } #endif -void *ncgalloc (yy_size_t size ) +void *yyalloc (yy_size_t size ) { - return (void *) malloc( size ); + return malloc(size); } -void *ncgrealloc (void * ptr, yy_size_t size ) +void *yyrealloc (void * ptr, yy_size_t size ) { - + /* The cast to (char *) in the following accommodates both * implementations that use char* generic pointers, and those * that use void* generic pointers. It works with the latter @@ -3150,27 +3369,25 @@ void *ncgrealloc (void * ptr, yy_size_t size ) * any pointer type to void*, and deal with argument conversions * as though doing an assignment. */ - return (void *) realloc( (char *) ptr, size ); + return realloc(ptr, size); } -void ncgfree (void * ptr ) +void yyfree (void * ptr ) { - free( (char *) ptr ); /* see ncgrealloc() for (char *) cast */ + free( (char *) ptr ); /* see yyrealloc() for (char *) cast */ } #define YYTABLES_NAME "yytables" #line 572 "ncgen.l" - - static int lexdebug(int token) { if(debug >= 2) { - char* text = ncgtext; - text[ncgleng] = 0; + char* text = yytext; + text[yyleng] = 0; fprintf(stderr,"Token=%d |%s| line=%d\n",token,text,lineno); } return token; @@ -3453,3 +3670,4 @@ collecttag(char* text, char** stagp) } return tag; } + diff --git a/ncgen/ncgeny.c b/ncgen/ncgeny.c index ff628f87d..a532f05f5 100644 --- a/ncgen/ncgeny.c +++ b/ncgen/ncgeny.c @@ -80,7 +80,7 @@ static char SccsId[] = "$Id: ncgen.y,v 1.42 2010/05/18 21:32:46 dmh Exp $"; #include "ncgeny.h" #include "ncgen.h" #ifdef USE_NETCDF4 -#include "ncfilter.h" +#include "netcdf_filter.h" #endif /* Following are in ncdump (for now)*/ @@ -2981,7 +2981,7 @@ makeprimitivetype(nc_type nctype) sym->typ.typecode = nctype; sym->typ.size = ncsize(nctype); sym->typ.nelems = 1; - sym->typ.alignment = ncaux_class_alignment(nctype); + sym->typ.alignment = ncaux_class_alignment(nctype); /* Make the basetype circular so we can always ask for it */ sym->typ.basetype = sym; sym->prefix = listnew(); @@ -3284,7 +3284,7 @@ makespecial(int tag, Symbol* vsym, Symbol* tsym, void* data, int isconst) else if(tag == _NCPROPS_FLAG) { globalspecials._NCProperties = sdata; sdata = NULL; - } + } } else { Specialdata* special; /* Set up special info */ @@ -3361,7 +3361,7 @@ makespecial(int tag, Symbol* vsym, Symbol* tsym, void* data, int isconst) break; case _CHUNKSIZES_FLAG: { int i; - list = (isconst ? const2list(con) : list); + list = (isconst ? const2list(con) : list); special->nchunks = list->length; special->_ChunkSizes = (size_t*)ecalloc(sizeof(size_t)*special->nchunks); for(i=0;inchunks;i++) { diff --git a/oc2/occompile.c b/oc2/occompile.c index c22c78bb3..9274f293c 100644 --- a/oc2/occompile.c +++ b/oc2/occompile.c @@ -108,7 +108,7 @@ occompile1(OCstate* state, OCnode* xnode, XXDR* xxdrs, OCdata** datap) if(!xxdr_uint(xxdrs,&xdrcount)) {ocstat = OC_EXDR; goto fail;} if(xdrcount != nelements) - {ocstat=OC_EINVALCOORDS; goto fail;} + {ocstat=OCTHROW(OC_EINVALCOORDS); goto fail;} /* allocate space to capture all the element instances */ data->instances = (OCdata**)malloc(nelements*sizeof(OCdata*)); @@ -160,7 +160,7 @@ occompile1(OCstate* state, OCnode* xnode, XXDR* xxdrs, OCdata** datap) break; /* we are done with the this sequence instance*/ } else { nclog(NCLOGERR,"missing/invalid begin/end record marker\n"); - ocstat = OC_EINVALCOORDS; + ocstat = OCTHROW(OC_EINVALCOORDS); goto fail; } } @@ -301,11 +301,11 @@ occompileatomic(OCstate* state, OCdata* data, XXDR* xxdrs) nelements = octotaldimsize(xnode->array.rank,xnode->array.sizes); /* Get first copy of the dimension count */ if(!xxdr_uint(xxdrs,&xxdrcount)) {ocstat = OC_EXDR; goto fail;} - if(xxdrcount != nelements) {ocstat=OC_EINVALCOORDS; goto fail;} + if(xxdrcount != nelements) {ocstat=OCTHROW(OC_EINVALCOORDS); goto fail;} if(xnode->etype != OC_String && xnode->etype != OC_URL) { /* Get second copy of the dimension count */ if(!xxdr_uint(xxdrs,&xxdrcount)) {ocstat = OC_EXDR; goto fail;} - if(xxdrcount != nelements) {ocstat=OC_EINVALCOORDS; goto fail;} + if(xxdrcount != nelements) {ocstat=OCTHROW(OC_EINVALCOORDS); goto fail;} } } else { /*scalar*/ nelements = 1; diff --git a/plugins/H5Zmisc.c b/plugins/H5Zmisc.c index 8b7d262d2..a4b335262 100644 --- a/plugins/H5Zmisc.c +++ b/plugins/H5Zmisc.c @@ -6,6 +6,7 @@ #include /* Older versions of the hdf library may define H5PL_type_t here */ #include +#include "h5misc.h" #ifndef DLL_EXPORT #define DLL_EXPORT @@ -27,8 +28,6 @@ will generate an error. */ -#include "h5misc.h" - #undef DEBUG /* The C standard apparently defines all floating point constants as double; @@ -37,9 +36,10 @@ will generate an error. #define DBLVAL 12345678.12345678 static int paramcheck(size_t nparams, const unsigned int* params); -static void byteswap8(unsigned char* mem); static void mismatch(size_t i, const char* which); +extern void NC_filterfix8(void* mem, int decode); + const H5Z_class2_t H5Z_TEST[1] = {{ H5Z_CLASS_T_VERS, /* H5Z_class_t version */ (H5Z_filter_t)(H5Z_FILTER_TEST), /* Filter id number */ @@ -138,12 +138,10 @@ static int paramcheck(size_t nparams, const unsigned int* params) { size_t i; - /* Test endianness of this machine */ - const unsigned char b[4] = {0x0,0x0,0x0,0x1}; /* value 1 in big-endian*/ - int bigendian = (1 == *(unsigned int*)b); /* 1=>big 0=>little*/ + unsigned char mem[8]; if(nparams != 14) { - fprintf(stderr,"Too few parameters: need=16 sent=%ld\n",(unsigned long)nparams); + fprintf(stderr,"Too few parameters: need=14 sent=%ld\n",(unsigned long)nparams); goto fail; } @@ -190,33 +188,37 @@ paramcheck(size_t nparams, const unsigned int* params) {mismatch(i,"float"); goto fail; }; break; case 8: {/*double*/ - double x = *(double*)¶ms[i]; + double x; + memcpy(mem,¶ms[i],sizeof(mem)); + NC_filterfix8(mem,1); /* Fix up endianness */ + x = *(double*)mem; dval = DBLVAL; i++; /* takes two parameters */ - if(bigendian) - byteswap8((unsigned char*)&x); if(dval != x) { mismatch(i,"double"); goto fail; } }; break; case 10: {/*signed long long*/ - signed long long x = *(signed long long*)¶ms[i]; + signed long long x; + memcpy(mem,¶ms[i],sizeof(mem)); + NC_filterfix8(mem,1); /* Fix up endianness */ + x = *(signed long long*)mem; + NC_filterfix8(&x,1); /* Fix up endianness */ lval = -9223372036854775807L; i++; /* takes two parameters */ - if(bigendian) - byteswap8((unsigned char*)&x); if(lval != x) { mismatch(i,"signed long long"); goto fail; } }; break; case 12: {/*unsigned long long*/ - unsigned long long x = *(unsigned long long*)¶ms[i]; + unsigned long long x; + memcpy(mem,¶ms[i],sizeof(mem)); + NC_filterfix8(mem,1); /* Fix up endianness */ + x = *(unsigned long long*)mem; lval = 18446744073709551615UL; i++; /* takes two parameters */ - if(bigendian) - byteswap8((unsigned char*)&x); if(lval != x) { mismatch(i,"unsigned long long"); goto fail; @@ -245,24 +247,6 @@ fail: return 0; } -static void -byteswap8(unsigned char* mem) -{ - unsigned char c; - c = mem[0]; - mem[0] = mem[7]; - mem[7] = c; - c = mem[1]; - mem[1] = mem[6]; - mem[6] = c; - c = mem[2]; - mem[2] = mem[5]; - mem[5] = c; - c = mem[3]; - mem[3] = mem[4]; - mem[4] = c; -} - static void mismatch(size_t i, const char* which) { diff --git a/plugins/H5Zutil.c b/plugins/H5Zutil.c new file mode 100644 index 000000000..3ec890f4c --- /dev/null +++ b/plugins/H5Zutil.c @@ -0,0 +1,65 @@ +/* + * Copyright 2018, University Corporation for Atmospheric Research + * See netcdf/COPYRIGHT file for copying and redistribution conditions. + */ + + +#include + +/* +Common utilities related to filters. +Taken from libdispatch/dfilters.c. +*/ + +#ifdef WORDS_BIGENDIAN +/* Byte swap an 8-byte integer in place */ +static void +byteswap8(unsigned char* mem) +{ + unsigned char c; + c = mem[0]; + mem[0] = mem[7]; + mem[7] = c; + c = mem[1]; + mem[1] = mem[6]; + mem[6] = c; + c = mem[2]; + mem[2] = mem[5]; + mem[5] = c; + c = mem[3]; + mem[3] = mem[4]; + mem[4] = c; +} + +/* Byte swap an 8-byte integer in place */ +static void +byteswap4(unsigned char* mem) +{ + unsigned char c; + c = mem[0]; + mem[0] = mem[3]; + mem[3] = c; + c = mem[1]; + mem[1] = mem[2]; + mem[2] = c; +} +#endif /*WORDS_BIGENDIAN*/ + +void +NC_filterfix8(void* mem0, int decode) +{ +#ifdef WORDS_BIGENDIAN + unsigned char* mem = mem0; + if(decode) { /* Apply inverse of the encode case */ + byteswap4(mem); /* step 1: byte-swap each piece */ + byteswap4(mem+4); + byteswap8(mem); /* step 2: convert to little endian format */ + } else { /* encode */ + byteswap8(mem); /* step 1: convert to little endian format */ + byteswap4(mem); /* step 2: byte-swap each piece */ + byteswap4(mem+4); + } +#else /* Little endian */ + /* No action is necessary */ +#endif +} diff --git a/plugins/Makefile.am b/plugins/Makefile.am index 709a81485..4c8380eb7 100644 --- a/plugins/Makefile.am +++ b/plugins/Makefile.am @@ -8,7 +8,7 @@ PLUGINSRC=H5Zbzip2.c PLUGINHDRS=h5bzip2.h EXTRA_DIST=${PLUGINSRC} ${BZIP2SRC} ${PLUGINHDRS} ${BZIP2HDRS} \ - H5Ztemplate.c H5Zmisc.c CMakeLists.txt + H5Ztemplate.c H5Zmisc.c H5Zutil.c CMakeLists.txt # WARNING: This list must be kept consistent with the corresponding # AC_CONFIG_LINK commands near the end of configure.ac. @@ -21,7 +21,7 @@ lib_LTLIBRARIES = libbzip2.la libmisc.la libbzip2_la_SOURCES = ${HDF5PLUGINSRC} libbzip2_la_LDFLAGS = -module -avoid-version -shared -export-dynamic -no-undefined -libmisc_la_SOURCES = H5Zmisc.c h5misc.h +libmisc_la_SOURCES = H5Zmisc.c H5Zutil.c h5misc.h libmisc_la_LDFLAGS = -module -avoid-version -shared -export-dynamic -no-undefined -rpath ${abs_builddir} endif #ENABLE_FILTER_TESTING