netcdf-c/libdap4/d4swap.c

260 lines
7.0 KiB
C
Raw Normal View History

Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
/*********************************************************************
2018-12-07 05:24:28 +08:00
* Copyright 2018, UCAR/Unidata
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
* See netcdf/COPYRIGHT file for copying and redistribution conditions.
*********************************************************************/
#include "d4includes.h"
#include <stdarg.h>
#include "d4includes.h"
/*
2018-04-24 06:38:08 +08:00
The primary purpose of this code is to recursively traverse
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
the incoming data to get the endianness correct.
*/
/* Forward */
static int walkAtomicVar(NCD4meta*, NCD4node*, NCD4node*, void** offsetp);
static int walkOpaqueVar(NCD4meta*,NCD4node*, NCD4node*, void** offsetp);
static int walkStructArray(NCD4meta*,NCD4node*, NCD4node*, void** offsetp);
static int walkStruct(NCD4meta*, NCD4node*, NCD4node*, void** offsetp);
static int walkSeqArray(NCD4meta*, NCD4node*, NCD4node*, void** offsetp);
static int walkSeq(NCD4meta*,NCD4node*, NCD4node*, void** offsetp);
/**************************************************/
/*
Assumes that compiler->swap is true; does necessary
byte swapping.
*/
int
NCD4_swapdata(NCD4meta* compiler, NClist* topvars)
{
int ret = NC_NOERR;
int i;
void* offset;
offset = compiler->serial.dap;
for(i=0;i<nclistlength(topvars);i++) {
NCD4node* var = (NCD4node*)nclistget(topvars,i);
var->data.dap4data.memory = offset;
switch (var->subsort) {
default:
if((ret=walkAtomicVar(compiler,var,var,&offset))) goto done;
break;
case NC_OPAQUE:
/* The only thing we need to do is swap the counts */
if((ret=walkOpaqueVar(compiler,var,var,&offset))) goto done;
break;
case NC_STRUCT:
if((ret=walkStructArray(compiler,var,var,&offset))) goto done;
break;
case NC_SEQ:
if((ret=walkSeqArray(compiler,var,var,&offset))) goto done;
break;
}
var->data.dap4data.size = DELTA(offset,var->data.dap4data.memory);
/* skip checksum, if there is one */
if(var->data.remotechecksummed)
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
offset = INCR(offset,CHECKSUMSIZE);
}
done:
return THROW(ret);
}
static int
walkAtomicVar(NCD4meta* compiler, NCD4node* topvar, NCD4node* var, void** offsetp)
{
int ret = NC_NOERR;
void* offset;
d4size_t i;
nc_type subsort;
d4size_t dimproduct;
NCD4node* basetype;
basetype = (var->sort == NCD4_TYPE ? var : var->basetype);
subsort = basetype->subsort;
dimproduct = (var->sort == NCD4_TYPE ? 1 : NCD4_dimproduct(var));
offset = *offsetp;
if(subsort == NC_ENUM)
subsort = var->basetype->basetype->subsort;
/* Only need to swap multi-byte integers and floats */
if(subsort != NC_STRING) {
int typesize = NCD4_typesize(subsort);
d4size_t totalsize = typesize*dimproduct;
if(typesize == 1) {
offset = INCR(offset,totalsize);
} else { /*(typesize > 1)*/
for(i=0;i<dimproduct;i++) {
char* sp = (char*)offset;
if(compiler->swap) {
switch (typesize) {
case 2: swapinline16(sp); break;
case 4: swapinline32(sp); break;
case 8: swapinline64(sp); break;
default: break;
}
}
offset = INCR(offset,typesize);
}
}
} else if(subsort == NC_STRING) { /* remaining case; just convert the counts */
COUNTERTYPE count;
for(i=0;i<dimproduct;i++) {
/* Get string count */
if(compiler->swap)
swapinline64(offset);
count = GETCOUNTER(offset);
SKIPCOUNTER(offset);
/* skip count bytes */
offset = INCR(offset,count);
}
}
*offsetp = offset;
return THROW(ret);
}
static int
walkOpaqueVar(NCD4meta* compiler, NCD4node* topvar, NCD4node* var, void** offsetp)
{
int ret = NC_NOERR;
void* offset;
d4size_t i;
unsigned long long count;
d4size_t dimproduct = NCD4_dimproduct(var);
dimproduct = (var->sort == NCD4_TYPE ? 1 : NCD4_dimproduct(var));
offset = *offsetp;
for(i=0;i<dimproduct;i++) {
/* Get and swap opaque count */
if(compiler->swap)
swapinline64(offset);
count = GETCOUNTER(offset);
SKIPCOUNTER(offset);
offset = INCR(offset,count);
}
*offsetp = offset;
return THROW(ret);
}
static int
walkStructArray(NCD4meta* compiler, NCD4node* topvar, NCD4node* var, void** offsetp)
{
int ret = NC_NOERR;
void* offset;
d4size_t i;
d4size_t dimproduct = NCD4_dimproduct(var);
NCD4node* basetype = var->basetype;
offset = *offsetp;
for(i=0;i<dimproduct;i++) {
/* Swap, possibly recursively, the single struct pointed to by offset*/
if((ret=walkStruct(compiler,topvar,basetype,&offset))) goto done;
}
*offsetp = offset;
done:
return THROW(ret);
}
static int
walkStruct(NCD4meta* compiler, NCD4node* topvar, NCD4node* structtype, void** offsetp)
{
int ret = NC_NOERR;
int i;
void* offset;
offset = *offsetp;
for(i=0;i<nclistlength(structtype->vars);i++) {
NCD4node* field = (NCD4node*)nclistget(structtype->vars,i);
NCD4node* fieldbase = field->basetype;
switch (fieldbase->subsort) {
default:
if((ret=walkAtomicVar(compiler,topvar,field,&offset))) goto done;
break;
case NC_OPAQUE:
/* The only thing we need to do is swap the counts */
if((ret=walkOpaqueVar(compiler,topvar,field,&offset))) goto done;
break;
case NC_STRUCT:
if((ret=walkStructArray(compiler,topvar,field,&offset))) goto done;
break;
case NC_SEQ:
if((ret=walkSeqArray(compiler,topvar,field,&offset))) goto done;
break;
}
}
*offsetp = offset;
done:
return THROW(ret);
}
static int
walkSeqArray(NCD4meta* compiler, NCD4node* topvar, NCD4node* var, void** offsetp)
{
int ret = NC_NOERR;
void* offset;
d4size_t i;
d4size_t dimproduct;
NCD4node* seqtype;
assert(var->sort == NCD4_VAR);
dimproduct = NCD4_dimproduct(var);
seqtype = var->basetype;
offset = *offsetp;
for(i=0;i<dimproduct;i++) {
/* Swap, possibly recursively, the single seq pointed to by offset*/
if((ret=walkSeq(compiler,topvar,seqtype,&offset))) goto done;
}
*offsetp = offset;
done:
return THROW(ret);
}
/*
Remember that the base type of var is a vlen.
*/
static int
walkSeq(NCD4meta* compiler, NCD4node* topvar, NCD4node* vlentype, void** offsetp)
{
int ret = NC_NOERR;
int i;
void* offset;
d4size_t recordcount;
NCD4node* basetype;
offset = *offsetp;
/* process the record count */
if(compiler->swap)
swapinline64(offset);
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
recordcount = GETCOUNTER(offset);
SKIPCOUNTER(offset);
basetype = vlentype->basetype; /* This may be of any type potentially */
2017-11-04 19:34:36 +08:00
assert(basetype->sort == NCD4_TYPE);
Primary change: add dap4 support Specific changes: 1. Add dap4 code: libdap4 and dap4_test. Note that until the d4ts server problem is solved, dap4 is turned off. 2. Modify various files to support dap4 flags: configure.ac, Makefile.am, CMakeLists.txt, etc. 3. Add nc_test/test_common.sh. This centralizes the handling of the locations of various things in the build tree: e.g. where is ncgen.exe located. See nc_test/test_common.sh for details. 4. Modify .sh files to use test_common.sh 5. Obsolete separate oc2 by moving it to be part of netcdf-c. This means replacing code with netcdf-c equivalents. 5. Add --with-testserver to configure.ac to allow override of the servers to be used for --enable-dap-remote-tests. 6. There were multiple versions of nctypealignment code. Try to centralize in libdispatch/doffset.c and include/ncoffsets.h 7. Add a unit test for the ncuri code because of its complexity. 8. Move the findserver code out of libdispatch and into a separate, self contained program in ncdap_test and dap4_test. 9. Move the dispatch header files (nc{3,4}dispatch.h) to .../include because they are now shared by modules. 10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts. 11. Make use of MREMAP if available 12. Misc. minor changes e.g. - #include <config.h> -> #include "config.h" - Add some no-install headers to /include - extern -> EXTERNL and vice versa as needed - misc header cleanup - clean up checking for misc. unix vs microsoft functions 13. Change copyright decls in some files to point to LICENSE file. 14. Add notes to RELEASENOTES.md
2017-03-09 08:01:10 +08:00
for(i=0;i<recordcount;i++) {
switch(basetype->subsort) {
default: /* atomic basetype */
if((ret=walkAtomicVar(compiler,topvar,basetype,&offset))) goto done;
break;
case NC_OPAQUE:
if((ret=walkOpaqueVar(compiler,topvar,basetype,&offset))) goto done;
break;
case NC_STRUCT:
/* We can treat each record like a structure instance */
if((ret=walkStruct(compiler,topvar,basetype,&offset))) goto done;
break;
case NC_SEQ:
if((ret=walkSeq(compiler,topvar,basetype,&offset))) goto done;
break;
}
}
*offsetp = offset;
done:
return THROW(ret);
}