mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2024-12-27 08:49:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
d1a3f6fbe6
netcdf-c/nczarr_test/tst_zchunks3.c

79 lines
2.3 KiB
C
Raw Normal View History

Fix nccopy -c dim/x to actually use the dim/x value. As it was, nccopy -c dim/x was sometimes being ignored. So modify nccopy to properly take into account. This also required a change to the nczarr code because it was not applying default chunking in the same way as libhdf5. Modify ncdump/tst_nccopy4.sh to test this feature properly. Also add a similar test to nczarr_test. Additionally, fix some other things that were causing Visual Studio builds with testing to not work. * fix curl testing under CMake to properly handle case where DAP is disabled, but byterange support is enabled. * properly test and/or define uintptr_t * Convert _O_XXX to O_XXX flags used by open();
2020-09-02 03:44:24 +08:00
/* This is part of the netCDF package. Copyright 2018 University
Corporation for Atmospheric Research/Unidata. See COPYRIGHT file for
conditions of use. See www.unidata.ucar.edu for more info.
Create a chunkable test file for nccopy to test chunking.
*/
#include "ut_includes.h"
#include "test_nczarr_utils.h"
Explicitly disallow variable length type compression re: https://github.com/Unidata/netcdf-c/issues/2189 Compression of a variable whose type is variable length fails for all current filters. This is because at some point, the compression buffer will contain pointers to data instead of the actual data. Compression of pointers of course is meaningless. The PR changes the behavior of nc_def_var_filter so that it will fail with error NC_EFILTER if an attempt is made to add a filter to a variable whose type is variable-length. A variable is variable-length if it is of type string or VLEN or transitively (via a compound type) contains a string or VLEN. Also added a test case for this. ## Misc Changes 1. Turn off a number of debugging statements
2022-02-20 07:47:31 +08:00
#undef DEBUG
Fix nccopy -c dim/x to actually use the dim/x value. As it was, nccopy -c dim/x was sometimes being ignored. So modify nccopy to properly take into account. This also required a change to the nczarr code because it was not applying default chunking in the same way as libhdf5. Modify ncdump/tst_nccopy4.sh to test this feature properly. Also add a similar test to nczarr_test. Additionally, fix some other things that were causing Visual Studio builds with testing to not work. * fix curl testing under CMake to properly handle case where DAP is disabled, but byterange support is enabled. * properly test and/or define uintptr_t * Convert _O_XXX to O_XXX flags used by open();
2020-09-02 03:44:24 +08:00
static int ret = NC_NOERR;
Add zip file support The primary change is to support the use of a zip file as a storage format. Simultaneously the .nz4 support is made obsolete Use of zip requires the libzip support library, so a number of changes to the build files (Makefile.am, CMakeLists.txt) are necessary to locate and incorporate libzip. The nczarr_tests tests are also changed to add zip testing. Other changes: * Make sure distcheck leaves no files around. * Add some functions to netcdf_aux to export some functions of libnetcdf. * Add a new error NC_EFOUND as the complement of NC_EEMPTY. * Add tracing support to nclog and use it in libnczarr. * Modify the zmap interface to support the writeonce semantics of zip. * Create a new s3util.c to support a variety of S3 auxilliary functions. * EXTERNL'ize a number of functions so they can be used in s3util. * Add support for the S3 ListObjects CommonPrefixes mechanism to improve search. * Add experimental support for running nczarr X s3 tests against the actual Amazon S3 cloud.
2021-01-29 11:11:01 +08:00
#define FILE_NAME "tmp_chunks3.nc"
Fix nccopy -c dim/x to actually use the dim/x value. As it was, nccopy -c dim/x was sometimes being ignored. So modify nccopy to properly take into account. This also required a change to the nczarr code because it was not applying default chunking in the same way as libhdf5. Modify ncdump/tst_nccopy4.sh to test this feature properly. Also add a similar test to nczarr_test. Additionally, fix some other things that were causing Visual Studio builds with testing to not work. * fix curl testing under CMake to properly handle case where DAP is disabled, but byterange support is enabled. * properly test and/or define uintptr_t * Convert _O_XXX to O_XXX flags used by open();
2020-09-02 03:44:24 +08:00
#define VAR_RANK 7
#define IVAR_NAME "ivar"
#define FVAR_NAME "fvar"
#define NVALS 45360 /* 7 * 4 * 2 * 3 * 5 * 6 * 9 */
/* Make trackable ERR macro replacement */
static int lerr(int stat, const char* file, int lineno) {
fflush(stdout); /* Make sure our stdout is synced with stderr. */
err++;
fprintf(stderr, "Sorry! Unexpected result(%d), %s, line: %d\n",ret,file,lineno);
fflush(stderr); \
return 2; \
}
#define LERR lerr(ret,__FILE__,__LINE__)
static const char *dim_names[VAR_RANK] = {"dim0", "dim1", "dim2", "dim3", "dim4", "dim5", "dim6"};
static const size_t dim_lens[VAR_RANK] = {7, 4, 2, 3, 5, 6, 9};
int
main(int argc, char** argv)
{
/* file metadata */
int mode = NC_CLOBBER|NC_NETCDF4;
int ncid;
int ivarid, fvarid;
int ivar_dims[VAR_RANK];
int fvar_dims[VAR_RANK];
int ivar_data[NVALS];
float fvar_data[NVALS];
int r, i;
char* file_name = FILE_NAME;
printf("*** Creating chunkable test file %s...\n", file_name);
if ((ret=nc_create(FILE_NAME, mode, &ncid))) LERR;
for(r = 0; r < VAR_RANK; r++) {
if ((ret=nc_def_dim(ncid, dim_names[r], dim_lens[r], &ivar_dims[r]))) LERR;
fvar_dims[VAR_RANK - 1 - r] = ivar_dims[r];
}
if ((ret=nc_def_var(ncid, IVAR_NAME, NC_INT, VAR_RANK, ivar_dims, &ivarid))) LERR;
/* fvar is unchanged */
if ((ret=nc_def_var(ncid, FVAR_NAME, NC_FLOAT, VAR_RANK, fvar_dims, &fvarid))) LERR;
if ((ret=nc_enddef (ncid))) LERR;
/* Fill in the data */
for(i=0; i < NVALS; i++) {
ivar_data[i] = i;
}
if ((ret=nc_put_var(ncid, ivarid, ivar_data))) LERR;
/* fvar is unchanged */
for(i=0; i < NVALS; i++) {
fvar_data[i] = NVALS - i;
}
if ((ret=nc_put_var(ncid, fvarid, fvar_data))) LERR;
if ((ret=nc_close(ncid))) LERR;
SUMMARIZE_ERR;
Add zip file support The primary change is to support the use of a zip file as a storage format. Simultaneously the .nz4 support is made obsolete Use of zip requires the libzip support library, so a number of changes to the build files (Makefile.am, CMakeLists.txt) are necessary to locate and incorporate libzip. The nczarr_tests tests are also changed to add zip testing. Other changes: * Make sure distcheck leaves no files around. * Add some functions to netcdf_aux to export some functions of libnetcdf. * Add a new error NC_EFOUND as the complement of NC_EEMPTY. * Add tracing support to nclog and use it in libnczarr. * Modify the zmap interface to support the writeonce semantics of zip. * Create a new s3util.c to support a variety of S3 auxilliary functions. * EXTERNL'ize a number of functions so they can be used in s3util. * Add support for the S3 ListObjects CommonPrefixes mechanism to improve search. * Add experimental support for running nczarr X s3 tests against the actual Amazon S3 cloud.
2021-01-29 11:11:01 +08:00
FINAL_RESULTS;
Fix nccopy -c dim/x to actually use the dim/x value. As it was, nccopy -c dim/x was sometimes being ignored. So modify nccopy to properly take into account. This also required a change to the nczarr code because it was not applying default chunking in the same way as libhdf5. Modify ncdump/tst_nccopy4.sh to test this feature properly. Also add a similar test to nczarr_test. Additionally, fix some other things that were causing Visual Studio builds with testing to not work. * fix curl testing under CMake to properly handle case where DAP is disabled, but byterange support is enabled. * properly test and/or define uintptr_t * Convert _O_XXX to O_XXX flags used by open();
2020-09-02 03:44:24 +08:00
}
Reference in New Issue Copy Permalink