netcdf-c/ncgen/bytebuffer.c

295 lines
5.8 KiB
C
Raw Normal View History

2018-12-07 06:40:43 +08:00
/* Copyright 2018, UCAR/Unidata and OPeNDAP, Inc.
2010-06-03 21:24:43 +08:00
See the COPYRIGHT file for more information. */
2012-08-02 01:18:58 +08:00
#include "config.h"
#include <stddef.h>
2010-06-03 21:24:43 +08:00
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "bytebuffer.h"
#include "debug.h"
#ifndef TRUE
#define TRUE 1
#endif
#ifndef FALSE
#define FALSE 0
#endif
#define DEFAULTALLOC 1024
#define ALLOCINCR 1024
int bbdebug = 1;
/* For debugging purposes*/
static int
2010-06-03 21:24:43 +08:00
bbFail(void)
{
fflush(stdout);
fprintf(stderr,"bytebuffer failure\n");
fflush(stderr);
2017-11-01 04:03:57 +08:00
if(bbdebug) abort();
2010-06-03 21:24:43 +08:00
return FALSE;
}
Bytebuffer*
bbNew(void)
{
Fix more memory leaks in netcdf-c library This is a follow up to PR https://github.com/Unidata/netcdf-c/pull/1173 Sorry that it is so big, but leak suppression can be complex. This PR fixes all remaining memory leaks -- as determined by -fsanitize=address, and with the exceptions noted below. Unfortunately. there remains a significant leak that I cannot solve. It involves vlens, and it is unclear if the leak is occurring in the netcdf-c library or the HDF5 library. I have added a check_PROGRAM to the ncdump directory to show the problem. The program is called tst_vlen_demo.c To exercise it, build the netcdf library with -fsanitize=address enabled. Then go into ncdump and do a "make clean check". This should build tst_vlen_demo without actually executing it. Then do the command "./tst_vlen_demo" to see the output of the memory checker. Note the the lost malloc is deep in the HDF5 library (in H5Tvlen.c). I am temporarily working around this error in the following way. 1. I modified several test scripts to not execute known vlen tests that fail as described above. 2. Added an environment variable called NC_VLEN_NOTEST. If set, then those specific tests are suppressed. This should mean that the --disable-utilities option to ./configure should not need to be set to get a memory leak clean build. This should allow for detection of any new leaks. Note: I used an environment variable rather than a ./configure option to control the vlen tests. This is because it is temporary (I hope) and because it is a bit tricky for shell scripts to access ./configure options. Finally, as before, this only been tested with netcdf-4 and hdf5 support.
2018-11-16 01:00:38 +08:00
Bytebuffer* bb = (Bytebuffer*)emalloc(sizeof(Bytebuffer));
Support MSYS2/Mingw platform re: The current netcdf-c release has some problems with the mingw platform on windows. Mostly they are path issues. Changes to support mingw+msys2: ------------------------------- * Enable option of looking into the windows registry to find the mingw root path. In aid of proper path handling. * Add mingw+msys as a specific platform in configure.ac and move testing of the platform to the front so it is available early. * Handle mingw X libncpoco (dynamic loader) properly even though mingw does not yet support it. * Handle mingw X plugins properly even though mingw does not yet support it. * Alias pwd='pwd -W' to better handle paths in shell scripts. * Plus a number of other minor compile irritations. * Disallow the use of multiple nc_open's on the same file for windows (and mingw) because windows does not seem to handle these properly. Not sure why we did not catch this earlier. * Add mountpoint info to dpathmgr.c to help support mingw. * Cleanup dpathmgr conversions. Known problems: --------------- * I have not been able to get shared libraries to work, so plugins/filters must be disabled. * There is some kind of problem with libcurl that I have not solved, so all uses of libcurl (currently DAP+Byterange) must be disabled. Misc. other fixes: ------------------ * Cleanup the relationship between ENABLE_PLUGINS and various other flags in CMakeLists.txt and configure.ac. * Re-arrange the TESTDIRS order in Makefile.am. * Add pseudo-breakpoint to nclog.[ch] for debugging. * Improve the documentation of the path manager code in ncpathmgr.h * Add better support for relative paths in dpathmgr.c * Default the mode args to NCfopen to include "b" (binary) for windows. * Add optional debugging output in various places. * Make sure that everything builds with plugins disabled. * Fix numerous (s)printf inconsistencies betweenb the format spec and the arguments.
2021-12-24 13:18:56 +08:00
if(bb == NULL) {bbFail(); return NULL;}
2010-06-03 21:24:43 +08:00
bb->alloc=0;
bb->length=0;
bb->content=NULL;
bb->nonextendible = 0;
2014-07-30 01:00:44 +08:00
return bb;
2010-06-03 21:24:43 +08:00
}
int
bbSetalloc(Bytebuffer* bb, const unsigned int sz0)
{
unsigned int sz = sz0;
char* newcontent;
if(bb == NULL) return bbFail();
if(sz <= 0) {sz = (bb->alloc?2*bb->alloc:DEFAULTALLOC);}
else if(bb->alloc >= sz) return TRUE;
else if(bb->nonextendible) return bbFail();
Fix more memory leaks in netcdf-c library This is a follow up to PR https://github.com/Unidata/netcdf-c/pull/1173 Sorry that it is so big, but leak suppression can be complex. This PR fixes all remaining memory leaks -- as determined by -fsanitize=address, and with the exceptions noted below. Unfortunately. there remains a significant leak that I cannot solve. It involves vlens, and it is unclear if the leak is occurring in the netcdf-c library or the HDF5 library. I have added a check_PROGRAM to the ncdump directory to show the problem. The program is called tst_vlen_demo.c To exercise it, build the netcdf library with -fsanitize=address enabled. Then go into ncdump and do a "make clean check". This should build tst_vlen_demo without actually executing it. Then do the command "./tst_vlen_demo" to see the output of the memory checker. Note the the lost malloc is deep in the HDF5 library (in H5Tvlen.c). I am temporarily working around this error in the following way. 1. I modified several test scripts to not execute known vlen tests that fail as described above. 2. Added an environment variable called NC_VLEN_NOTEST. If set, then those specific tests are suppressed. This should mean that the --disable-utilities option to ./configure should not need to be set to get a memory leak clean build. This should allow for detection of any new leaks. Note: I used an environment variable rather than a ./configure option to control the vlen tests. This is because it is temporary (I hope) and because it is a bit tricky for shell scripts to access ./configure options. Finally, as before, this only been tested with netcdf-4 and hdf5 support.
2018-11-16 01:00:38 +08:00
newcontent=(char*)ecalloc(sz*sizeof(char));
2010-06-03 21:24:43 +08:00
if(bb->alloc > 0 && bb->length > 0 && bb->content != NULL) {
memcpy((void*)newcontent,(void*)bb->content,sizeof(char)*bb->length);
}
if(bb->content != NULL) efree(bb->content);
bb->content=newcontent;
bb->alloc=sz;
return TRUE;
}
void
bbFree(Bytebuffer* bb)
{
if(bb == NULL) return;
if(bb->content != NULL) efree(bb->content);
efree(bb);
}
int
bbSetlength(Bytebuffer* bb, const unsigned int sz)
{
if(bb == NULL) return bbFail();
if(bb->length < sz) {
if(!bbSetalloc(bb,sz)) return bbFail();
}
2010-06-03 21:24:43 +08:00
bb->length = sz;
return TRUE;
}
int
bbFill(Bytebuffer* bb, const char fill)
{
unsigned int i;
if(bb == NULL) return bbFail();
2014-07-30 01:00:44 +08:00
for(i=0;i<bb->length;i++) bb->content[i] = fill;
2010-06-03 21:24:43 +08:00
return TRUE;
}
int
bbGet(Bytebuffer* bb, unsigned int index)
{
if(bb == NULL) return -1;
if(index >= bb->length) return -1;
return bb->content[index];
}
int
bbSet(Bytebuffer* bb, unsigned int index, char elem)
{
if(bb == NULL) return bbFail();
if(index >= bb->length) return bbFail();
bb->content[index] = elem;
return TRUE;
}
int
bbAppend(Bytebuffer* bb, char elem)
{
if(bb == NULL) return bbFail();
/* We need space for the char + null */
2014-09-23 04:31:19 +08:00
while(bb->length+1 >= bb->alloc) {
if(!bbSetalloc(bb,0))
return bbFail();
}
2010-06-03 21:24:43 +08:00
bb->content[bb->length] = elem;
bb->length++;
bb->content[bb->length] = '\0';
return TRUE;
}
/* This assumes s is a null terminated string*/
int
bbCat(Bytebuffer* bb, const char* s)
{
bbAppendn(bb,(void*)s,strlen(s)+1); /* include trailing null*/
/* back up over the trailing null*/
if(bb->length == 0) return bbFail();
bb->length--;
return 1;
}
int
bbCatbuf(Bytebuffer* bb, const Bytebuffer* s)
{
if(bbLength(s) > 0)
bbAppendn(bb,bbContents(s),bbLength(s));
bbNull(bb);
return 1;
}
int
bbAppendn(Bytebuffer* bb, const void* elem, const size_t n0)
2010-06-03 21:24:43 +08:00
{
size_t n = n0;
2010-06-03 21:24:43 +08:00
if(bb == NULL || elem == NULL) return bbFail();
if(n == 0) {n = strlen((char*)elem);}
while(!bbNeed(bb,(n+1))) {if(!bbSetalloc(bb,0)) return bbFail();}
memcpy((void*)&bb->content[bb->length],(void*)elem,n);
bb->length += (unsigned int)n;
2010-06-03 21:24:43 +08:00
bb->content[bb->length] = '\0';
return TRUE;
}
int
bbInsert(Bytebuffer* bb, const unsigned int index, const char elem)
{
char tmp[2];
tmp[0]=elem;
return bbInsertn(bb,index,tmp,1);
}
int
bbInsertn(Bytebuffer* bb, const unsigned int index, const char* elem, const unsigned int n)
{
unsigned int i;
unsigned int j;
2014-07-30 01:00:44 +08:00
unsigned int newlen = 0;
2010-06-03 21:24:43 +08:00
if(bb == NULL) return bbFail();
2014-07-30 01:00:44 +08:00
newlen = bb->length + n;
2010-06-03 21:24:43 +08:00
if(newlen >= bb->alloc) {
if(!bbExtend(bb,n)) return bbFail();
}
2014-07-30 01:00:44 +08:00
/*
2010-06-03 21:24:43 +08:00
index=0
n=3
len=3
newlen=6
a b c
x y z a b c
-----------
0 1 2 3 4 5
i=0 1 2
j=5 4 3
2 1 0
*/
for(j=newlen-1,i=index;i<bb->length;i++) {
bb->content[j]=bb->content[j-n];
}
memcpy((void*)(bb->content+index),(void*)elem,n);
bb->length += n;
return TRUE;
}
2014-10-04 02:38:32 +08:00
/*! Pop head off of a byte buffer.
*
* @param Bytebuffer bb Pointer to Bytebuffer.
* @param char* pelem pointer to location for head element.
*
* @return Returns TRUE on success.
*/
int bbHeadpop(Bytebuffer* bb, char* pelem)
2010-06-03 21:24:43 +08:00
{
if(bb == NULL) return bbFail();
if(bb->length == 0) return bbFail();
*pelem = bb->content[0];
2014-10-04 02:38:32 +08:00
memmove((void*)&bb->content[0],
(void*)&bb->content[1],
sizeof(char)*(bb->length - 1));
2014-07-30 01:00:44 +08:00
bb->length--;
return TRUE;
2010-06-03 21:24:43 +08:00
}
int
bbTailpop(Bytebuffer* bb, char* pelem)
{
if(bb == NULL) return bbFail();
if(bb->length == 0) return bbFail();
*pelem = bb->content[bb->length-1];
2014-07-30 01:00:44 +08:00
bb->length--;
return TRUE;
2010-06-03 21:24:43 +08:00
}
int
bbHeadpeek(Bytebuffer* bb, char* pelem)
{
if(bb == NULL) return bbFail();
if(bb->length == 0) return bbFail();
*pelem = bb->content[0];
2014-07-30 01:00:44 +08:00
return TRUE;
2010-06-03 21:24:43 +08:00
}
int
bbTailpeek(Bytebuffer* bb, char* pelem)
{
if(bb == NULL) return bbFail();
if(bb->length == 0) return bbFail();
*pelem = bb->content[bb->length - 1];
2014-07-30 01:00:44 +08:00
return TRUE;
2010-06-03 21:24:43 +08:00
}
char*
bbDup(const Bytebuffer* bb)
{
Fix more memory leaks in netcdf-c library This is a follow up to PR https://github.com/Unidata/netcdf-c/pull/1173 Sorry that it is so big, but leak suppression can be complex. This PR fixes all remaining memory leaks -- as determined by -fsanitize=address, and with the exceptions noted below. Unfortunately. there remains a significant leak that I cannot solve. It involves vlens, and it is unclear if the leak is occurring in the netcdf-c library or the HDF5 library. I have added a check_PROGRAM to the ncdump directory to show the problem. The program is called tst_vlen_demo.c To exercise it, build the netcdf library with -fsanitize=address enabled. Then go into ncdump and do a "make clean check". This should build tst_vlen_demo without actually executing it. Then do the command "./tst_vlen_demo" to see the output of the memory checker. Note the the lost malloc is deep in the HDF5 library (in H5Tvlen.c). I am temporarily working around this error in the following way. 1. I modified several test scripts to not execute known vlen tests that fail as described above. 2. Added an environment variable called NC_VLEN_NOTEST. If set, then those specific tests are suppressed. This should mean that the --disable-utilities option to ./configure should not need to be set to get a memory leak clean build. This should allow for detection of any new leaks. Note: I used an environment variable rather than a ./configure option to control the vlen tests. This is because it is temporary (I hope) and because it is a bit tricky for shell scripts to access ./configure options. Finally, as before, this only been tested with netcdf-4 and hdf5 support.
2018-11-16 01:00:38 +08:00
char* result = (char*)emalloc(bb->length+1);
2010-06-03 21:24:43 +08:00
memcpy((void*)result,(const void*)bb->content,bb->length);
result[bb->length] = '\0'; /* just in case it is a string*/
return result;
}
int
bbSetcontents(Bytebuffer* bb, char* contents, const unsigned int alloc)
{
if(bb == NULL) return bbFail();
bbClear(bb);
if(!bb->nonextendible && bb->content != NULL) efree(bb->content);
bb->content = contents;
bb->length = 0;
bb->alloc = alloc;
bb->nonextendible = 1;
return 1;
}
/* Add invisible NULL terminator */
int
bbNull(Bytebuffer* bb)
{
bbAppend(bb,'\0');
bb->length--;
return 1;
}
Fix more memory leaks in netcdf-c library This is a follow up to PR https://github.com/Unidata/netcdf-c/pull/1173 Sorry that it is so big, but leak suppression can be complex. This PR fixes all remaining memory leaks -- as determined by -fsanitize=address, and with the exceptions noted below. Unfortunately. there remains a significant leak that I cannot solve. It involves vlens, and it is unclear if the leak is occurring in the netcdf-c library or the HDF5 library. I have added a check_PROGRAM to the ncdump directory to show the problem. The program is called tst_vlen_demo.c To exercise it, build the netcdf library with -fsanitize=address enabled. Then go into ncdump and do a "make clean check". This should build tst_vlen_demo without actually executing it. Then do the command "./tst_vlen_demo" to see the output of the memory checker. Note the the lost malloc is deep in the HDF5 library (in H5Tvlen.c). I am temporarily working around this error in the following way. 1. I modified several test scripts to not execute known vlen tests that fail as described above. 2. Added an environment variable called NC_VLEN_NOTEST. If set, then those specific tests are suppressed. This should mean that the --disable-utilities option to ./configure should not need to be set to get a memory leak clean build. This should allow for detection of any new leaks. Note: I used an environment variable rather than a ./configure option to control the vlen tests. This is because it is temporary (I hope) and because it is a bit tricky for shell scripts to access ./configure options. Finally, as before, this only been tested with netcdf-4 and hdf5 support.
2018-11-16 01:00:38 +08:00
/* Extract the content and leave content null */
char*
bbExtract(Bytebuffer* bb)
{
char* x = NULL;
if(bb == NULL || bb->content == NULL)
return NULL;
x = bb->content;
bb->content = NULL;
bb->length = 0;
bb->alloc = 0;
return x;
}