re: https://github.com/Unidata/netcdf-c/issues/365
1. Added to RELEASENOTES.md
2. Add a range check to more closely
mimic unix sscanf
3. locate and fix same sscanf problems in ncgen/cvt.c
Still need a stable url for a test case.
Following command fails under visual studio.
ncdump -h http://thredds.ucar.edu/thredds/dodsC/nexrad/composite/gini/n0r/1km/20170216/Level3_Composite_n0r_1km_20170216_1635.gini
The problem is that sscanf for windows does not appear to support
scanning 8bit integers: it appears to only allow scanning of characters.
Solution:
Scan the input as an integer (for type Byte) or unsigned int (for type UByte)
and then recast the result as a char or unsigned char.
Primary code fix is in libdap2/dapcvt.c#dapcvtattrval
Problem was in oc2/dap.y.
In definition of errormsg:, change WORD_WORD to WORD_STRING
since the msg field of an opendap error response is a quoted
string.
Also took the opportunity to modify ncgen to
transfer the logging level (-L flag) into the c-code
generated using -lc.
the multiple definitions of
typedef struct DCEparsestate
near lines 10 and 42 causes compiler
problems with some versions of gcc.
remove the second typedef.
(Re: [netcdfgroup] nccopy fails with corrupted double link list)
shows that ncdump/nccopy was returning EPERM instead of
NC_EDAPCONSTRAINT as an error when we have a malformed constraint.
Also clean up a potential bug that might occur if the user invokes
nc_set_default_format before calling nc_open on a dap url.
re: github netcdf-c issue #271
This occurs for several reasons, including:
1. using H5Aopen_name instead of H5Aexists to test if attribute exists.
2. using H5Eset_auto instead of H5Eset_auto2.
There are probably others that will have to be extinguished as encountered.
p.s Hope I did not overdo this and kill too much.
Re e-eupport VGQ-678069
It was noticed that an attribute value of "nan." was being treated as
legal (it should be "nan"). The reason is that sscanf was not be checked
to see that all the attribute value characters were being read.
Solution is to verify that all characters were being consumed.
It appears the problem is that synth9 was erroneously
included in testing. It involves a nested sequence which
is not translatable. Not sure why it was still there,
but fix is to suppress the test.
This supports better authorization
handling for DAP requests, especially redirection
based authorization. I also added a new test case
ncdap_tests/testauth.sh.
Specifically, suppose I have a netrc file /tmp/netrc
containing this.
machine uat.urs.earthdata.nasa.gov login xxxxxx password yyyyyy
Also suppose I have a .ocrc file containing these lines
HTTP.COOKIEJAR=/tmp/cookies
HTTP.NETRC=/tmp/netrc
Assume that .ocrc is in the local directory or HOME.
Then this command should work (assuming a valid login and password).
ncdump -h "https://54.86.135.31/opendap/data/nc/fnoc1.nc"
Jennifer Adams has requested a reversion in behavior so that
all dap requests include a constraint.
Problem is caused by change in prefetch where if all variables
are requested, then no constraint is generated.
Fix is to always generate a constraint in prefetch.
re: e support ZCL-340681 and CPW-270700
HDF4 supports compression (and chunking)
but the chunking was not being recorded
for HDF4 files. So, I modified the necessary
files to support HDF4 chunking.
using cygwin64 causes a couple of bugs during make check.
Fixes are as follows:
1. in ncdump.c and nccopy.c change all occurrences of
'return EXIT_FAILURE'
and
'return EXIT_SUCCESS'
with
exit(EXIT_FAILURE)
and
exit(EXIT_SUCCESS)
respectively.
I have no idea why this works.
2. in libdap2/ncdap3a.c#freeNCDAPCOMMON,
remove the call to nc_abort, it is
causing a loop; not sure why this
is not caught under other operating systems.
Some servers do not properly
implement the current DAP2 spec.
It turns out that this server is one of those:
http://nomads.ncep.noaa.gov:9090/
When a reference such as this is made:
http://nomads.ncep.noaa.gov:9090/dods/gens/gens20140123/gep_all_12z?prmslmsl[0][0][0][0:359]
tt is returning this:
Dataset {
float prmslmsl[ens=1][time=1][lat=1][lon=360];
} gens%2fgens20140123%2fgep_all_12z;
when it should be returning this:
Dataset {
Structure {
float prmslmsl[ens=1][time=1][lat=1][lon=360];
} prmslmsl;
} gens%2fgens20140123%2fgep_all_12z;
The reason is that when picking fields out of a grid,
one must maintain the fully qualified name, so the grid
is converted to an enclosing structure.
It turns out that the problem was that
when I create the new structure node, I was
improperly linking it into the existing graph.
This caused a null pointer failure.
Fix is to make sure the relevant field (node->root)
is set.
The code that tests if a path is a url is
faulting when the url does not end in a slash
(e.g. http://thredds-tests.ucar.edu).
The code that tests if a path is a url is
faulting when the url does not end in a slash
(e.g. http://thredds-tests.ucar.edu).
CF-273]/HZY-708311
Solution was to do a null pointer test.
Added a test (tst_misc).
error occurs after an "exit:" label.
Corrected a dozen Coverity errors (mainly allocation issues, along with a few
other things):
711711, 711802, 711803, 711905, 970825, 996123, 996124, 1025787,
1047274, 1130013, 1130014, 1139538
Refactored internal fill-value code to correctly handle string types, and
especially to allow NULL pointers and null strings (ie. "") to be
distinguished. The code now avoids partially aliasing the two together
(which only happened on the 'write' side of things and wasn't reflected on
the 'read' side, adding to the previous confusion).
Probably still weak on handling fill-values of variable-length and compound
datatypes.
Refactored the recursive metadata reads a bit more, to process HDF5 named
datatypes and datasets immediately, avoiding chewing up memory for those
types of objects, etc.
Finished uncommenting and updating the nc_test4/tst_fills2.c code (as I'm
proceeding alphabetically through the nc_test4 code files).
Add a new function called nc_inq_format_extended that
returns more detailed format information (vis-a-vis
nc_inq_format) about an open dataset.
Note that the netcdf API will present the file as if it had
the format specified by nc_inq_format. The true file
format, however, may not even be a netcdf file; it might be
DAP, HDF4, or PNETCDF, for example. This function returns
that true file type. It also returns the effective mode for
the file.
signature: nc_inq_format_extended(int ncid, int* formatp, int* modep)
where
* ncid is the NetCDF ID from a previous call to nc_open() or
nc_create().
* formatp is a pointer to a location for returned true format.
* modep is a pointer to a location for returned mode flags.
Refer to the actual list in the file netcdf.h to see the
currently defined set.
Also added test cases (tst_formatx*).
Fix bug where leading backslash digit
in name was not being properly handled.
The reason was that I accidentally attemped to allow \x... and \0...
escapes in identifiers. This make identifiers
with leading escaped digits not work any more.
Also added test case.
Fix Http Basic Authorization.
The problem is really in oc2.0.
In order for it to work,
the CURLOPT_COOKIEJAR must have
a non-null value. The code
was already there, but not being
used for some reason.
1. fixed cookiejar code in oc2.0
2. synched oc2.0 with netcdf-c/oc2
3. added a test case
effectively o(n cubed); modified to be
o(n squared).
2. If the list of prefetched variables is too long,
(something on the order of 400 variables), then
the server may reject it. Modified code so that
in the case that the set of prefetch'd vars is
the in fact all variables, it does not create a long
request. This does not actually solve the problem
if the prefetch list is long, but not all inclusive.
effectively o(n cubed); modified to be
o(n squared).
2. If the list of prefetched variables is too long,
(something on the order of 400 variables), then
the server may reject it. Modified code so that
in the case that the set of prefetch'd vars is
the in fact all variables, it does not create a long
request. This does not actually solve the problem
if the prefetch list is long, but not all inclusive.
group renaming. The primary API
is 'nc_rename_grp(int grpid, const char* name)'.
No test cases provided yet.
This also required adding a rename_grp entry
to the dispatch tables.
