mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2024-12-09 08:11:38 +08:00
Code Issues Packages Projects Releases Wiki Activity
1442762331
netcdf-c/libdap2/env

137 lines
5.4 KiB
Plaintext
Raw Normal View History

Coverity fixing
2014-08-18 02:03:23 +08:00
[NCF-287/MCI-552767 Problem related to NCF-284. Namely an old thredds server that does not properly implement the dap2 spec. It is accessing a field of a grid but is not wrapping it as a structure. url:http://thredds.aodn.org.au/thredds/dodsC/IMOS/ACORN/gridded_1h-avg-current-map_QC/SAG/2010/01/27/IMOS_ACORN_V_20100127T023000Z_SAG_FV01_1-hour-avg.nc.dds?SPEED.SPEED In any case, this exposed another bug. Also, in looking at this, I realized that the code could be simplified.
2014-02-06 08:05:33 +08:00
alias xx="cd ..;make; cd libdap2"
Fix Jirs 255 The DAP code compiles constraints into an internal tree form. When it goes to fetch data, it converts that tree form into a string suitable for use in a url. The tree->string code was using '<' when it should be using '<=' and vice versa. The fix is to make sure the right operator is used.
2013-05-17 02:22:17 +08:00
PARMS=""; ARGS=""; CON="" ; CE=""; OCON="" ; VAR=""; SHARP='#'
move from oc1.0 to oc2.0; create new dir oc2
2012-08-01 04:34:13 +08:00
alias q0=;alias qq=;alias qv=;alias q=;alias qh=;alias qqh=;alias qall=;alias qv=;alias qo=;
ckp
2017-03-12 06:20:20 +08:00
F='https://cida.usgs.gov/thredds/dodsC/new_gmo?longitude[0:1:461],latitude[0:1:221],time[0:1:22644]'
Github issue: https://github.com/Unidata/netcdf-c/issues/206 Re e-eupport VGQ-678069 It was noticed that an attribute value of "nan." was being treated as legal (it should be "nan"). The reason is that sscanf was not be checked to see that all the attribute value characters were being read. Solution is to verify that all characters were being consumed.
2016-01-29 13:18:51 +08:00
ckp
2017-03-12 06:20:20 +08:00
if test -e "/cygdrive/d/git/netcdf-c" ; then
Github issue: https://github.com/Unidata/netcdf-c/issues/206 Re e-eupport VGQ-678069 It was noticed that an attribute value of "nan." was being treated as legal (it should be "nan"). The reason is that sscanf was not be checked to see that all the attribute value characters were being read. Solution is to verify that all characters were being consumed.
2016-01-29 13:18:51 +08:00
TOP="/cygdrive/d/git/netcdf-c"
ckp
2017-03-12 06:20:20 +08:00
elif test -e "/cygdrive/f/git/netcdf-c" ; then
TOP="/cygdrive/f/git/netcdf-c"
Github issue: https://github.com/Unidata/netcdf-c/issues/206 Re e-eupport VGQ-678069 It was noticed that an attribute value of "nan." was being treated as legal (it should be "nan"). The reason is that sscanf was not be checked to see that all the attribute value characters were being read. Solution is to verify that all characters were being consumed.
2016-01-29 13:18:51 +08:00
elif test -e "/home/dmh/git/netcdf-c" ; then
TOP="/home/dmh/git/netcdf-c"
else
echo "cannot locate ncdump"
fi
JIRA: NCF-308 Jennifer Adams has requested a reversion in behavior so that all dap requests include a constraint. Problem is caused by change in prefetch where if all variables are requested, then no constraint is generated. Fix is to always generate a constraint in prefetch.
2014-07-08 04:05:27 +08:00
oc synch
2014-12-01 11:30:23 +08:00
if test -f ./ncd ; then
PROG=./ncd
else
[NCF-280]/TLO-836022 Fix bug where leading backslash digit in name was not being properly handled. The reason was that I accidentally attemped to allow \x... and \0... escapes in identifiers. This make identifiers with leading escaped digits not work any more. Also added test case.
2013-11-20 12:58:56 +08:00
PROG="$TOP/ncdump/ncdump"
oc synch
2014-12-01 11:30:23 +08:00
fi
move from oc1.0 to oc2.0; create new dir oc2
2012-08-01 04:34:13 +08:00
P=`pwd`
change format of url parameters in fragment part
2013-05-14 03:12:37 +08:00
PARMS="log"
ckp
2017-03-12 06:20:20 +08:00
#PARMS="${PARMS}[netcdf3]"
#PARMS="${PARMS}[fetch=disk]"
#PARMS="${PARMS}[cache]"
#PARMS="${PARMS}[nocache]"
#PARMS="${PARMS}[wholevar]"
PARMS="${PARMS}[show=fetch]"
#PARMS="${PARMS}[noprefetch]"
#PARMS="${PARMS}[prefetch]"
#PARMS="${PARMS}[prefetch=eager]"
Sync with oc project. This supports better authorization handling for DAP requests, especially redirection based authorization. I also added a new test case ncdap_tests/testauth.sh. Specifically, suppose I have a netrc file /tmp/netrc containing this. machine uat.urs.earthdata.nasa.gov login xxxxxx password yyyyyy Also suppose I have a .ocrc file containing these lines HTTP.COOKIEJAR=/tmp/cookies HTTP.NETRC=/tmp/netrc Assume that .ocrc is in the local directory or HOME. Then this command should work (assuming a valid login and password). ncdump -h "https://54.86.135.31/opendap/data/nc/fnoc1.nc"
2014-12-25 01:22:47 +08:00
PARMS="[log][cache][noprefetch]"
2011-04-18 02:56:10 +08:00
sync with oc2.0
2013-11-15 06:13:20 +08:00
VARGS="--leak-check=full"
move from oc1.0 to oc2.0; create new dir oc2
2012-08-01 04:34:13 +08:00
# Pick in order
if test "x$PROG" = x ; then
sync with oc2.0
2013-11-15 06:13:20 +08:00
for f in ./ncd ../ncdump/.libs/lt-ncdump ../ncdump/.libs/ncdump ../ncdump/ncdump ; do
move from oc1.0 to oc2.0; create new dir oc2
2012-08-01 04:34:13 +08:00
if test -f $f ; then
PROG="$f"
break;
fi
done
fi
if test "x$PROG" = x ; then
echo "cannot locate ncdump; defaulting"
PROG="../ncdump/ncdump"
fi
added test file
2011-09-22 03:17:57 +08:00
change format of url parameters in fragment part
2013-05-14 03:12:37 +08:00
U="$F"
if test "x$CON" != "x" ; then U="$U?$CON"; fi
ckp
2017-03-12 06:20:20 +08:00
UALL="${PARMS}$U"
2011-09-17 02:36:08 +08:00
#ARGS="-h $ARGS"
#ARGS="-w $ARGS"
#ARGS="-c $ARGS"
1. The duplicate name checking in oc2 was effectively o(n cubed); modified to be o(n squared). 2. If the list of prefetched variables is too long, (something on the order of 400 variables), then the server may reject it. Modified code so that in the case that the set of prefetch'd vars is the in fact all variables, it does not create a long request. This does not actually solve the problem if the prefetch list is long, but not all inclusive.
2013-07-28 04:20:13 +08:00
if test "x$VAR" != "x" ; then ARGS="$ARGS -v $VAR" ; fi
changed form of some static data to work with cxx4
2012-03-09 06:21:12 +08:00
2011-09-17 02:36:08 +08:00
alias qq="gdb --args $PROG $ARGS '$U'"
alias qv="valgrind $VARGS PROG $ARGS '$U'"
change format of url parameters in fragment part
2013-05-14 03:12:37 +08:00
alias q0="$PROG $ARGS '$UALL'"
alias qh="$PROG -h $ARGS '$UALL'"
alias qk="$PROG -k $ARGS '$UALL'"
alias qvar="$PROG -v $VAR $ARGS '$UALL'"
alias qqh="gdb --args $PROG -h $ARGS '$UALL'"
2011-09-17 02:36:08 +08:00
alias qall="$PROG -h $ARGS '${UALL}'"
change format of url parameters in fragment part
2013-05-14 03:12:37 +08:00
alias qv="valgrind $VARGS $PROG $ARGS '$UALL'"
fix a bug in libdap2 handling of strings
2012-01-30 12:32:12 +08:00
if test "x$CON" = "x" ; then
1)Integrate the oc with improved performance
2012-01-30 02:56:29 +08:00
alias qo="~/svn/oc/octest -p dds -p datadds -DN -DX1 $F"
fix a bug in libdap2 handling of strings
2012-01-30 12:32:12 +08:00
else
alias qo="~/svn/oc/octest -p dds -p datadds -DN -DX1 -C $CON $F"
fi
fix NCF-120
2011-09-16 00:57:16 +08:00
fixed esupport:RVK-949505 and DML-490284
2011-05-25 05:48:44 +08:00
if test 1 = 0; then
fix NCF-120
2011-09-16 00:57:16 +08:00
F="http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20110908/gfs_hd_00z"
2011-09-17 02:36:08 +08:00
CON="absvprs[0:10][6:6][0:10][0:10]"
fix NCF-120
2011-09-16 00:57:16 +08:00
CON="absvprs[0:1:64][6:1:6][0:1:360][0:1:719]"
F="http://ferret.pmel.noaa.gov/geoide/dodsC/ct_flux"
fixed escape handling for names
2011-05-28 02:57:35 +08:00
F="http://motherlode.ucar.edu:9080/thredds/dodsC/fmrc/NCEP/GFS/Alaska_191km/NCEP-GFS-Alaska_191km_fmrc.ncx"
fixed esupport:RVK-949505 and DML-490284
2011-05-25 05:48:44 +08:00
F="file:///home/dmh/nc/oc/dataset-duacs-nrt-over30d-global-merged-madt-h"
F="http://oceanwatch.pfeg.noaa.gov/opendap/GLOBEC/GLOBEC_cetaceans"
CON="number&number>6"
F="http://dapper.pmel.noaa.gov/dapper/argo/argo_all.cdp"
#CON="&location.LATITUDE<1&location.LATITUDE>-1"
F="http://test.opendap.org:8080/dods/dts/test.03"
#CON="s0,s1"
F="http://oceanwatch.pfeg.noaa.gov/opendap/GLOBEC/GLOBEC_cetaceans"
CON="lat,lon&lat>42.0&lat<=42.5"
CON="number&number>6"
F="http://motherlode.ucar.edu:8080/thredds/dodsC/testdods/in.nc"
F="http://ceda.ac.uk/dap/neodc/casix/seawifs_plankton/data/monthly/PSC_monthly_1998.nc"
F="http://test.opendap.org:8080/dods/dts/test.02"
F="http://test.opendap.org/opendap/data/nc/coads_climatology.nc"
move from oc1.0 to oc2.0; create new dir oc2
2012-08-01 04:34:13 +08:00
F="file://${TOP}/ncdap_test/testdata3/test.PointFile"
F="file://${TOP}/ncdap_test/testdata3/synth1"
fixed esupport:RVK-949505 and DML-490284
2011-05-25 05:48:44 +08:00
F="http://dods.ndbc.noaa.gov/thredds/dodsC/data/stdmet/46029/46029h9999.nc"
CON="wind_dir[1:10][0:0][0:0]"
completely rewritten constraint system
2011-11-14 12:20:19 +08:00
F="http://nomads.ncep.noaa.gov:9090/dods/gens/gens20111011/gep20_00z"
CON="ens,time,lat,lon,prmslmsl[0:1:0][0:1:64][0:1:10][0:1:10]"
- Fix NCF-158 to modify ncgen flag defaults. - Fix NCF-157 to modify DAP code to support partial variable retrieval. - Fix of NCF-154 to solve problem of ncgen improperly processing data lists for variables of size greater than 2**18 bytes. - Fix ncgen processing of char variables that have multiple unlimited dimensions. - Partly fix Jira issue: NCF-145 (vlen issues). - Benchmark program nc_test4/tst_ar4_*) requires arguments and should only be invoked inside a shell script; fixed so that they terminate cleanly if invoked with no arguments. - Fix the Doxygen processing so it will work with make distcheck. - Begin switchover to using an alternative to ncio. - Begin support for in-memory (diskless) files.
2012-03-15 07:26:48 +08:00
F="http://esrl.noaa.gov/psd/thredds/dodsC/Datasets/noaa.oisst.v2/sst.wkmean.1990-present.nc"
1)Integrate the oc with improved performance
2012-01-30 02:56:29 +08:00
F="http://motherlode.ucar.edu:8080/thredds/dodsC/station/metar/Surface_METAR_20111122_0000.nc"
CON="weather[0:10]"
- Fix NCF-158 to modify ncgen flag defaults. - Fix NCF-157 to modify DAP code to support partial variable retrieval. - Fix of NCF-154 to solve problem of ncgen improperly processing data lists for variables of size greater than 2**18 bytes. - Fix ncgen processing of char variables that have multiple unlimited dimensions. - Partly fix Jira issue: NCF-145 (vlen issues). - Benchmark program nc_test4/tst_ar4_*) requires arguments and should only be invoked inside a shell script; fixed so that they terminate cleanly if invoked with no arguments. - Fix the Doxygen processing so it will work with make distcheck. - Begin switchover to using an alternative to ncio. - Begin support for in-memory (diskless) files.
2012-03-15 07:26:48 +08:00
F="http://nomad1.ncep.noaa.gov:9090/dods/reanalyses/reanalysis-1/6hr/pgb/pgb"
VAR=hgtprs
Add quick check for Error {...} at start of the datadds data part
2013-02-08 06:14:57 +08:00
F="file://$TOP/ncdap_test/testdata3/$T"
#F="http://motherlode.ucar.edu:8081/dts/$T?f64"
#F="http://nomads.ncep.noaa.gov:9090/dods/gfs_hd/gfs_hd20120801/gfs_hd_00z"
#F="http://hycom.coaps.fsu.edu/thredds/dodsC/atl_ops"
#F="http://testbedapps-dev.sura.org/thredds/dodsC/auto/in.usf.fvcom.ike.ultralite.vardrag.nowave.2d"
#F="http://motherlode.ucar.edu:8081/dts/test.06"
#F="http://motherlode.ucar.edu:8081/thredds/dodsC/testdods/nametest3.nc"
#F="http://ingrid.ldeo.columbia.edu/SOURCES/.USTOPO/dods"
jira: NCF-232: bad conversion of grids
2013-02-26 12:31:06 +08:00
F="http://goldsmr2.sci.gsfc.nasa.gov/opendap/hyrax/MERRA/MAT1NXSLV.5.2.0/1990/01/MERRA100.prod.assim.tavg1_2d_slv_Nx.19900101.hdf"
CON="U50M_EOSGRID_Data_Fields[0:23][282:282][441:441]"
#F="http://motherlode.ucar.edu:9080/thredds/dodsC/grib/NCEP/NAM/CONUS_12km/best"
#CON="OneD.amp,TwoD.amp,ThreeD.amp"
Primary change is to add ability to do prefetch on either a lazy or eager basis. Lazy means that the prefetch does not occur until and unless the client actually makes a get_var request. Also repaired a problem where doing prefetch wrt a url that has a constraint will prefetch a whole variable if its constrained size is small enough, even if the underlying variable is too large to warrant prefetch.
2013-03-03 12:06:43 +08:00
F="http://goldsmr2.sci.gsfc.nasa.gov/opendap/hyrax/MERRA/MAT1NXSLV.5.2.0/1990/01/MERRA100.prod.assim.tavg1_2d_slv_Nx.19900101.hdf"
CON="U50M_EOSGRID_Data_Fields[0:23][282:282][441:441],XDim_EOSGRID"
It turns out that my dap code is not properly checking the incoming count argument for nc_get_vars. It is treating the count as pre-stride rather than post-stride.
2013-05-02 06:31:01 +08:00
F="http://opendap.co-ops.nos.noaa.gov/dods/IOOS/SixMin_Verified_Water_Level"
CON="&WATERLEVEL_6MIN_VFD_PX._STATION_ID=%228779770%22&WATERLEVEL_6MIN_VFD_PX._DATUM=%22MSL%22&WATERLEVEL_6MIN_VFD_PX._BEGIN_DATE=%2220061001%22&WATERLEVEL_6MIN_VFD_PX._END_DATE=%2220061030%22"
define mode_t when under windows
2013-11-16 06:41:17 +08:00
F="http://iridl.ldeo.columbia.edu/SOURCES/.Models/.NMME/.NASA-GMAO/.MONTHLY/.sst/dods"
F="http://tiggeUser:tigge@thredds-test.ucar.edu/thredds/dodsC/restrict/testData.nc"
F="http://iridl.ldeo.columbia.edu/SOURCES/.Models/.NMME/.NASA-GMAO/.MONTHLY/.sst/dods"
[NCF-287/MCI-552767 Problem related to NCF-284. Namely an old thredds server that does not properly implement the dap2 spec. It is accessing a field of a grid but is not wrapping it as a structure. url:http://thredds.aodn.org.au/thredds/dodsC/IMOS/ACORN/gridded_1h-avg-current-map_QC/SAG/2010/01/27/IMOS_ACORN_V_20100127T023000Z_SAG_FV01_1-hour-avg.nc.dds?SPEED.SPEED In any case, this exposed another bug. Also, in looking at this, I realized that the code could be simplified.
2014-02-06 08:05:33 +08:00
#F="http://ticket:ticket1@utmea.enea.it:8080/thredds/dodsC/UNIDATA_passwd/head_out.nc"
F="http://nomads.ncep.noaa.gov:9090/dods/gens/gens20140123/gep_all_12z"
VAR=prmslmsl
fix t_auth failure now that tiggeUser password is set
2014-03-01 05:39:30 +08:00
F="http://data.nodc.noaa.gov/thredds/dodsC/testdata/pathfinderAgg/pathFinderV5.2_night.ncml"
CON="sst_dtime.sst_dtime"
ckp
2017-03-12 06:20:20 +08:00
F="http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis.dailyavgs/pressure/air.1947.nc"
completely rewritten constraint system
2011-11-14 12:20:19 +08:00
fi
Reference in New Issue Copy Permalink