Commit Graph

104 Commits

Author SHA1 Message Date
Ward Fisher
f9f27c5ab7
Merge branch 'master' into newhash0.dmh 2018-02-21 14:19:43 -07:00
Dennis Heimbigner
727b613459 This is the initial step in moving to the new higher performance
(I hope) metadata mechanism. This mostly just adds new pieces of
code (e.g. nclistmap) and does some minor fixes.

It should be transparent to everything else.
The next set of changes will be the big step.
2018-02-08 19:53:40 -07:00
Ed Hartnett
2358d4a910 moved HDF4 to its own dispatch layer 2018-02-08 06:20:58 -07:00
Dennis Heimbigner
99fccab359 1. Keep up to date by merging master
2. Fixed plugin building (nc_test4/hdf5plugins)
   to be done properly by cmake and automake.
4. Duplicated part of the nc_test4 filter test code
   in examples/C

An incomplete and untested set of hooks exist
for OS-X in nc_test4/findplugins.in. They need testing.
2018-01-16 11:00:09 -07:00
Ed Hartnett
ea49cc1d3f merged ejh_notbuilt_errors 2017-12-04 13:23:31 -07:00
Ed Hartnett
1153f0f084 merged in branch ejh_cleanup 2017-12-04 13:20:42 -07:00
Ed Hartnett
4de61e21f2 more docs, more cleaning 2017-12-04 12:21:14 -07:00
Ed Hartnett
857dda487f fixed two doxygen warnings 2017-11-30 08:45:53 -07:00
Ed Hartnett
34c9248e6f
Merge branch 'master' into ejh_notbuilt_errors 2017-11-28 17:16:56 +00:00
Dennis Heimbigner
4db4393e69 Begin changing over to use strlcat instead of strncat because
strlcat provides better protection against buffer overflows.

Code is taken from the FreeBSD project source code. Specifically:
https://github.com/freebsd/freebsd/blob/master/lib/libc/string/strlcat.c
License appears to be acceptable, but needs to be checked by e.g. Debian.

Step 1:
1. Add to netcdf-c/include/ncconfigure.h to use our version
   if not already available as determined by HAVE_STRLCAT in config.h.
2. Add the strlcat code to libdispatch/dstring.c
3. Turns out that strlcat was already defined in several places.
   So remove it from:
	ncgen3/genlib.c
	ncdump/dumplib.c
3. Define strlcat extern definition in ncconfigure.h.
4. Modify following directories to use strlcat:
	libdap2 libdap4 ncdap_test dap4_test
   Will do others in subsequent steps.
2017-11-23 10:55:24 -07:00
Ed Hartnett
eecade6af9
Merge branch 'master' into ejh_notbuilt_errors 2017-11-18 21:24:11 +00:00
Ed Hartnett
a9ba19b43c merged in changes from master 2017-11-15 12:57:11 -07:00
Ed Hartnett
98fbd794f2 fixed doxygen warnings by adding docs 2017-11-14 06:48:38 -07:00
Ed Hartnett
03753c0344 now return NC_ENOTBUILT for not built library features 2017-11-13 08:09:15 -07:00
Ed Hartnett
6d31b47543
Merge branch 'master' into ejh_more_warnings 2017-11-10 02:52:46 -07:00
Ed Hartnett
fcef323154 fixed warning in dfile.c 2017-11-09 05:58:15 -07:00
Ed Hartnett
7872785cf0
Merge branch 'master' into ejh_enum_mem_fix 2017-11-03 19:44:40 -06:00
Ward Fisher
f98635be1b Merge remote-tracking branch 'origin/h5offset.dmh' into dmh_multi_merge 2017-11-03 12:40:52 -06:00
Ed Hartnett
406e979fb4 fixed tst_enums memory issue 2017-11-01 04:31:12 -06:00
Ed Hartnett
44ded96029 fix for leaks in nc_create 2017-10-31 11:37:30 -06:00
Dennis Heimbigner
261ed4c99d Merge branch 'master' into h5offset.dmh 2017-10-27 12:33:17 -06:00
Ed Hartnett
36940e2ea4 fixed memory leaks in nc_open() 2017-10-26 14:19:48 -06:00
Dennis Heimbigner
9983b9d911 re e-support UBS-599337
re pull request https://github.com/Unidata/netcdf-c/pull/405
re pull request https://github.com/Unidata/netcdf-c/pull/446

Notes:
1. This branch is a cleanup of the magic.dmh branch.
2. magic.dmh was originally merged, but caused problems with parallel IO.
   It was re-issued as pull request https://github.com/Unidata/netcdf-c/pull/446.
3. This branch + pull request replace any previous pull requests and magic.dmh branch.

Given an otherwise valid netCDF file that has a corrupted header,
the netcdf library currently crashes. Instead, it should return
NC_ENOTNC.

Additionally, the NC_check_file_type code does not do the
forward search required by hdf5 files. It currently only looks
at file position 0 instead of 512, 1024, 2048,... Also, it turns
out that the HDF4 magic number is assumed to always be at the
beginning of the file (unlike HDF5).
The change is localized to libdispatch/dfile.c See
https://support.hdfgroup.org/release4/doc/DSpec_html/DS.pdf

Also, it turns out that the code in NC_check_file_type is duplicated
(mostly) in the function libsrc4/nc4file.c#nc_check_for_hdf.

This branch does the following.
1. Make NC_check_file_type return NC_ENOTNC instead of crashing.
2. Remove nc_check_for_hdf and centralize all file format checking
   NC_check_file_type.
3. Add proper forward search for HDF5 files (but not HDF4 files)
   to look for the magic number at offsets of 0, 512, 1024...
4. Add test tst_hdf5_offset.sh. This tests that hdf5 files with
   an offset are properly recognized. It does so by prefixing
   a legal file with some number of zero bytes: 512, 1024, etc.
5. Off-topic: Added -N flag to ncdump to force a specific output dataset name.
2017-10-24 16:25:09 -06:00
Ed Hartnett
a7d29db7b5 fixed memory leak in nc_open when cmode includes both NC_MPIIO and NC_MPIPOSIX 2017-10-24 09:46:08 -06:00
Dennis Heimbigner
c05c0a69e2 Update docs/internal.dox to
provide a more accurate description
of how to add a new dispatch table.
Also remove dispatch.dox as superfluous.
2017-10-04 13:41:38 -06:00
Ward Fisher
1a56d3fdc8 Making cdf5 tests conditional on cdf5 support setting at configure time. 2017-09-14 14:18:56 -06:00
Ward Fisher
f82bcf9d15 Added USE_CDF5 to dfile.c 2017-09-14 12:38:48 -06:00
Ward Fisher
8ab714f1d8 Corrected a missing endif. 2017-08-07 10:23:15 -06:00
Ward Fisher
05d3d7e9f5 Merge remote-tracking branch 'origin/d4tempfile.dmh' into v4.5.0-release-branch 2017-08-07 10:12:36 -06:00
Dennis Heimbigner
4d8fbbf501 If DAP (2 or 4) is enabled, but diskless is disabled, then the
dap code will create a real temporary file in which to store the
converted metadata for the DAP .dds or .dmr.

It was assumed that the nc_close code would reclaim the
temporary file. For DAP2, reclamation occurs in the ncio
code. For DAP4, it was assumed that the libsrc4 code would do
the reclamation, but for whatever reason, this is not happening.
Thus, in this situation, a temporary file is left in the file
system. Aside from being irritating to users, this screws up
'make distcheck'.

So the DAP4 code is fixed to ensure that the temporary file is
properly reclaimed independent of the libsrc4 code.
2017-08-05 22:41:31 -06:00
Dennis Heimbigner
715a6fe5eb The files libdispatch/dwinpath.c and include/ncwinpath.h
were added to provide a path name converter from e.g. cygwin
paths to e.g. windows paths. This is necessary because
the shell scripts may produce cygwin paths, but the code
may have been compiled with Visual Studio. Similar issues
arise with Mingw.

At appropriate places, and if using Visual Studio or Mingw,
I added calls to the path conversion code.
Apparently I forgot to find all the places where this
conversion was needed. So this pr does the following:
1. Push the calls to the converter to the various libXXX
   directories and out of libdispatch/dfile.c.
2. Add conversion calls to other parts of the code like oc2.

I also turns out that conversion code in dapcvt.c
had a bug when handling DAP Byte type under visual studio.

Notes:
1. there may still be places I missed that need to do path conversion.
2. need to make sure that calls to e.g. H5open also use converted path.
2017-07-13 10:40:07 -06:00
Ward Fisher
68f5e3a2d6 No idea how this was possibly working, previously. 2017-05-23 16:55:29 -06:00
Ward Fisher
42ba263377 Corrected an else statement. 2017-05-12 15:49:41 -06:00
Alexander Mohr
90609f60ab fix typo 2017-04-27 22:38:18 -07:00
Alexander Mohr
674d212e6b fix nc_open_mem 2017-04-27 22:26:19 -07:00
Dennis Heimbigner
32fe709615 ckp 2017-04-14 11:05:30 -06:00
Dennis Heimbigner
3db4f013bf Primary change: add dap4 support
Specific changes:
1. Add dap4 code: libdap4 and dap4_test.
   Note that until the d4ts server problem is solved, dap4 is turned off.
2. Modify various files to support dap4 flags:
	configure.ac, Makefile.am, CMakeLists.txt, etc.
3. Add nc_test/test_common.sh. This centralizes
   the handling of the locations of various
   things in the build tree: e.g. where is
   ncgen.exe located. See nc_test/test_common.sh
   for details.
4. Modify .sh files to use test_common.sh
5. Obsolete separate oc2 by moving it to be part of
   netcdf-c. This means replacing code with netcdf-c
   equivalents.
5. Add --with-testserver to configure.ac to allow
   override of the servers to be used for --enable-dap-remote-tests.
6. There were multiple versions of nctypealignment code. Try to
   centralize in libdispatch/doffset.c and include/ncoffsets.h
7. Add a unit test for the ncuri code because of its complexity.
8. Move the findserver code out of libdispatch and into
   a separate, self contained program in ncdap_test and dap4_test.
9. Move the dispatch header files (nc{3,4}dispatch.h) to
   .../include because they are now shared by modules.
10. Revamp the handling of TOPSRCDIR and TOPBUILDDIR for shell scripts.
11. Make use of MREMAP if available
12. Misc. minor changes e.g.
	- #include <config.h> -> #include "config.h"
	- Add some no-install headers to /include
	- extern -> EXTERNL and vice versa as needed
	- misc header cleanup
	- clean up checking for misc. unix vs microsoft functions
13. Change copyright decls in some files to point to LICENSE file.
14. Add notes to RELEASENOTES.md
2017-03-08 17:01:10 -07:00
Ward Fisher
f19f1cc6c1 Tweaked fix. 2017-01-09 11:57:23 -07:00
Ward Fisher
7d75fd3731 Updated code, location of variable declarations in support of https://github.com/Unidata/netcdf-c/issues/304 2017-01-09 11:37:35 -07:00
Ward Fisher
ba21beb3b5 Brought over rest of old documentation for nc_def_var_chunking in support of https://github.com/Unidata/netcdf-c/issues/318 2016-09-30 16:35:16 -06:00
Ward Fisher
4e46a31088 Corrected an issue reported in https://github.com/Unidata/netcdf-c/issues/317 2016-09-13 13:55:33 -06:00
Ward Fisher
5a7d5c6eca Merge branch 'substrate' 2016-04-08 11:16:37 -06:00
dmh
764a1c40a3 ckp 2016-04-06 19:51:40 -06:00
Ward Fisher
6409b11640 Modified nc_inq_type so that it will return valid information for valid data types even if the associated ncid doesn't have access to those data types. See https://github.com/Unidata/netcdf-c/issues/240 for more information. 2016-04-05 20:13:42 +00:00
Ward Fisher
b19b807e8b Working around a couple of problematic calls to fstat in support of https://github.com/Unidata/netcdf-c/issues/188 . No idea why these were a problem in VS14 (Visual Studio 2015) and not in VS12 (Visual Studio 2013). It's possible there are other, similar issues out there. We'll see what turns up when we run all the tests. These changes were tested against nc_test4/tst_large2.c 2016-03-02 13:03:29 -07:00
Ward Fisher
fbdea5c4a0 Working in support of https://github.com/Unidata/netcdf-c/issues/188 . Exploratory work. 2016-02-29 15:08:09 -07:00
Ward Fisher
30e1cca0d0 Reorganized file/datastream IO functions in documentation. 2016-01-14 15:50:50 -07:00
Ward Fisher
3f27ec6db4 Tightening the Doxygen documentation. 2016-01-14 15:33:50 -07:00
Ward Fisher
612b35a84c Merge branch 'master' into cdf-5, in preparation for merging the CDF-5 functionality into the master branch. This will be the key new feature for netcdf 4.4.0. 2015-11-05 13:40:35 -07:00
dmh
7480d7d97d - clean up the dfile.c confusion about NC_64_BIT_OFFSET
- sync oc2 with  master for https://DennisHeimbigner@github.com/Unidata/oc.git
- Cleanup auth.html documentation.
- Cleanup obsolete documentation.
2015-10-24 21:45:13 -06:00