Commit Graph

327 Commits

Author SHA1 Message Date
Ed Hartnett
5b56e66096 removed changes to nc4hdf5.c 2018-05-24 15:26:17 -06:00
Ed Hartnett
e0f92efaa2 fixed logging 2018-05-24 14:55:34 -06:00
Ed Hartnett
d26df3fb27 comment cleanup 2018-05-24 14:43:00 -06:00
Ed Hartnett
f2cb4678ee moving HDF5 functions to libhdf5 2018-05-24 14:27:16 -06:00
Ed Hartnett
0c0da02b49
Merge branch 'master' into ejh_comm_info 2018-05-24 13:39:42 -06:00
Dennis Heimbigner
ee509ff4f3 Re-Implement the nc_get/put_vars operations for netcdf-4 using the
corresponding HDF5 operations.

re: github issue https://github.com/Unidata/netcdf-c/issues/908
also in reference to https://github.com/pydata/xarray/issues/2004

The netcdf-c library has implemented the nc_get_vars and nc_put_vars
operations as element at a time. This has resulted in very slow
operation.

This pr attempts to improve the situation for netcdf-4/hdf5 files
by using the slab operations provided by the hdf5 library. The new
implementation passes the get/put vars stride information down to
the hdf5 slab operations.

The result appears to improve performance significantly. Some simple
tests on large 2-D arrays shows speedups in excess of 150.

Misc. other changes:
1. fix bug in ncgen/semantics.c; using a list's allocated length
   instead of actual length.
2. Added a temporary hook in the netcdf library plus a performance
   test case (tst_varsperf.c) to estimate the speedup. After users
   have had some experience with this, I will remove it, probably
   after the 4.7 release.
2018-05-22 16:50:52 -06:00
Ed Hartnett
29427ef3ec
Merge branch 'master' into ejh_comm_info 2018-05-18 05:32:53 -06:00
Ed Hartnett
fb6d19c3bb
Merge branch 'master' into ejh_hdf5_2 2018-05-18 05:32:09 -06:00
Ed Hartnett
7f4717bf97 removed more dead code 2018-05-17 05:58:46 -06:00
Ed Hartnett
7552e1588e
Merge branch 'master' into ejh_clean 2018-05-17 04:24:38 -06:00
Ward Fisher
3f83cc4926 Merge branch 'master' into inmem2.dmh 2018-05-16 14:24:19 -06:00
Ed Hartnett
7b41bded0f fixed comments 2018-05-16 10:24:51 -06:00
Ed Hartnett
72a7ddb15a changes from other branches to get CI to pass 2018-05-16 10:03:24 -06:00
Ed Hartnett
485f42a51b no longer allow default communicator of MPI_COMM_WORLD for open 2018-05-16 09:27:40 -06:00
Ed Hartnett
978fc99f3f fixed indentation 2018-05-16 08:35:32 -06:00
Ed Hartnett
2ff0e14765 removed unused function 2018-05-16 08:31:23 -06:00
Ed Hartnett
40c552bd43 cleaned up nc4hdf5.c 2018-05-16 03:18:50 -06:00
Ed Hartnett
89cc8a9cd3 fixed memory leak issue in nc4file.c with inmemory 2018-05-16 02:48:44 -06:00
Ed Hartnett
2e181fbb88 fixed cmake build 2018-05-15 07:12:56 -06:00
Ed Hartnett
d7f7c1465b merged ininmem2 branch to pass CI 2018-05-15 06:53:28 -06:00
Ed Hartnett
8e7a617c98 added files 2018-05-15 06:51:44 -06:00
Ed Hartnett
3e320a5bfb moved more HDF5 functions to libhdf5 2018-05-15 06:47:52 -06:00
Ed Hartnett
57e378a2a0 merged master 2018-05-14 17:16:28 -06:00
Ed Hartnett
af70577e6d merged master 2018-05-09 14:43:42 -06:00
Ed Hartnett
a68f57a0e5 added missing files 2018-05-08 12:20:55 -06:00
Ed Hartnett
c402b79ed8 merged changes from master 2018-05-08 12:13:03 -06:00
Ed Hartnett
5526ca65d1 created libhdf5, moved some files 2018-05-08 11:58:01 -06:00