Dennis Heimbigner
d2316f866c
Additional Fixes to NCZarr
...
Primary Fixes:
* Add a whole variable optimization -- used in the rare case that nc_get/put_vara covers the whole of a variable and the variable has a single chunk.
* Fix chunking error when stride causes whole chunks to be skipped.
* Fix some memory leaks
* Add test cases
* Add one performance test to nczarr_test/. This uses the timer utils from unit_test: timer_utils.[ch].
* Move ncdumpchunks utility from ncdump to nczarr_test
Misc. Other Changes:
* Make check for aws libraries conditional on --enable-nczarr-s3
* Remove all but one bm tests from nczarr_test until they are working.
* Remove another dependency on HDF5 from supposedly non-HDF5 specific code; specifically hdf5_log_hdf5.
* Make the BAIL2 macro be hdf5 specific and replace elsewhere with an HDF5 independent equivalent.
* Move hdf5cache.c to libsrc4/nc4cache.c because it is used by nczarr.
* Modify unit_tests so that some of them are run even if using Windows.
* Misc. small bug fixes and refactors and memory leaks.
* Rename some conflicting tests for cmake.
* Attempted to make nc_perf work with cmake and failed.
2020-12-16 20:48:02 -07:00
Edward Hartnett
98088adb46
getting tst_gfs_data_1 ready to merge
2020-07-08 14:20:42 -06:00
Edward Hartnett
2571552686
changed output to be more readable
2020-07-07 11:25:49 -06:00
Edward Hartnett
2dad8d1bff
now checking lat/lon coord values
2020-07-07 09:04:07 -06:00
Edward Hartnett
d539e03c68
further checking of metadata
2020-07-07 09:01:38 -06:00
Edward Hartnett
d13035b01e
further checking of metadata
2020-07-07 08:58:38 -06:00
Edward Hartnett
a13c0a2d23
starting to use get_vara functions to check metadata
2020-07-07 08:49:12 -06:00
Edward Hartnett
f9e753495e
more checking of metadata and coord vars
2020-07-07 08:04:06 -06:00
Edward Hartnett
6b2820ca95
now checking some coord vars
2020-07-06 08:26:35 -06:00
Edward Hartnett
661cde5203
more checking metadata
2020-07-06 08:14:15 -06:00
Edward Hartnett
7b477784f5
starting to check metadata
2020-07-06 07:53:11 -06:00
Edward Hartnett
8faa7ed946
added new metadata writing function
2020-07-05 15:57:07 -06:00
Edward Hartnett
a9b5c33ba7
dealing with coord var cleanup
2020-07-05 15:33:26 -06:00
Edward Hartnett
6e2a3cb3e3
cleaned up grid code
2020-07-05 14:27:19 -06:00
Edward Hartnett
aa1af2f456
cleaned up lat/lon code
2020-07-05 12:23:27 -06:00
Edward Hartnett
4a80190c64
code cleanup
2020-07-05 08:54:50 -06:00
Edward Hartnett
86c39cf7a2
working on 2D decomposition code
2020-07-04 10:45:33 -06:00
Edward Hartnett
7a9397aed8
cleanup of 2d decomposition
2020-07-04 10:31:12 -06:00
Edward Hartnett
b72dee72a1
starting to add decomp for 36 tasks
2020-07-02 09:37:55 -06:00
Edward Hartnett
ab25b62892
more benchmark development
2020-07-01 15:33:48 -06:00
Edward Hartnett
5b14cc1496
added attributes
2020-07-01 14:48:10 -06:00
Edward Hartnett
fd4e388002
added attributes
2020-07-01 14:47:59 -06:00
Edward Hartnett
527589a0d7
now create metadata in a function with many parameters
2020-07-01 13:30:24 -06:00
Edward Hartnett
3f6a367381
further development
2020-07-01 12:26:12 -06:00
Edward Hartnett
0568ca5e67
move data decomposition calculations to decomp_4D() function
2020-07-01 09:46:47 -06:00
Edward Hartnett
010c1b2766
fixed Makefile to refrain from building nc_perf/tst_gfs_data_1 for non-parallel builds
2020-07-01 08:28:58 -06:00
Edward Hartnett
1c8a361a62
more benchmark development for tst_gfs_data_1.c
2020-07-01 07:10:43 -06:00
Edward Hartnett
9a75e1a471
further test development
2020-06-30 14:31:54 -06:00
Edward Hartnett
3508eba09e
better handling for multiple data vars
2020-06-30 13:36:57 -06:00
Edward Hartnett
8029db1d4f
further benchmark development, moved to benchmark dir
2020-06-30 13:20:28 -06:00
Edward Hartnett
16a7b34c61
fixed warnings in bm_file.c
2020-06-30 10:42:28 -06:00
Edward Hartnett
ba6ab38a11
fixed warnings in tst_create_files.c
2020-06-30 10:40:05 -06:00
Edward Hartnett
af6b6787bf
fix for memory leak due to HDF5 types
2020-02-09 11:47:13 -07:00
Edward Hartnett
555b9384fc
turned down number of runs
2020-02-09 11:24:19 -07:00
Edward Hartnett
23656e31ed
further development of mem testing program
2020-02-09 09:04:23 -07:00
Edward Hartnett
2d3f4be2ea
now detect memory leak open/closing files with tst_mem1.c
2020-02-09 08:51:11 -07:00
Edward Hartnett
8ffa18c385
adding tst_mem1.c
2020-02-09 08:44:02 -07:00
Ward Fisher
438119dd69
Merge pull request #1560 from NOAA-GSD/ejh_cache_docs
...
increase default cache size for netCDF-4/HDF5 files, also improve cache docs and add benchmarking program
2020-01-07 11:46:30 -07:00
Ward Fisher
e0717731b9
Merge pull request #1555 from NOAA-GSD/ejh_bm_file
...
update of benchmark program bm_file to prevent integer overflows on very big files
2020-01-07 11:45:22 -07:00
Edward Hartnett
2682ffd68d
improved docs for cache functions, added libhdf5/hdf5cache.c to Doxyfile.in, added benchmark program for cache settings
2019-11-25 16:33:04 -07:00
Edward Hartnett
ee0f2a1b50
update of bm_file to prevent integer overflows on very big files
2019-11-22 12:51:35 -07:00
Edward Hartnett
b439490995
fixed nc_perf Makefile.am to include benchmark scripts in EXTRA_DIST
2019-11-19 09:21:00 -07:00
Greg Sjaardema
56c0d5cf8a
Spelling fixes
2019-09-18 08:03:01 -06:00
Dennis Heimbigner
7901353cf5
Restore nc_perf/CMakeLists.txt
2019-05-25 12:15:56 -06:00
Dennis Heimbigner
8d11b80df3
fix nc_perf/CMakeLists.txt problem
2019-05-24 20:43:33 -06:00
Dennis Heimbigner
06498ff16a
various fixes
2019-05-23 16:35:03 -06:00
Dennis Heimbigner
62feacee00
missing cmake file
2019-04-29 20:55:28 -06:00
Dennis Heimbigner
62e2b472b4
Minor config.h changes to support filters in Fortran
2019-04-29 16:36:08 -06:00
Ward Fisher
c776039eba
Updated call to NC4_read_provenance.
2019-04-18 10:53:16 -06:00
Ed Hartnett
e7d0a7b671
change to trigger CI
2019-03-21 13:35:49 -06:00
