Commit Graph

113 Commits

Author SHA1 Message Date
Scot Breitenfeld
7b1b06b5ca Merge remote-tracking branch 'upstream/master' 2020-04-23 15:36:14 -05:00
Dennis Heimbigner
b0e0d81aa9 Fix reclamation of the ->format_XXX_info fields
nc4internal.c contains code to free the format_XXX_info
fields. Since these are format specific, this code
was moved to the dispatch code (libhdf5 and libhdf4
in the current case).

Additionally, there are some fields in nc4internal.h (e.g.
dimscale fields) that are specific to HDF5 and have been moved
to the corresponding HDF5 data structures and code.

Misc. other changes:
1. NC_VAR_INFO_T->hdf5_name renamed to alt_name to avoid
   implying it is necessarily HDF5 specific.
2. prefix NC_FILE_INFO_T with an instance of NC_OBJ for consistency.
   this also requires wrapping move_in_NCList() to keep
   hdr.id consistent.
2020-03-29 12:48:59 -06:00
Scot Breitenfeld
c5d2e99417 Updated to use H5O_info2_t for HDF5 1.12 and the use of H5Oget_info3 instead of H5Gget_objinfo 2020-03-12 15:50:24 +00:00
Edward Hartnett
053752440b stop setting contiguous field in nc4hdf5.c 2020-03-08 07:18:52 -06:00
Edward Hartnett
1761850795 continuing to switch to storage field 2020-03-08 07:05:51 -06:00
Edward Hartnett
b98a37e0b3 using storage field in nc4var.c 2020-03-08 06:38:44 -06:00
Dennis Heimbigner
b488c272d5 Fix conflicts with master 2020-02-27 14:06:45 -07:00
Dennis Heimbigner
44d0dcaad2 Add support for multiple filters per variable.
re: https://github.com/Unidata/netcdf-c/issues/1584

Support has been added for multiple filters per variable.  This
affects a number of components in netcdf. The new APIs are
documented in NUG/filters.md.

The primary changes are:
* A set of new functions are provided (see __include/netcdf_filter.h__).
    - Obtain a list of the filters associated with a variable
    - Obtain the parameters for a specific filter.
* The existing __nc_inq_var_filter__ function now returns info
  about the first defined filter.
* The utilities (ncgen, ncdump, and nccopy) now support
  an extended format for specifying a sequence of filters.
  The general form is __<filter>|<filter>..._.
* The ncdump **_Filter** attribute now dumps a list of all the
  filters associated with a variable using the above new format.
* Filter specifications can now use a filter name instead of number
  for filters known to the netcdf library, which in turn is taken
  from the HDF5 filter registration page.
* New errors are defined: NC_EFILTER and NC_ENOFILTER. The latter
  is returned if an attempt is made to access an unknown filter.
* Internally, the dispatch table has been extended to add a function
  to handle all of the filter functions.
* New, filter-related, tests were added to nc_test4.
* A new plugin was added to the plugins directory to help with testing.

Notes:
1. The shuffle and fletcher32 filters are not part of the multifilter system.

Misc. changes:
1. A debug module was added to libhdf5 to help catch error locations.
2020-02-16 12:59:33 -07:00
Edward Hartnett
1817790c6b rely completely on nc_def_var_filter for setting szip 2020-02-05 10:32:16 -07:00
Ward Fisher
aadd5a2d81
Merge pull request #1589 from NOAA-GSD/ejh_szip
re-implement the nc_def_var_szip() function, including for parallel I/O
2020-01-22 16:27:33 -07:00
Edward Hartnett
6103f442cb fixed compile error 2020-01-21 08:50:38 -07:00
Edward Hartnett
c839a2d6c5
Merge branch 'master' into ejh_var_cache 2020-01-21 07:48:59 -07:00
Edward Hartnett
184507be5f now using members in NC_VAR_INFO_T to hold szip info 2020-01-06 08:46:03 -07:00
Edward Hartnett
bacf017699 better handling of var cache for parallel builds 2019-12-17 06:39:09 -07:00
Edward Hartnett
06896f432d got compact storage test working 2019-12-04 08:49:37 -07:00
Edward Hartnett
82df2876b6 starting to support compact storage 2019-12-04 07:53:37 -07:00
edwardhartnett
8083b3596e fixed problem of unlim dim and var sharing the same name but not being related 2019-11-15 09:18:42 -07:00
Constantine Khrulev
6abbf8d429 Whitespace changes 2019-11-13 10:07:10 -09:00
Constantine Khrulev
098f2c1056 Modify the condition used to check if an attribute can be re-used
This should make the code a bit cleaner.
2019-11-13 08:38:12 -09:00
Constantine Khrulev
dd181deca9 Improve the fix for #350 included in #1119
1) We have to use H5Tequal() to compare HDF5 type IDs.
2) When checking if we can re-use an NC_CHAR attribute it is enough to
   compare data types (H5Tequal() takes care of the size comparison).
3) This commit adds missing code (reuse_att was set but not used).

Now an attribute in a NetCDF-4 file can be modified as many times as
necessary, as long its type and length remain the same.

Modifications changing either type or length of an attribute require
deleting and re-creating an attribute which increments the attribute
order creation index. Once this index reaches 65535 all attribute
modifications (for a particular group or variable) will fail.

For reference:

Issue 350 title: NetCDF-4 limits the number of times an attribute can
be modified

Pull request 1119 title: Fix checking for HDF5 max dims, no longer
re-create atts if not needed, confirm behavior for HDF5 cyclical
files, allow user to set mpiexec
2019-11-12 21:45:47 -09:00
Greg Sjaardema
56c0d5cf8a Spelling fixes 2019-09-18 08:03:01 -06:00
Dennis Heimbigner
8d0bced60d Allow in-line definition of filters
Priority: Low

re: issue https://github.com/Unidata/netcdf-c/issues/1329

HDF5 has the ability to programmatically define new filters,
as opposed to using HDF5_PLUGIN_PATH env variable.
This PR adds support for that feature.
Not clear how useful this is, though.
See docs/filters.md for details.
2019-03-21 11:33:27 -06:00
Ed Hartnett
b1d30a0f67 cleanup of whitespace in HDF5 directory 2019-02-19 05:18:53 -07:00
Ed Hartnett
8e6f38b099 detecting conditions for mandatory rename of vars with varid > renamed var 2019-02-03 06:35:29 -07:00
Ed Hartnett
1dd76c996e added tst_rename3.c for more rename testing 2019-02-02 05:53:45 -07:00
Ed Hartnett
e74ec6f2a0 made function give_var_secret_name() not static, fixed warning 2019-01-27 11:10:41 -07:00
Ed Hartnett
627a55cf78 added function give_var_secret_name() 2019-01-27 11:06:02 -07:00
Ed Hartnett
42c64598dc created function create_dim_wo_var() 2019-01-27 10:51:25 -07:00
Ed Hartnett
60132a0ed7 made function static, removed unneeded if statement 2019-01-20 09:53:00 -07:00
Ed Hartnett
9d40e0a2af made function static, removed unneeded if statement 2019-01-20 09:52:42 -07:00
Ed Hartnett
029aa5f626 now using hidden coordinates att to speed file opens 2018-12-20 05:59:31 -07:00
Ed Hartnett
6e3d284fcc write coordinates hidden attribute for all variables 2018-12-19 09:16:21 -07:00
Ed Hartnett
8ca5a1ac17
Merge branch 'master' into ejh_fast_var_prep 2018-12-12 07:05:45 -07:00
Ward Fisher
30ea33435c Merge remote-tracking branch 'origin/license_update.wif' into pr-aggregation.wif 2018-12-11 17:08:21 -05:00
Ed Hartnett
28aa22f8cf cleanup and some comments for rec_write_metadata() 2018-12-11 09:33:46 -07:00
Ed Hartnett
8e1d781ea3 fixed error handling in write_dim(), also converted dimscale creation to H5Dcreate2() 2018-12-11 09:28:23 -07:00
Ed Hartnett
1a0cfb729e fixed error handling in write_var() 2018-12-11 08:10:08 -07:00
Ed Hartnett
26239a0897 cleaned up loop reattaching dimscales 2018-12-11 07:21:09 -07:00
Ed Hartnett
7b72d0b832 fixed error handling in attach_dimscales() 2018-12-11 07:15:46 -07:00
Ed Hartnett
39a0f822fa changed loop to use ncindexlookup() instead of looping through the names 2018-12-11 07:07:00 -07:00
Ed Hartnett
d7b657af4e cleaned up comments, changed loop to use ncindexlookup() instead of looping through the names 2018-12-11 06:43:27 -07:00
Ed Hartnett
53bdb74869 fixed error handling of write_netcdf4_dimid() 2018-12-11 06:38:23 -07:00
Ed Hartnett
3c9a141ee3 moved function detect_preserve_dimids and made it static 2018-12-11 06:15:47 -07:00
Ward Fisher
87110139f6 More synchronization of copyright stanzas. 2018-12-06 14:34:40 -07:00
Ed Hartnett
c824ad9ec9 further removal of libhdf5 code from libsrc4 2018-11-28 15:48:08 -07:00
Ed Hartnett
64b45b5e75 changes in support of separating HDF5 and libsrc4 2018-11-26 04:40:33 -07:00
Ed Hartnett
f6c3093fb3 moved convert function to libsrc4 2018-11-21 14:49:52 -07:00
Ed Hartnett
ab963e3d41 removing HDF5 type info from libsrc4 2018-11-20 14:24:40 -07:00
Ed Hartnett
52d58ecd2e changing over native_hdf_typeid 2018-11-20 10:49:26 -07:00
Ed Hartnett
5104262f6b changing types 2018-11-20 08:00:48 -07:00