mirror/netcdf-c
2
0
Fork 0
mirror of https://github.com/Unidata/netcdf-c.git synced 2024-11-21 03:13:42 +08:00
Code Issues Packages Projects Releases Wiki Activity
f29cff2408
netcdf-c/docs/tutorial.dox
Edward Hartnett 6849f44896 more doxygen warning fixes
2022-05-03 12:54:14 -06:00

993 lines
40 KiB
Plaintext
Raw Blame History

/*! \page tutorial.dox The NetCDF-C Tutorial
\brief The NetCDF-C Tutorial
\tableofcontents
\section sec_tut Tutorial Documentation
This page contains references to various other NetCDF background and tutorial pages.
\subsection tutorial_pages Tutorial Pages
- \subpage netcdf_working_with_netcdf_files
- \subpage tutorial_ncids
- \subpage creating
- \subpage reading_known
- \subpage reading_unknown
- \subpage accessing_subsets
\subsection background_further_reading Background and Further Reading
- \ref what_is_netcdf
- \subpage netcdf_documentation
\subsection sub_sec_netcdf_data_model NetCDF Data Model:
- \subpage netcdf_data_model
- \ref classic_model
- \ref enhanced_model
- \subpage unlimited_dims
- \subpage groups
- \subpage user_defined_types
- \subpage string_type
\subsection sub_sec_important_conventions Important Conventions:
- \subpage fill_values
\subsection sub_sec_tools_for_netcdf Tools for NetCDF:
- \ref netcdf_utilities
- \ref external_netcdf_tools
\subsection sub_sec_programming_with_netcdf Programming with NetCDF:
- \subpage netcdf_apis
- \subpage error_handling
- \subpage interoperability_hdf5
- \subpage parallel_io
\subsection sub_sec_example_programs Example Programs:
- \subpage examples1
\page netcdf_working_with_netcdf_files Working with NetCDF Files from the command line.
\brief Options for working with netcdf-formatted files from the command line or with an external program.
\tableofcontents
\section netcdf_utilities The NetCDF Utilities
The ncdump command reads a netCDF file and outputs text in a format
called CDL. The ncgen command reads a text file in CDL format, and
generates a netCDF data file. The nccopy command copies a netCDF file
from one binary format to another, optionally changing compression and
chunksize settings.
\subsection netcdf_utilities_ncdump ncdump
The utility ncdump can be used to show the contents of netCDF
files. By default, ncdump shows the CDL description of the file. This
CDL description can be fed into ncgen to create the data file.
See also: [The ncdump User's Guide](https://docs.unidata.ucar.edu/nug/current/netcdf_utilities_guide.html#ncdump_guide)
\subsection netcdf_utilities_ncgen ncgen
The ncgen utility can take an ASCII input file, in CDL format, and
generate a binary netCDF file. It is the opposite of ncdump.
See also:
- [ncgen](https://docs.unidata.ucar.edu/nug/current/netcdf_utilities_guide.html#guide_ncgen)
- [ncgen3](https://docs.unidata.ucar.edu/nug/current/netcdf_utilities_guide.html#guide_ncgen3)
\subsection netcdf_utilities_nccopy nccopy
The nccopy utility can copy a netCDF file, changing binary format,
chunk sizes, compression, and other storage settings.
See also: [nccopy](https://docs.unidata.ucar.edu/nug/current/netcdf_utilities_guide.html#guide_nccopy)
\section external_netcdf_tools Tools for Manipulating NetCDF Files
Many existing 3rd-party software applications can read and manipulate netCDF
files. Before writing your own program, check to see if any existing
programs meet your needs.
Three utilities come with the netCDF distribution: ncdump, ncgen, and
nccopy. (See \ref netcdf_utilities).
The following general-purpose tools have been found to be useful in
many situations. Some of the tools on this list are developed at
Unidata. The others are developed elsewhere, and we can make no
guarantees about their continued availability or success. All of these
tools are open-source.
- <a href="https://www.unidata.ucar.edu/software/udunits">UDUNITS</a> - Unidata library to help with scientific units.
- <a href="https://www.unidata.ucar.edu/software/idv">IDV</a> - Unidata's Integrated Data Viewer, a 3D visualization and analysis package (Java based).
- <a href="http://www.ncl.ucar.edu">NCL</a> - NCAR Command Language, a graphics and data manipulation package.
- <a href="http://grads.iges.org/grads/grads.html">GrADS</a> - The Grid Analysis and Display System package.
- <a href="http://nco.sourceforge.net">NCO</a> - NetCDF Command line Operators, tools to manipulate netCDF files.
A <a href="https://www.unidata.ucar.edu/netcdf/software.html">list of
netCDF tools</a> that we know about can be found on the website. If
you know of any that should be added to this list, send email to
support-netcdf@unidata.ucar.edu.
\page netcdf_apis The NetCDF Programming APIs
Unidata supports netCDF APIs in C, C++, Fortran 77, Fortran 90, and
Java.
The <a href="https://www.unidata.ucar.edu/software/netcdf-java/">netCDF
Java</a> API is a complete implementation of netCDF in Java. It is
distributed independently of the other APIs. If you are writing web
server software, you should certainly be doing so in Java.
The main netCDF distribution contains the C library and the netCDF
utilities, ncgen/ncdump/nccopy.
The C++, Fortran 77 and Fortran 90 APIs are distributed separately
from the C library. The C library must be installed before any of
these APIs may be built. They depend on the C API.
Due to the nature of C++ and Fortran 90, users of those languages can
also use the C and Fortran 77 APIs (respectively) directly.
Full documentation exists for each API (see \ref
netcdf_documentation).
In addition, many other language APIs exist, including Perl, Python,
and Ruby. Most of these APIs were written and supported by netCDF
users. Some of them are listed on
the <a href="https://www.unidata.ucar.edu/netcdf/software.html">netCDF
software page</a>. Since these generally use the C API, they should
work well with netCDF-4/HDF5 files, but the maintainers of the APIs
must add support for netCDF-4 advanced features.
In addition to the main netCDF-3 C API, there is an additional (older)
C API, the netCDF-2 API. This API produces exactly the same files as
the netCDF-3 API - only the API is different. That is, users can
create either classic CDF-1 format files, the default, 64-bit offset
files (CDF-2), 64-bit data files (CDF-5), or netCDF-4/HDF5 files.
The version 2 API was the API before netCDF-3.0 came out. It is still
fully supported, however. Programs written to the version 2 API will
continue to work. The version 2 API is built with the netCDF library
by default.
Users writing new programs should use the netCDF-3 API, which contains
better type checking, better error handling, and better documentation.
The netCDF-2 API is provided for backward compatibility. Documentation
for the netCDF-2 API can be found on the netCDF website, see
https://www.unidata.ucar.edu/netcdf/old_docs/really_old/guide_toc.html.
\page netcdf_documentation NetCDF Documentation
Language specific programming guides are available for netCDF for the
C, C++, Fortran 77, Fortran 90, and Java APIs:
- C - The NetCDF C Interface Guide.
- C++ - The NetCDF C++ Interface Guide.
- Fortran 77 - The NetCDF Fortran 77 Interface Guide.
- Fortran 90 - The NetCDF Fortran 90 Interface Guide.
- Java - <a href="https://docs.unidata.ucar.edu/netcdf-java/current/userguide/">The netCDF-Java User Guide</a>.
Man pages for the C, F77, and F90 interfaces, and ncgen and ncdump,
are available on the documentation page of the netCDF web site
(https://docs.unidata.ucar.edu/netcdf-c), and are installed with the
libraries.
The latest version of all netCDF documentation can always be found at
the <a href="https://docs.unidata.ucar.edu/netcdf-c">netCDF
documentation page</a>.
\page netcdf_data_model The NetCDF Data Model
\tableofcontents
The netCDF data model is the way that we think about data. The data
model of dimensions, variables, and attributes, which define the \ref
classic_model, was extended starting with netCDF-4.0. The new \ref
enhanced_model supports the classic model in a completely
backward-compatible way, while allowing access to new features such as
groups, multiple unlimited dimensions, and new types, including
user-defined types.
For maximum interoparability with existing code, new data should be
created with the \ref classic_model.
<p>\image html nc4-model.png "The NetCDF Enhanced Data Model"
\section classic_model The Classic Model
The classic netCDF data model consists of variables, dimensions, and
attributes. This way of thinking about data was introduced with the
very first netCDF release, and is still the core of all netCDF files.
<p>
\image html nc-classic-uml.png "NetCDF Classic Data Model"
In version 4.0, the netCDF data model has been expanded. See \ref
enhanced_model.
<table>
<tr><td>
Variables
</td><td>
N-dimensional arrays of data. Variables in netCDF files can be one
of six types (char, byte, short, int, float, double).
</td></tr>
<tr><td>
Dimensions
</td><td>
describe the axes of the data arrays. A dimension has a name and a
length. An unlimited dimension has a length that can be expanded at
any time, as more data are written to it. NetCDF files can contain at
most one unlimited dimension.
</td></tr>
<tr><td>
Attributes
</td><td>
annotate variables or files with small notes or supplementary
metadata. Attributes are always scalar values or 1D arrays, which can
be associated with either a variable or the file as a whole. Although
there is no enforced limit, the user is expected to keep attributes
small.
</td></tr>
</table>
\section enhanced_model The Enhanced Data Model
With netCDF-4, the netCDF data model has been extended, in a backwards
compatible way.
The new data model, which is known as the “Common Data Model” is part
of an effort here at Unidata to find a common engineering language for
the development of scientific data solutions. It contains the
variables, dimensions, and attributes of the classic data model, but
adds:
<ul>
<li>groups - A way of hierarchically organizing data, similar to
directories in a Unix file system.
<li>user-defined types - The user can now define compound types
(like C structures), enumeration types, variable length arrays, and
opaque types.
</ul>
These features may only be used when working with a netCDF-4/HDF5
file. Files created in classic formats cannot support groups or
user-defined types (see [NetCDF File
Formats](https://docs.unidata.ucar.edu/nug/current/netcdf_introduction.html#netcdf_format)).
<p>\image html nc4-model.png
With netCDF-4/HDF5 files, the user may define groups, which may
contain variables, dimensions, and attributes. In this way, a group
acts as a container for the classic netCDF dataset. But netCDF-4/HDF5
files can have many groups, organized hierarchically.
Each file begins with at least one group, the root group. The user may
then add more groups, receiving a new ncid for each group created.
Since each group functions as a complete netCDF classic dataset, it is
possible to have variables with the same name in two or more different
groups, within the same netCDF-4/HDF5 data file.
Dimensions have a special scope: they may be seen by all variables in
their group, and all descendant groups. This allows the user to define
dimensions in a top-level group, and use them in many sub-groups.
Since it may be necessary to write code which works with all types of
netCDF data files, we also introduce the ability to create
netCDF-4/HDF5 files which follow all the rules of the classic netCDF
model. That is, these files are in HDF5, but will not support multiple
unlimited dimensions, user-defined types, groups, etc. They act just
like a classic netCDF file.
\section met_example Meteorological Example
NetCDF can be used to store many kinds of data, but it was originally
developed for the Earth science community.
NetCDF views the world of scientific data in the same way that an
atmospheric scientist might: as sets of related arrays. There are
various physical quantities (such as pressure and temperature) located
at points at a particular latitude, longitude, vertical level, and
time.
A scientist might also like to store supporting information, such as
the units, or some information about how the data were produced.
The axis information (latitude, longitude, level, and time) would be
stored as netCDF dimensions. Dimensions have a length and a name.
The physical quantities (pressure, temperature) would be stored as
netCDF variables. Variables are N-dimensional arrays of data, with a
name and an associated set of netCDF dimensions.
It is also customary to add one variable for each dimension, to hold
the values along that axis. These variables are called “coordinate
variables.” The latitude coordinate variable would be a
one-dimensional variable (with latitude as its dimension), and it
would hold the latitude values at each point along the axis.
The additional bits of metadata would be stored as netCDF attributes.
Attributes are always single values or one-dimensional arrays. (This
works out well for a string, which is a one-dimensional array of ASCII
characters.)
The pres_temp_4D_wr.c/pres_temp_4D_rd.c examples show
how to write and read a file containing some four-dimensional pressure
and temperature data, including all the metadata needed.
\page fill_values Fill Values
Sometimes there are missing values in the data, and some value is
needed to represent them.
For example, what value do you put in a sea-surface temperature
variable for points over land?
In netCDF, you can create an attribute for the variable (and of the
same type as the variable) called “_FillValue” that contains a value
that you have used for missing data. Applications that read the data
file can use this to know how to represent these values.
Using attributes it is possible to capture metadata that would
otherwise be separated from the data. Various conventions have been
established. By using a set of conventions, a data producer is more
likely to produce files that can be easily shared within the research
community, and that contain enough details to be useful as a long-term
archive. Conventions also make it easier to develop software that
interprets information represented in data, because a convention
selects one conventional way to represent information when multiple
equivalent representations are possible.
For more information on _FillValue and other attribute conventions,
see \ref attribute_conventions.
Climate and meteorological users are urged to follow the Climate and
Forecast (CF) metadata conventions when producing data files. For more
information about the CF conventions, see http://cf-pcmdi.llnl.gov.
\page error_handling Error Handling
\addtogroup error
Each netCDF function in the C, Fortran 77, and Fortran 90 APIs returns
0 on success, in the tradition of C.
When programming with netCDF in these languages, always check return
values of every netCDF API call. The return code can be looked up in
netcdf.h (for C programmers) or netcdf.inc (for Fortran programmers),
or you can use the strerror function to print out an error
message. (See nc_strerror/NF_STRERROR/NF90_STRERROR).
In general, if a function returns an error code, you can assume it
didn't do what you hoped it would. The exception is the NC_ERANGE
error, which is returned by any of the reading or writing functions
when one or more of the values read or written exceeded the range for
the type. (For example if you were to try to read 1000 into an
unsigned byte.)
In the case of NC_ERANGE errors, the netCDF library completes the
read/write operation, and then returns the error. The type conversion
is handled like a C type conversion, whether or not it is within
range. This may yield bad data, but the netCDF library just returns
NC_ERANGE and leaves it up to the user to handle. (For more
information about type conversion see Type Conversion).
\page unlimited_dims Unlimited Dimensions
Sometimes you don't know the size of all dimensions when you create a
file, or you would like to arbitrarily extend the file along one of
the dimensions.
For example, model output usually has a time dimension. Rather than
specifying that there will be forty-two output times when creating the
file, you might like to create it with one time, and then add data for
additional times, until you wanted to stop.
For this purpose netCDF provides the unlimited dimension. By
specifying a length of “unlimited” when defining a dimension, you
indicate to netCDF that the dimension may be extended, and its length
may increase.
In netCDF classic files, there can only be one unlimited dimension,
and it must be declared first in the list of dimensions for a
variable.
For programmers, the unlimited dimension will correspond with the
slowest-varying dimension. In C this is the first dimension of an
array, in Fortran, the last.
The third example in this tutorial, pres_temp_4D, demonstrates how to
write and read data one time step at a time along an unlimited
dimension in a classic netCDF file. See pres_temp_4D.
In netCDF-4/HDF5 files, any number of unlimited dimensions may be
used, and there is no restriction as to where they appear in a
variable's list of dimension IDs.
\page examples1 NetCDF Example Programs
\tableofcontents
The netCDF example programs show how to use netCDF.
In the netCDF distribution, the “examples” directory contains examples
in C and CDL. The examples create, and then read, example data files
of increasing complexity.
There are three sets of netCDF classic example programs; corresponding
examples are included with the netCDF Fortran and C++ APIs.
- \ref example_simple_xy
- \ref example_sfc_pres_temp
- \ref example_pres_temp_4D
Additionally, there is a example program demonstrating
how to use the filter API. This is C only and only accessible
when built with automake currently.
- \ref example_filter
Any existing netCDF applications can be converted to generate
netCDF-4/HDF5 files. Simply change the file creation call to include
the correct mode flag.
In one of the netCDF classic examples which write a data file, change
the nc_create() call so that ::NC_NETCDF4 is one of the flags set on
the create.
The corresponding read example will work without modification; netCDF
will notice that the file is a NetCDF-4/HDF5 file, and will read it
automatically, just as if it were a netCDF classic format file.
In the example in this section we show some of the advanced features
of netCDF-4.
- \ref example_simple_nc4
- \ref example_simple_xy_nc4
The examples are built and run with the “make check” command. (See
[Building
netCDF-C](https://docs.unidata.ucar.edu/nug/current/getting_and_building_netcdf.html#building)).
The corresponding examples in each language create identical netCDF
data files. For example, the C program sfc_pres_temp_wr.c produces the
same data file as the Fortran 77 program sfc_pres_temp_wr.f.
\section example_simple_xy The simple_xy Example
This example is an unrealistically simple netCDF file, to demonstrate
the minimum operation of the netCDF APIs. Users should seek to make
their netCDF files more self-describing than this primitive example.
- simple_xy_wr.c
- simple_xy_rd.c
As in all the netCDF tutorial examples, this example file can be
created by C and by ncgen, which creates it from a CDL script. Both
ncgen and the C example create identical files, “simple_xy.nc.”
The simple_xy.nc data file contains two dimensions, “x” and “y”, and
one netCDF variable, “data.”
The CDL for this example is shown below. For more information on
ncdump and ncgen see NetCDF Utilities.
\code
netcdf simple_xy {
dimensions:
x = 6 ;
y = 12 ;
variables:
int data(x, y) ;
data:
data =
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,
24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,
60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71 ;
}
\endcode
\section example_sfc_pres_temp The sfc_pres_temp Example
This example has been constructed for the meteorological mind.
Suppose you have some data you want to write to a netCDF file. For
example, you have one time step of surface temperature and surface
pressure, on a 6 x 12 latitude longitude grid.
To store this in netCDF, create a file, add two dimensions (latitude
and longitude) and two variables (pressure and temperature).
- sfc_pres_temp_wr.c
- sfc_pres_temp_rd.c
In this example we add some netCDF attributes, as is typical in
scientific applications, to further describe the data. In this case we
add a units attribute to every netCDF variable.
In this example we also add additional netCDF variables to describe
the coordinate system. These “coordinate variables” allow us to
specify the latitudes and longitudes that describe the data grid.
The CDL version of the data file, generated by ncdump, is shown below
(see \ref netcdf_utilities).
\code
netcdf sfc_pres_temp {
dimensions:
latitude = 6 ;
longitude = 12 ;
variables:
float latitude(latitude) ;
latitude:units = "degrees_north" ;
float longitude(longitude) ;
longitude:units = "degrees_east" ;
float pressure(latitude, longitude) ;
pressure:units = "hPa" ;
float temperature(latitude, longitude) ;
temperature:units = "celsius" ;
data:
latitude = 25, 30, 35, 40, 45, 50 ;
longitude = -125, -120, -115, -110, -105, -100, -95, -90, -85, -80, -75, -70 ;
pressure =
900, 906, 912, 918, 924, 930, 936, 942, 948, 954, 960, 966,
901, 907, 913, 919, 925, 931, 937, 943, 949, 955, 961, 967,
902, 908, 914, 920, 926, 932, 938, 944, 950, 956, 962, 968,
903, 909, 915, 921, 927, 933, 939, 945, 951, 957, 963, 969,
904, 910, 916, 922, 928, 934, 940, 946, 952, 958, 964, 970,
905, 911, 917, 923, 929, 935, 941, 947, 953, 959, 965, 971 ;
temperature =
9, 10.5, 12, 13.5, 15, 16.5, 18, 19.5, 21, 22.5, 24, 25.5,
9.25, 10.75, 12.25, 13.75, 15.25, 16.75, 18.25, 19.75, 21.25, 22.75, 24.25,
25.75,
9.5, 11, 12.5, 14, 15.5, 17, 18.5, 20, 21.5, 23, 24.5, 26,
9.75, 11.25, 12.75, 14.25, 15.75, 17.25, 18.75, 20.25, 21.75, 23.25, 24.75,
26.25,
10, 11.5, 13, 14.5, 16, 17.5, 19, 20.5, 22, 23.5, 25, 26.5,
10.25, 11.75, 13.25, 14.75, 16.25, 17.75, 19.25, 20.75, 22.25, 23.75,
25.25, 26.75 ;
}
\endcode
\section example_pres_temp_4D The pres_temp_4D Example
This example expands on the previous example by making our
two-dimensional data into four-dimensional data, adding a vertical
level axis and an unlimited time step axis.
- pres_temp_4D_wr.c
- pres_temp_4D_rd.c
Additionally, in this example the data are written and read one time
step at a time, as is typical in scientific applications that use the
unlimited dimension.
The sample data file created by pres_temp_4D_wr can be examined with
the utility ncdump (see \ref netcdf_utilities).
\code
netcdf pres_temp_4D {
dimensions:
level = 2 ;
latitude = 6 ;
longitude = 12 ;
time = UNLIMITED ; // (2 currently)
variables:
float latitude(latitude) ;
latitude:units = "degrees_north" ;
float longitude(longitude) ;
longitude:units = "degrees_east" ;
float pressure(time, level, latitude, longitude) ;
pressure:units = "hPa" ;
float temperature(time, level, latitude, longitude) ;
temperature:units = "celsius" ;
data:
latitude = 25, 30, 35, 40, 45, 50 ;
longitude = -125, -120, -115, -110, -105, -100, -95, -90, -85, -80, -75, -70 ;
pressure =
900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911,
912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923,
924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935,
936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947,
948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959,
960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971,
972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983,
984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995,
996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007,
1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019,
1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031,
1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043,
900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911,
912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923,
924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935,
936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947,
948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959,
960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971,
972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983,
984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995,
996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007,
1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019,
1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031,
1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043 ;
temperature =
9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,
33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56,
57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68,
69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,
81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128,
129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152,
9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,
33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44,
45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56,
57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68,
69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80,
81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92,
93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104,
105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128,
129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152 ;
}
\endcode
\section example_simple_nc4 The simple_nc4 Example
This example, like the simple_xy netCDF-3 example above, is an overly simplified example which demonstrates how to use groups in a netCDF-4 file.
This example is only available in C for this version of netCDF-4. The example creates and then reads the file “simple_nc4.nc.”
The simple_xy.nc data file contains two dimensions, “x” and “y”, two groups, “grp1” and “grp2”, and two data variables, one in each group, both named: “data.” One data variable is an unsigned 64-bit integer, the other a user-defined compound type.
The example program simple_nc4_wr.c creates the example data file simple_nc4.nc. The example program simple_nc4_rd.c reads the data file.
- simple_nc4_wr.c
- simple_nc4_rd.c
\section example_simple_xy_nc4 The simple_xy_nc4 Example
This example, like the simple_xy netCDF-3 example above, is an overly simplified example. It is based on the simple_xy example, but used data chunking, compression, and the fletcher32 filter.
(These are all HDF5 features. For more information see http://hdfgroup.org/HDF5/).
This example is not yet available in C++. We hope to have the C++ example in a future release of netCDF.
The example creates and then reads the file “simple_xy_nc4.nc.”
The example program simple_xy_nc4_wr.c creates the example data file
simple_xy_nc4.nc. The example program simple_xy_nc4_rd.c reads the
data file.
- simple_xy_nc4_wr.c
- simple_xy_nc4_rd.c
\section example_filter The filter example
This example demonstrates how to write and read a variable that is
compressed using, in this example, bzip2 compression.
- filter_example.c
\page interoperability_hdf5 Interoperability with HDF5
NetCDF-4 allows some interoperability with HDF5.
\section reading_with_hdf5 Reading and Editing NetCDF-4 Files with HDF5
The HDF5 Files produced by netCDF-4 are perfectly respectable HDF5
files, and can be read by any HDF5 application.
NetCDF-4 relies on several new features of HDF5, including dimension
scales. The HDF5 dimension scales feature adds a bunch of attributes
to the HDF5 file to keep track of the dimension information.
It is not just wrong, but wrong-headed, to modify these attributes
except with the HDF5 dimension scale API. If you do so, then you will
deserve what you get, which will be a mess.
Additionally, netCDF stores some extra information for dimensions
without dimension scale information. (That is, a dimension without an
associated coordinate variable). So HDF5 users should not write data
to a netCDF-4 file which extends any unlimited dimension, or change
any of the extra attributes used by netCDF to track dimension
information.
Also there are some types allowed in HDF5, but not allowed in netCDF-4
(for example the time type). Using any such type in a netCDF-4 file
will cause the file to become unreadable to netCDF-4. So don't do it.
NetCDF-4 ignores all HDF5 references. Can't make head nor tail of
them. Also netCDF-4 assumes a strictly hierarchical group
structure. No looping, you weirdo!
Attributes can be added (they must be one of the netCDF-4 types),
modified, or even deleted, in HDF5.
\section accessing_hdf5 Reading and Editing HDF5 Files with NetCDF-4
Assuming a HDF5 file is written in accordance with the netCDF-4 rules
(i.e. no strange types, no looping groups), and assuming that *every*
dataset has a dimension scale attached to each dimension, the netCDF-4
API can be used to read and edit the file, quite easily.
In HDF5 (version 1.8.0 and later), dimension scales are (generally) 1D
datasets, that hold dimension data. A multi-dimensional dataset can
then attach a dimension scale to any or all of its dimensions. For
example, a user might have 1D dimension scales for lat and lon, and a
2D dataset which has lat attached to the first dimension, and lon to
the second.
If dimension scales are not used, then netCDF-4 can still edit the
file, and will invent anonymous dimensions for each variable
shape. This is done by iterating through the space of each dataset. As
each space size is encountered, a phony dimension of that size is
checked for. It it does not exist, a new phony dimension is created
for that size. In this way, a HDF5 file with datasets that are using
shared dimensions can be seen properly in netCDF-4. (There is no
shared dimension in HDF5, but data users will freqently write many
datasets with the same shape, and intend these to be shared
dimensions.)
Starting with version 4.7.3, if a dataset is encountered with uses the
same size for two or more of its dataspace lengths, then a new phony
dimension will be created for each. That is, a dataset with size
[100][100] will result in two phony dimensions, each of size 100.
\page groups Groups
NetCDF-4 files can store attributes, variables, and dimensions in
hierarchical groups.
This allows the user to create a structure much like a Unix file
system. In netCDF, each group gets an ncid. Opening or creating a file
returns the ncid for the root group (which is named “/”). Groups can
be added with the nc_def_grp() function. Get the number of groups, and
their ncids, with the nc_inq_grps() function.
Dimensions are scoped such that they are visible to all child
groups. For example, you can define a dimension in the root group, and
use its dimension id when defining a variable in a sub-group.
Attributes defined as ::NC_GLOBAL apply to the group, not the entire
file.
The degenerate case, in which only the root group is used, corresponds
exactly with the classic data mode, before groups were introduced.
\page user_defined_types User Defined Types
\section compound_types Compound Types
In netCDF-4 files it's possible to create a data type which
corresponds to a C struct. These are known as “compound” types
(following HDF5 nomenclature).
That is, a netCDF compound type is a data structure which contains an
arbitrary collection of other data types, including other compound
types.
To define a new compound type, use nc_def_compound(). Then call
nc_insert_compound() for each type within the compound type.
Read and write arrays of compound data with the nc_get_vara() and
nc_put_vara() functions. These functions were actually part of the
netCDF-2 API, brought out of semi-retirement to handle user-defined
types in netCDF-4.
\section opaque_types Opaque Types
Store blobs of bits in opaque types. Create an opaque type with
nc_def_opaque. Read and write them with nc_get_vara()/nc_put_vara().
\section vlen_type Variable Length Arrays (VLEN)
Create a VLEN type to store variable length arrays of a known base
type. Use nc_def_vlen() to define a VLEN type, read and write them with
nc_get_vara()/nc_put_vara().
\page string_type Strings
Use the ::NC_STRING type to store arrays of strings. Read and write them
with nc_get_vara()/nc_put_vara().
\page parallel_io Parallel I/O with NetCDF-4
NetCDF-4 provides parallel file access to both classic and
netCDF-4/HDF5 files. The parallel I/O to netCDF-4 files is achieved
through the HDF5 library while the parallel I/O to classic files is
through PnetCDF. A few functions have been added to the netCDF C API
to handle parallel I/O. You must build netCDF-4 properly to take
advantage of parallel features (see [Building with Parallel I/O
Support](https://docs.unidata.ucar.edu/nug/current/getting_and_building_netcdf.html#build_parallel)).
The nc_open_par() and nc_create_par() functions are used to
create/open a netCDF file with parallel access.
\note The parallel access associated with these functions is not a characteristic of the data file, but the way it was opened.
\section collective_independent Collective/Independent Access
Parallel file access is either collective (all processors must participate) or
independent (any processor may access the data without waiting for others). All
netCDF metadata writing operations are collective. That is, all creation of
groups, types, variables, dimensions, or attributes. Data reads and writes
(e.g. calls to nc_put_vara_int() and nc_get_vara_int()) may be independent, the
default) or collective. To change from collective to independent mode or vis
versa, call nc_var_par_access() with argument 'access' set to either
NC_INDEPENDENT or NC_COLLECTIVE. Note when using PnetCDF, the argument 'varid'
is ignored, as PnetCDF does not support per-variable collective/independent
mode change.
\page tutorial_ncids Numbering of NetCDF IDs
In C, Fortran 77, and Fortran 90, netCDF objects are identified by an
integer: the ID. NetCDF functions use this ID to identify the
object. It's helpful for the programmer to understand these IDs.
Open data files, dimensions, variables, and attributes, and
used-defined types are each numbered independently, and are always
numbered in the order in which they were defined. (They also appear in
this order in ncdump output.) Numbering starts with 0 in C, and 1 in
Fortran 77/90.
For example, the first variable defined in a file will have an ID of 0
in C programs, and 1 in Fortran programs, and functions that apply to
a variable will need to know the ID of the variable you mean.
IDs for netCDF dimensions and variables are persistent, but deleting
an attribute changes subsequent attribute numbers.
Although netCDF refers to everything by an integer id (varid, dimid,
attnum), there are inquiry functions which, given a name, will return
an ID. For example, nc_inq_varid() will take a character
string (the name), and give back the ID of the variable of that
name. The variable ID is then used in subsequent calls (to read the
data, for example).
The ncid used to identify a file in the classic model, or a group
within that file in the enhanced model (see \ref netcdf_data_model), or a
user-defined type, are not permanently associated with the file. They
may change the next time the file is opened.
\page creating Creating New Files and Metadata, an Overview
To construct a netCDF file you need to:
- create the file - Specify the name, optionally the format: classic
(the default), 64bit-offset, or 64-bit data.
- define metadata - Specify the names and types of dimensions, data
variables, and attributes.
- write data - Write arrays of data from program variables to the
netCDF file. Arrays of data may be written all at once, or in subsets.
- close the file - Close the file to flush all buffers to the disk and
free all resources allocated for this file.
\page reading_known Reading NetCDF Files of Known Structure
To read a netCDF file of known structure, you need to:
- open the file - Specify the file name and whether you want
read-write or read-only access.
- read variable or attribute data - Read the data or attributes of
interest.
- close the file - Release all resources associated with this file.
Use ncdump to learn the structure of a file (use the -h option).
\page reading_unknown Reading NetCDF Files of Unknown Structure
Perhaps you would like to write your software to handle more general
cases, so that you don't have to adjust your source every time the
grid size changes, or a variable is added to the file.
There are inquiry functions that let you find out everything you need
to know about a file. These functions contain “inq” or “INQ” in their
names.
Using the inquiry functions, it is possible to write code that will
read and understand any netCDF file, whatever its contents. (For
example, ncdump does just that.)
First use nc_inq(), which will tell you how many variables and global
attributes there are in the file.
Start with global attribute 0, and proceed to natts - 1, the number of
global attributes minus one. The nc_inq_att() function will tell you
the name, type, and length of each global attribute.
Then start with dimid 0, and proceed to dimid ndims - 1, calling
nc_inq_dim(). This will tell you the name and length of each
dimension, and whether it is unlimited.
Then start with varid 0, and proceed to varid nvars - 1, calling
nc_inq_var(). This will tell you the number of dimensions of this
variable, and their associated IDs. It will also get the name and type
of this variable, and whether there are any attributes attached. If
there are attributes attached, use the nc_inq_att() function to get
their names, types, and lengths.
(To read an attribute, use the appropriate nc_get_att_<TYPE> function,
like nc_get_att_int() to get the data from an attribute that is an
array of integers.)
There are also functions that return an item's ID, given its name. To
find IDs from the names, use functions nc_inq_dimid(),
nc_inq_attnum(), and nc_inq_varid().
The inquiry functions are:
- nc_inq() Find number of dimensions, variables, and global
attributes, and the unlimited dimid.
- nc_inq_att() Find attribute name, type, and length.
- nc_inq_dim() Find dimension name and length.
- nc_inq_var() Find variable name, type, num dimensions, dim IDs, and
num attributes.
- nc_inq_dimid() Find dimension ID from its name.
- nc_inq_varid() Find variable ID from its name.
- nc_inq_format() Find file format: classic CDF-1, 64-bit offset CDF-2, or 64-bit data CDF-5
- nc_inq_libvers() Find the netCDF library version.
\page accessing_subsets Reading and Writing Subsets of Data
Usually users are interested in reading or writing subsets of
variables in a netCDF data file. The netCDF APIs provide a variety of
functions for this purpose.
In the simplest case, you will use the same type for both file and
in-memory storage, but in some cases you may wish to use different
types. For example, you might have a netCDF file that contains integer
data, and you wish to read it into floating-point storage, converting
the data as it is read. The same sort of type conversion can be done
when writing the data.
To convert to a type while reading data, use the appropriate
nc_get_vara_<TYPE> or NF_GET_VARA_<TYPE> function. For example, the C
function nc_get_vara_float(), and the Fortran function
NF_GET_VARA_REAL will read netCDF data of any numeric type into a
floating-point array, automatically converting each element to the
desired type.
To convert from a type while writing data, use the appropriate
nc_put_vara_<TYPE> or NF_PUT_VARA_<TYPE> function. For example, the C
function nc_put_vara_float() will write floating-point data into
netCDF arrays, automatically converting each element of the array to
the type of the netCDF variable.
The TYPE in the function name refers to the type of the in-memory
data, in both cases. They type of the file data is determined when the
netCDF variable is defined.
The type of the data may be automatically converted on read or
write.
\example simple_xy_wr.c
\example simple_xy_rd.c
\example sfc_pres_temp_wr.c
\example sfc_pres_temp_rd.c
\example pres_temp_4D_wr.c
\example pres_temp_4D_rd.c
\example simple_nc4_wr.c
\example simple_nc4_rd.c
\example simple_xy_nc4_wr.c
\example simple_xy_nc4_rd.c
*/
Reference in New Issue View Git Blame Copy Permalink