hdf5/doxygen/dox/LearnBasics1.dox

/** @page LBFileOrg HDF5 File Organization

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

\section secLBFileOrg HDF5 file
An HDF5 file is a container for storing a variety of scientific data and is composed of two primary types of objects: groups and datasets.

\li HDF5 group: a grouping structure containing zero or more HDF5 objects, together with supporting metadata
\li HDF5 dataset: a multidimensional array of data elements, together with supporting metadata

Any HDF5 group or dataset may have an associated attribute list. An HDF5 attribute is a user-defined HDF5 structure
that provides extra information about an HDF5 object.

Working with groups and datasets is similar in many ways to working with directories and files in UNIX. As with UNIX
directories and files, an HDF5 object in an HDF5 file is often referred to by its full path name (also called an absolute path name).

\li <code style="background-color:whitesmoke;">/</code>   signifies the root group.

\li <code style="background-color:whitesmoke;">/foo</code>  signifies a member of the root group called foo.

\li <code style="background-color:whitesmoke;">/foo/zoo</code>  signifies a member of the group foo, which in turn is a member of the root group.

<hr>
Next Chapter \ref LBAPI

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBAPI The HDF5 API

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

\section secLBAPI HDF5 C API
The HDF5 library provides several interfaces, or APIs. These APIs provide routines for creating,
accessing, and manipulating HDF5 files and objects.

The library itself is implemented in C. To facilitate the work of FORTRAN 90, C++ and Java programmers,
HDF5 function wrappers have been developed in each of these languages. This tutorial discusses the use
of the C functions and the FORTRAN wrappers.

All C routines in the HDF5 library begin with a prefix of the form H5*, where * is one or two uppercase
letters indicating the type of object on which the function operates.
The FORTRAN wrappers come in the form of subroutines that begin with h5 and end with _f.
Java routine names begin with “H5*” and are prefixed with “H5.” as the class. Constants are
in the HDF5Constants class and are prefixed with "HDF5Constants.".
The APIs are listed below:
<table>
<tr>
<th><strong>API</strong>
</th>
<th><strong>DESCRIPTION</strong>
</th>
</tr>
<tr>
<th><strong>H5</strong>
</th>
<td>Library Functions: general-purpose H5 functions
</td>
</tr>
<tr>
<th><strong>H5A</strong>
</th>
<td>Annotation Interface: attribute access and manipulation routines
</td>
</tr>
<tr>
<th><strong>H5D</strong>
</th>
<td>Dataset Interface: dataset access and manipulation routines
</td>
</tr>
<tr>
<th><strong>H5E</strong>
</th>
<td>Error Interface: error handling routines
</td>
</tr>
<tr>
<th><strong>H5F</strong>
</th>
<td>File Interface: file access routines
</td>
</tr>
<tr>
<th><strong>H5G</strong>
</th>
<td>Group Interface: group creation and operation routines
</td>
</tr>
<tr>
<th><strong>H5I</strong>
</th>
<td>Identifier Interface: identifier routines
</td>
</tr>
<tr>
<th><strong>H5L</strong>
</th>
<td>Link Interface: link routines
</td>
</tr>
<tr>
<th><strong>H5O</strong>
</th>
<td>Object Interface: object routines
</td>
</tr>
<tr>
<th><strong>H5P</strong>
</th>
<td>Property List Interface: object property list manipulation routines
</td>
</tr>
<tr>
<th><strong>H5R</strong>
</th>
<td>Reference Interface: reference routines
</td>
</tr>
<tr>
<th><strong>H5S</strong>
</th>
<td>Dataspace Interface: dataspace definition and access routines
</td>
</tr>
<tr>
<th><strong>H5T</strong>
</th>
<td>Datatype Interface: datatype creation and manipulation routines
</td>
</tr>
<tr>
<th><strong>H5Z</strong>
</th>
<td>Compression Interface: compression routine(s)
</td>
</tr>
</table>

<hr>
Previous Chapter \ref LBFileOrg - Next Chapter \ref LBProg

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBProg Programming Issues

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

Keep the following in mind when looking at the example programs included in this tutorial:

\section LBProgAPI APIs vary with different languages
\li C routines begin with the prefix “H5*” where * is a single letter indicating the object on which the operation is to be performed:
<table>
<tr>
<td>File Interface: </td>
<td>#H5Fopen</td>
</tr><tr>
<td>Dataset Interface:</td>
<td>#H5Dopen</td>
</tr>
</table>

\li FORTRAN routines begin with “h5*” and end with “_f”:
<table>
<tr>
<td>File Interface: </td>
<td>h5fopen_f</td>
</tr><tr>
<td>Dataset Interface:</td>
<td>h5dopen_f</td>
</tr>
</table>

\li Java routine names begin with “H5*” and are prefixed with “H5.” as the class. Constants are
in the HDF5Constants class and are prefixed with "HDF5Constants.".:
<table>
<tr>
<td>File Interface: </td>
<td>H5.H5Fopen</td>
</tr><tr>
<td>Dataset Interface:</td>
<td>H5.H5Dopen</td>
</tr>
</table>

\li APIS for languages like C++, Java, and Python use methods associated with specific objects.

\section LBProgTypes HDF5 library has its own defined types
\li #hid_t is used for object handles
\li hsize_t is used for dimensions
\li #herr_t is used for many return values

\section LBProgLang Language specific files must be included in applications
<ul>
<li>
Python:   Add <code>"import h5py / import numpy"</code>
</li>
<li>
C:        Add <code>"#include hdf5.h"</code>
</li>
<li>
FORTRAN:  Add <code>"USE HDF5"</code> and call h5open_f and h5close_f to initialize and close the HDF5 FORTRAN interface
</li>
<li>
Java:     Add <code>"import hdf.hdf5lib.H5;
                     import hdf.hdf5lib.HDF5Constants;"</code>
</li>
</ul>

<hr>
Previous Chapter \ref LBAPI - Next Chapter \ref LBFileCreate

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBFileCreate Creating an HDF5 File

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

An HDF5 file is a binary file containing scientific data and supporting metadata.
\section secLBFileCreate HDF5 File Access
To create an HDF5 file, an application must specify not only a file name, but a file access mode,
a file creation property list, and a file access property list. These terms are described below:
<ul>
<li><strong>File access mode:</strong><br />
When creating a file, the file access mode specifies the action to take if the file already exists:
<ul>
<li>#H5F_ACC_TRUNC specifies that if the file already exists, the current contents will be deleted so
that the application can rewrite the file with new data.
</li>
<li>#H5F_ACC_EXCL specifies that the open will fail if the file already exists. If the file does not
already exist, the file access parameter is ignored.
</li>
</ul>
In either case, the application has both read and write access to the successfully created file.
<br />
Note that there are two different access modes for opening existing files:
<ul>
<li>#H5F_ACC_RDONLY specifies that the application has read access but will not be allowed to write any data.
</li>
<li>#H5F_ACC_RDWR specifies that the application has read and write access.
</li>
</ul>
</li>
<li><strong>File creation property list:</strong><br />The file creation property list is used to
control the file metadata. File metadata contains information about the size of the user-block*,
the size of various file data structures used by the HDF5 library, etc. In this tutorial, the
default file creation property list, #H5P_DEFAULT, is used.<br />
 *The user-block is a fixed-length block of data located at the beginning of the file which is
ignored by the HDF5 library. The user-block may be used to store any data or information found
to be useful to applications.
</li>
<li><strong>File access property list:</strong><br />The file access property list is used to
control different methods of performing I/O on files. It also can be used to control how a file
is closed (whether or not to delay the actual file close until all objects in a file are closed).
The default file access property list, #H5P_DEFAULT, is used in this tutorial.
</li>
</ul>

Please refer to the \ref sec_file section of the \ref UG and \ref H5F section in the \ref RM for
detailed information regarding file access/creation property lists and access modes.

The steps to create and close an HDF5 file are as follows:
<ol>
<li>Specify the file creation and access property lists, if necessary.</li>
<li>Create the file.</li>
<li>Close the file, and if necessary, close the property lists.</li>
</ol>

\section secLBFileExample Programming Example

\subsection subsecLBFileExampleDesc Description
The following example code demonstrates how to create and close an HDF5 file.

<em>C</em>
\code
#include "hdf5.h"
   #define FILE "file.h5"

   int main() {

      hid_t       file_id;   /* file identifier */
      herr_t      status;

      /* Create a new file using default properties. */
      file_id = H5Fcreate(FILE, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);

      /* Terminate access to the file. */
      status = H5Fclose(file_id); 
   }
\endcode

<em>Fortran</em>
\code
  PROGRAM FILEEXAMPLE

     USE HDF5 ! This module contains all necessary modules 
        
     IMPLICIT NONE

     CHARACTER(LEN=8), PARAMETER :: filename = "filef.h5" ! File name
     INTEGER(HID_T) :: file_id                            ! File identifier
 
     INTEGER     ::   error  ! Error flag
     
!
!    Initialize FORTRAN interface.
!
     CALL h5open_f (error)
     !
     ! Create a new file using default properties.
     ! 
     CALL h5fcreate_f(filename, H5F_ACC_TRUNC_F, file_id, error)

     !
     ! Terminate access to the file.
     !
     CALL h5fclose_f(file_id, error)
!
!    Close FORTRAN interface.
!
     CALL h5close_f(error)
     END PROGRAM FILEEXAMPLE
\endcode

See \ref LBExamples for the examples used in the Learning the Basics tutorial.

For details on compiling an HDF5 application:
[ \ref LBCompiling ]

\subsection subsecLBFileExampleRem Remarks
\li <strong>In C</strong>: The include file <code style="background-color:whitesmoke;">hdf5.h</code> contains definitions and declarations and must be included
in any program that uses the HDF5 library.
<br />
<strong>In FORTRAN</strong>: The module <code style="background-color:whitesmoke;">HDF5</code> contains definitions and declarations and must be used in any
program that uses the HDF5 library. Also note that #H5open MUST be called at the beginning of an HDF5 Fortran
application (prior to any HDF5 calls) to initialize the library and variables. The #H5close call MUST be at
the end of the HDF5 Fortran application.
\li #H5Fcreate creates an HDF5 file and returns the file identifier.<br />
For Fortran, the file creation property list and file access property list are optional. They can be omitted if the
default values are to be used.<br />
The root group is automatically created when a file is created. Every file has a root group and the path name of
the root group is always <code style="background-color:whitesmoke;">/</code>.
\li #H5Fclose terminates access to an HDF5 file.<br />
When an HDF5 file is no longer accessed by a program, #H5Fclose must be called to release the resources used by the file.
This call is mandatory.<br />
Note that if #H5Fclose is called for a file, but one or more objects within the file remain open, those objects will
remain accessible until they are individually closed. This can cause access problems for other users, if objects were
inadvertently left open. A File Access property controls how the file is closed.

\subsection subsecLBFileExampleCont File Contents
The HDF Group has developed tools for examining the contents of HDF5 files. The tool used throughout the HDF5 tutorial
is the HDF5 dumper, <code style="background-color:whitesmoke;">h5dump</code>, which displays the file contents in human-readable form. The output of <code style="background-color:whitesmoke;">h5dump</code> is an ASCII
display formatted according to the HDF5 DDL grammar. This grammar is defined, using Backus-Naur Form, in the
\ref DDLBNF114.

To view the HDF5 file contents, simply type:
\code
h5dump <filename>
\endcode

<table>
<caption>Describe the file contents of file.h5 using a directed graph.</caption>
<tr>
<td>
\image html imgLBFile.gif
</td>
</tr>
</table>

The text description of <code style="background-color:whitesmoke;">file.h5</code>, as generated by <code style="background-color:whitesmoke;">h5dump</code>. The HDF5 file called <code style="background-color:whitesmoke;">file.h5</code>
contains a group called <code style="background-color:whitesmoke;">/</code>, or the root group. (The file called <code style="background-color:whitesmoke;">filef.h5</code>, created by the FORTRAN version of the example,
has the same output except that the filename shown is <code style="background-color:whitesmoke;">filef.h5</code>.)
\code
HDF5 "file.h5" {
         GROUP "/" {
         }
         }
\endcode

\subsection subsecLBFileExampleDDL File Definition in DDL
The simplified DDL file definition for creating an HDF5 file. For simplicity, a simplified DDL is used in this tutorial. A
complete and more rigorous DDL can be found in the \ref DDLBNF114.

The following symbol definitions are used in the DDL:
\code
        ::=               defined as
        <tname>           a token with the name tname
        <a> | <b>         one of <a> or <b>
        <a>*              zero or more occurrences of <a>
\endcode

The simplified DDL for file definition is as follows:
\code
        <file> ::= HDF5 "<file_name>" { <root_group> }

        <root_group> ::= GROUP "/" { <group_attribute>* 
                                        <group_member>* }

        <group_attribute> ::= <attribute>

        <group_member> ::= <group> | <dataset>
\endcode

<hr>
Previous Chapter \ref LBProg - Next Chapter \ref LBDsetCreate

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBDsetCreate Creating a Dataset
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

A dataset is a multidimensional array of data elements, together with supporting metadata. To create
a dataset, the application program must specify the location at which to create the dataset, the
dataset name, the datatype and dataspace of the data array, and the property lists.

\section secLBDsetCreateDtype Datatypes
A datatype is a collection of properties, all of which can be stored on disk, and which, when taken as
a whole, provide complete information for data conversion to or from that datatype.

There are two categories of datatypes in HDF5:
<ul>
<li><strong>Pre-defined</strong>:   These datatypes are opened and closed by HDF5.<br />
Pre-defined datatypes can be atomic or composite:
<ul><li>Atomic datatypes cannot be decomposed into smaller datatype units at the API level. For example: integer, float, reference, string.</li>
<li>Composite datatypes are aggregations of one or more datatypes. For example: array, variable length, enumeration, compound.</li></ul>
</li>
<li><strong>Derived</strong>:   These datatypes are created or derived from the pre-defined types.<br />
A simple example of creating a derived datatype is using the string datatype, H5T_C_S1, to create strings of more than one character:<br />
\code
      hid_t strtype;                     // Datatype ID
      herr_t status;

      strtype = H5Tcopy (H5T_C_S1);
      status = H5Tset_size (strtype, 5); // create string of length 5
\endcode
</li>
</ul>

Shown below is the HDF5 pre-defined datatypes.
\code
                                          +--  integer
                                          +--  floating point
                        +---- atomic  ----+--  date and time
                        |                 +--  character string
       HDF5 datatypes --|                 +--  bitfield
                        |                 +--  opaque
                        |
                        +---- compound
\endcode

Some of the HDF5 predefined atomic datatypes are listed below.

<table>
<caption>Examples of HDF5 predefined datatypes</caption>
<tr>
<th><strong>Datatype</strong></th>
<th><strong>Description</strong></th>
</tr>
<tr>
<th><strong>H5T_STD_I32LE</strong></th>
<td>Four-byte, little-endian, signed, two's complement integer</td>
</tr>
<tr>
<th><strong>H5T_STD_U16BE</strong></th>
<td>Two-byte, big-endian, unsigned integer</td>
</tr>
<tr>
<th><strong>H5T_IEEE_F32BE</strong></th>
<td>Four-byte, big-endian, IEEE floating point</td>
</tr>
<tr>
<th><strong>H5T_IEEE_F64LE</strong></th>
<td>Eight-byte, little-endian, IEEE floating point</td>
</tr>
<tr>
<th><strong>H5T_C_S1</strong></th>
<td>One-byte, null-terminated string of eight-bit characters</td>
</tr>
</table>

<table>
<caption>Examples of HDF5 predefined native datatypes</caption>
<tr>
<th><strong>Native Datatype</strong></th>
<th><strong>Corresponding C or FORTRAN Type</strong></th>
</tr>
<tr>
<th span="2"><strong>C</strong></th>
</tr>
<tr>
<th><strong>H5T_NATIVE_INT</strong></th>
<td>int</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_FLOAT</strong></th>
<td>float</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_CHAR</strong></th>
<td>char</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_DOUBLE</strong></th>
<td>double</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_LDOUBLE</strong></th>
<td>long double</td>
</tr>
<tr>
<th span="2"><strong>Fortran</strong></th>
</tr>
<tr>
<th><strong>H5T_NATIVE_INTEGER</strong></th>
<td>integer</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_REAL</strong></th>
<td>real</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_DOUBLE</strong></th>
<td>double precision</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_CHARACTER</strong></th>
<td>character</td>
</tr>
</table>

In this tutorial, we consider only HDF5 predefined integers.

For further information on datatypes, see \ref sec_datatype in the \ref UG, in addition to the \ref LBDatatypes tutorial topic.

\section secLBDsetCreateDspace Datasets and Dataspaces
A dataspace describes the dimensionality of the data array. A dataspace is either a regular N-dimensional
array of data points, called a simple dataspace, or a more general collection of data points organized
in another manner, called a complex dataspace. In this tutorial, we only consider simple dataspaces.

<em>HDF5 dataspaces</em>
\code
                         +-- simple
       HDF5 dataspaces --|
                         +-- complex
\endcode
The dimensions of a dataset can be fixed (unchanging), or they may be unlimited, which means that they are
extensible. A dataspace can also describe a portion of a dataset, making it possible to do partial
I/O operations on selections.

\section secLBDsetCreateProp Property Lists
Property lists are a mechanism for modifying the default behavior when creating or accessing objects. For
more information on property lists see the \ref LBPropsList tutorial topic.

The following property lists can be specified when creating a dataset:
\li Dataset Creation Property List<br />
When creating a dataset, HDF5 allows the user to specify how raw data is organized and/or compressed on
disk. This information is stored in a dataset creation property list and passed to the dataset interface.
The raw data on disk can be stored contiguously (in the same linear way that it is organized in memory),
partitioned into chunks, stored externally, etc. In this tutorial, we use the default dataset creation
property list (contiguous storage layout and no compression). For more information about dataset creation
property lists, see \ref sec_dataset in the \ref UG.
\li Link Creation Property List<br />
The link creation property list governs creation of the link(s) by which a new dataset is accessed and the
creation of any intermediate groups that may be missing.
\li Dataset Access Property List<br />
Dataset access property lists are properties that can be specified when accessing a dataset.

\section secLBDsetCreateSteps Steps to Create a Dataset
To create an empty dataset (no data written) the following steps need to be taken:
<ol>
<li>Obtain the location identifier where the dataset is to be created.</li>
<li>Define or specify the dataset characteristics:
<ol>
<li>Define a datatype or specify a pre-defined datatype.</li>
<li>Define a dataspace.</li>
<li>Specify the property list(s) or use the default.</li>
</ol></li>
<li>Create the dataset.</li>
<li>Close the datatype, the dataspace, and the property list(s) if necessary.</li>
<li>Close the dataset.</li>
</ol>
In HDF5, datatypes and dataspaces are independent objects which are created separately from any dataset
that they might be attached to. Because of this, the creation of a dataset requires the definition of
the datatype and dataspace. In this tutorial, we use the HDF5 predefined datatypes (integer) and consider
only simple dataspaces. Hence, only the creation of dataspace objects is needed.

\section secLBDsetCreateHL High Level APIs
The High Level \ref H5LT include functions that simplify and condense the steps for
creating datasets in HDF5. The examples in the following section use the standard APIs. For a
quick start you may prefer to look at the \ref H5LT at this time.

If you plan to work with images, please look at the High Level \ref H5IM, as well.

\section secLBDsetCreateProg Programming Example

\subsection subsecLBDsetCreateProgDesc Description
See \ref LBExamples for the examples used in the \ref LearnBasics tutorial.

The example shows how to create an empty dataset. It creates a file called <code style="background-color:whitesmoke;">dset.h5</code>
in the C version (<code style="background-color:whitesmoke;">dsetf.h5</code> in Fortran), defines the dataset dataspace, creates a
dataset which is a 4x6 integer array, and then closes the dataspace, the dataset, and the file.

For details on compiling an HDF5 application: [ \ref LBCompiling ]

\subsection subsecLBDsetCreateProgRem Remarks
#H5Screate_simple creates a new simple dataspace and returns a dataspace identifier.
#H5Sclose releases and terminates access to a dataspace.

<em>C</em>
\code
    dataspace_id = H5Screate_simple (rank, dims, maxdims);
    status = H5Sclose (dataspace_id );
\endcode

<em>FORTRAN</em>
\code
    CALL h5screate_simple_f (rank, dims, dataspace_id, hdferr, maxdims=max_dims)
         or
    CALL h5screate_simple_f (rank, dims, dataspace_id, hdferr)

    CALL h5sclose_f (dataspace_id, hdferr)
\endcode

#H5Dcreate creates an empty dataset at the specified location and returns a dataset identifier.
#H5Dclose closes the dataset and releases the resource used by the dataset. This call is mandatory.

<em>C</em>
\code
   dataset_id = H5Dcreate(file_id, "/dset", H5T_STD_I32BE, dataspace_id, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
   status = H5Dclose (dataset_id);
\endcode

<em>FORTRAN</em>
\code
   CALL h5dcreate_f (loc_id, name, type_id, dataspace_id, dset_id, hdferr)
   CALL h5dclose_f (dset_id, hdferr)
\endcode

Note that if using the pre-defined datatypes in FORTRAN, then a call must be made to initialize and terminate access to the pre-defined datatypes:
\code
  CALL h5open_f (hdferr)
  CALL h5close_f (hdferr)
\endcode

H5open must be called before any HDF5 library subroutine calls are made;
H5close must be called after the final HDF5 library subroutine call.

See the programming example for an illustration of the use of these calls.

\subsection subsecLBDsetCreateContent File Contents
The contents of the file dset.h5 (dsetf.h5 for FORTRAN) are shown below:
<table>
<caption>Contents of dset.h5 (dsetf.h5)</caption>
<tr>
<td>
\image html imgLBDsetCreate.gif
</td>
</tr>
</table>
<table>
<tr>
<th>dset.h5 in DDL</th>
<th>dsetf.h5 in DDL</th>
<tr>
<td>
\code
HDF5 "dset.h5" {
GROUP "/" {
   DATASET "dset" {
      DATATYPE { H5T_STD_I32BE }
      DATASPACE { SIMPLE ( 4, 6 ) / ( 4, 6 ) }
      DATA {
         0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0
      }
   }
}
}
\endcode
</td>
<td>
\code
HDF5 "dsetf.h5" {
GROUP "/" {
   DATASET "dset" {
      DATATYPE { H5T_STD_I32BE }
      DATASPACE { SIMPLE ( 6, 4 ) / ( 6, 4 ) }
      DATA {
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0
      }
   }
}
}
\endcode
</td>
</tr>
</table>
Note in above that #H5T_STD_I32BE, a 32-bit Big Endian integer, is an HDF atomic datatype.

\subsection subsecLBDsetCreateProgDDL Dataset Definition in DDL
The following is the simplified DDL dataset definition:
\code
      <dataset> ::= DATASET "<dataset_name>" { <datatype>
                                               <dataspace>
                                               <data>
                                               <dataset_attribute>* }

      <datatype> ::= DATATYPE { <atomic_type> }

      <dataspace> ::= DATASPACE { SIMPLE <current_dims> / <max_dims> }

      <dataset_attribute> ::= <attribute>
\endcode

<hr>

Previous Chapter \ref LBFileCreate - Next Chapter \ref LBDsetRW

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBDsetRW Reading From and Writing To a Dataset
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

\section secLBDsetRW Dataset I/O Operation
During a dataset I/O operation, the library transfers raw data between memory and the file. The data in memory
can have a datatype different from that of the file and can also be of a different size (i.e., the data in
memory is a subset of the dataset elements, or vice versa). Therefore, to perform read or write operations,
the application program must specify:
\li The dataset
\li The dataset's datatype in memory
\li The dataset's dataspace in memory
\li The dataset's dataspace in the file
\li The dataset transfer property list<br />
<ul>
<li>(The dataset transfer property list controls various aspects of the I/O operations, such as the number
of processes participating in a collective I/O request or hints to the library to control caching of raw
data. In this tutorial, we use the default dataset transfer property list.)</li>
</ul>
\li The data buffer

The steps to read from or write to a dataset are as follows:
<ol>
<li>Obtain the dataset identifier.</li>
<li>Specify the memory datatype.</li>
<li>Specify the memory dataspace.</li>
<li>Specify the file dataspace.</li>
<li>Specify the transfer properties.</li>
<li>Perform the desired operation on the dataset.</li>
<li>Close the dataset.</li>
<li>Close the dataspace, datatype, and property list if necessary.</li>
</ol>

To read from or write to a dataset, the #H5Dread and #H5Dwrite routines are used.

<em>C</em>
\code
   status = H5Dread (set_id, mem_type_id, mem_space_id, file_space_id, xfer_prp, buf );
   status = H5Dwrite (set_id, mem_type_id, mem_space_id, file_space_id, xfer_prp, buf);
\endcode

<em>Fortran</em>
\code
   CALL h5dread_f(dset_id, mem_type_id, buf, dims, error, &
                     mem_space_id=mspace_id, file_space_id=fspace_id, &
                     xfer_prp=xfer_plist_id)
        or
   CALL h5dread_f(dset_id, mem_type_id, buf, dims,  error)


   CALL h5dwrite_f(dset_id, mem_type_id, buf, dims, error, &
                     mem_space_id=mspace_id, file_space_id=fspace_id, &
                     xfer_prp=xfer_plist_id)
        or
   CALL h5dwrite_f(dset_id, mem_type_id, buf, dims, error)
\endcode

\section secLBDsetRWHL High Level APIs
The High Level \ref H5LT include functions that simplify and condense the steps for creating and
reading datasets. Please be sure to review them, in addition to this tutorial.

\section secLBDsetRWEx Programming Example

\subsection secLBDsetRWExDesc Description
See \ref LBExamples for the examples used in the \ref LearnBasics tutorial.

The example shows how to read and write an existing dataset. It opens the file created in the previous example,
obtains the dataset identifier for the dataset <code style="background-color:whitesmoke;">/dset</code>, writes the dataset to the file, then reads
the dataset back. It then closes the dataset and file.

Note that #H5S_ALL is passed in for both the memory and file dataspace parameters in the read and write calls.
This indicates that the entire dataspace of the dataset will be read or written to. #H5S_ALL by itself does not
necessarily have this meaning. See the \ref RM entry for #H5Dread or #H5Dwrite for more information on using #H5S_ALL.

For details on compiling an HDF5 application:
[ \ref LBCompiling ]

\subsection secLBDsetRWExRem Remarks
#H5Fopen opens an existing file and returns a file identifier.

#H5Dopen opens an existing dataset with the specified name and location.

#H5Dwrite writes raw data from an application buffer to the specified dataset, converting from the datatype and
dataspace of the dataset in memory to the datatype and dataspace of the dataset in the file. Specifying #H5S_ALL
for both the memory and file dataspaces indicates that the entire dataspace of the dataset is to be written to.
#H5S_ALL by itself does not necessarily have this meaning. See the \ref RM entry for #H5Dwrite for more information
on using #H5S_ALL.

#H5Dread reads raw data from the specified dataset to an application buffer, converting from the file datatype and
dataspace to the memory datatype and dataspace. Specifying #H5S_ALL for both the memory and file dataspaces
indicates that the entire dataspace of the dataset is to be read. #H5S_ALL by itself does not necessarily have
this meaning. See the \ref RM entry for #H5Dread for more information on using #H5S_ALL.

\subsection secLBDsetRWExCont File Contents

Shown below is the contents of dset.h5 (created by the C program).

<em>dset.h5 in DDL</em>
\code
      HDF5 "dset.h5" {
      GROUP "/" {
         DATASET "dset" {
            DATATYPE { H5T_STD_I32BE }
            DATASPACE { SIMPLE ( 4, 6 ) / ( 4, 6 ) }
            DATA {
               1, 2, 3, 4, 5, 6,
               7, 8, 9, 10, 11, 12,
               13, 14, 15, 16, 17, 18,
               19, 20, 21, 22, 23, 24
            }
         }
      }
      }
\endcode

Shown below is the contents of dsetf.h5 (created by the FORTRAN program).

<em>dsetf.h5 in DDL</em>
\code
      HDF5 "dsetf.h5" {
      GROUP "/" {
         DATASET "dset" {
            DATATYPE { H5T_STD_I32BE }
            DATASPACE { SIMPLE ( 6, 4 ) / ( 6, 4 ) }
                  DATA {
               1, 7, 13, 19,
               2, 8, 14, 20,
               3, 9, 15, 21,
               4, 10, 16, 22,
               5, 11, 17, 23,
               6, 12, 18, 24
            }
         }
      }
      }
\endcode

<hr>
Previous Chapter \ref LBDsetCreate - Next Chapter \ref LBAttrCreate

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBAttrCreate Creating an Attribute
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

Attributes are small datasets that can be used to describe the nature and/or the intended usage of
the object they are attached to. In this section, we show how to create, read, and write an attribute.

\section secLBAttrCreate Creating an attribute
Creating an attribute is similar to creating a dataset. To create an attribute, the application must
specify the object which the attribute is attached to, the datatype and dataspace of the attribute
data, and the attribute creation property list.

The steps to create an attribute are as follows:
<ol>
<li>Obtain the object identifier that the attribute is to be attached to.</li>
<li>Define the characteristics of the attribute and specify the attribute creation property list.
<ul>
<li>Define the datatype.</li>
<li>Define the dataspace.</li>
<li>Specify the attribute creation property list.</li>
</ul></li>
<li>Create the attribute.</li>
<li>Close the attribute and datatype, dataspace, and attribute creation property list, if necessary.</li>
</ol>

To create and close an attribute, the calling program must use #H5Acreate and #H5Aclose. For example:

<em>C</em>
\code
  attr_id = H5Acreate (dataset_id, "Units", H5T_STD_I32BE, dataspace_id, H5P_DEFAULT, H5P_DEFAULT)
  status = H5Aclose (attr_id);
\endcode

<em>Fortran</em>
\code
  CALL h5acreate_f (dset_id, attr_nam, type_id, space_id, attr_id, &
                    hdferr, creation_prp=creat_plist_id)
  or
  CALL h5acreate_f (dset_id, attr_nam, type_id, space_id, attr_id, hdferr)

  CALL h5aclose_f (attr_id, hdferr)
\endcode

\section secLBAttrCreateRW Reading/Writing an attribute
Attributes may only be read or written as an entire object; no partial I/O is supported. Therefore,
to perform I/O operations on an attribute, the application needs only to specify the attribute and
the attribute's memory datatype.

The steps to read or write an attribute are as follows.
<ol>
<li>Obtain the attribute identifier.</li>
<li>Specify the attribute's memory datatype.</li>
<li>Perform the desired operation.</li>
<li>Close the memory datatype if necessary.</li>
</ol>

To read and/or write an attribute, the calling program must contain the #H5Aread and/or
#H5Awrite routines. For example:

<em>C</em>
\code
  status = H5Aread (attr_id, mem_type_id, buf);
  status = H5Awrite (attr_id, mem_type_id, buf);
\endcode

<em>Fortran</em>
\code
  CALL h5awrite_f (attr_id, mem_type_id, buf, dims, hdferr)  
  CALL h5aread_f (attr_id, mem_type_id, buf, dims, hdferr)
\endcode

\section secLBAttrCreateHL High Level APIs
The High Level \ref H5LT include functions that simplify and condense the steps for creating and
reading datasets. Please be sure to review them, in addition to this tutorial.

\section secLBAttrCreateRWEx Programming Example

\subsection secLBAttrCreateRWExDesc Description
See \ref LBExamples for the examples used in the \ref LearnBasics tutorial.

The example shows how to create and write a dataset attribute. It opens an existing file <code style="background-color:whitesmoke;">dset.h5</code>
in C (<code style="background-color:whitesmoke;">dsetf.h5</code> in FORTRAN), obtains the identifier of the dataset <code style="background-color:whitesmoke;">/dset</code>, defines
the attribute's dataspace, creates the dataset attribute, writes the attribute, and then closes the attribute's
dataspace, attribute, dataset, and file.

For details on compiling an HDF5 application:
[ \ref LBCompiling ]

\subsection secLBAttrCreateRWExRem Remarks
#H5Acreate creates an attribute which is attached to the object specified by the first parameter, and returns an identifier.

#H5Awrite writes the entire attribute, and returns the status of the write.

When an attribute is no longer accessed by a program, #H5Aclose must be called to release the attribute from use.
An #H5Aclose/h5aclose_f call is mandatory.

\subsection secLBAttrCreateRWExCont File Contents

Shown below is the contents and the attribute definition of <code style="background-color:whitesmoke;">dset.h5</code> (created by the C program).

<em>dset.h5 in DDL</em>
\code
HDF5 "dset.h5" {
GROUP "/" {
DATASET "dset" {
DATATYPE { H5T_STD_I32BE }
DATASPACE { SIMPLE ( 4, 6 ) / ( 4, 6 ) }
DATA {
 1, 2, 3, 4, 5, 6,
 7, 8, 9, 10, 11, 12,
 13, 14, 15, 16, 17, 18,
 19, 20, 21, 22, 23, 24
}
ATTRIBUTE "attr" {
 DATATYPE { H5T_STD_I32BE }
 DATASPACE { SIMPLE ( 2 ) / ( 2 ) }
 DATA {
    100, 200
 }
}
}
}
}
\endcode

Shown below is the contents and the attribute definition of <code style="background-color:whitesmoke;">dsetf.h5</code> (created by the FORTRAN program).

<em>dsetf.h5 in DDL</em>
\code
HDF5 "dsetf.h5" {
GROUP "/" {
DATASET "dset" {
DATATYPE { H5T_STD_I32BE }
DATASPACE { SIMPLE ( 6, 4 ) / ( 6, 4 ) }
DATA {
 1, 7, 13, 19,
 2, 8, 14, 20,
 3, 9, 15, 21,
 4, 10, 16, 22,
 5, 11, 17, 23,
 6, 12, 18, 24
}
ATTRIBUTE "attr" {
 DATATYPE { H5T_STD_I32BE }
 DATASPACE { SIMPLE ( 2 ) / ( 2 ) }
 DATA {
    100, 200
 }
}
}
}
}
\endcode

\subsection secLBAttrCreateRWExDDL Attribute Definition in DDL

<em>HDF5 Attribute Definition</em>
\code
<attribute> ::= ATTRIBUTE "<attr_name>" { <datatype>
                       <dataspace>
                       <data>  }
\endcode

<hr>
Previous Chapter \ref LBDsetRW - Next Chapter \ref LBGrpCreate

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

*/