mirror of
https://github.com/HDFGroup/hdf5.git
synced 2024-12-21 07:51:46 +08:00
787 lines
25 KiB
Plaintext
787 lines
25 KiB
Plaintext
|
/** @page ViewToolsEdit Command-line Tools For Editing HDF5 Files
|
||
|
Navigate back: \ref index "Main" / \ref GettingStarted / \ref ViewToolsCommand
|
||
|
<hr>
|
||
|
|
||
|
\section secViewToolsEditTOC Contents
|
||
|
<ul>
|
||
|
<li>\ref secViewToolsEditRemove</li>
|
||
|
<li>\ref secViewToolsEditChange</li>
|
||
|
<li>\ref secViewToolsEditApply</li>
|
||
|
<li>\ref secViewToolsEditCopy</li>
|
||
|
<li>\ref secViewToolsEditAdd</li>
|
||
|
</ul>
|
||
|
|
||
|
\section secViewToolsEditRemove Remove Inaccessible Objects and Unused Space in a File
|
||
|
HDF5 files may accumulate unused space when they are read and rewritten to or if objects are deleted within
|
||
|
them. With many edits and deletions this unused space can add up to a sizable amount.
|
||
|
|
||
|
The <code style="background-color:whitesmoke;">h5repack</code> tool can be used to remove unused space in an HDF5
|
||
|
file. If no options other than the input and output HDF5 files are specified on the
|
||
|
<code style="background-color:whitesmoke;">h5repack</code> command line, it will write the file to the new
|
||
|
file, getting rid of the unused space:
|
||
|
\code
|
||
|
h5repack <input file> <output file>
|
||
|
\endcode
|
||
|
|
||
|
\section secViewToolsEditChange Change a Dataset's Storage Layout
|
||
|
The <code style="background-color:whitesmoke;">h5repack</code> utility can be used to change a dataset's storage
|
||
|
layout. By default, the storage layout of a dataset is defined at creation time and it cannot be changed.
|
||
|
However, with h5repack you can write an HDF5 file to a new file and change the layout for objects in the new file.
|
||
|
|
||
|
The <code style="background-color:whitesmoke;">-l</code> option in <code style="background-color:whitesmoke;">h5repack</code>
|
||
|
is used to change the layout for an object. The string following the <code style="background-color:whitesmoke;">-l</code>
|
||
|
option defines the layout type and parameters for specified objects (or all objects):
|
||
|
\code
|
||
|
h5repack -l [list of objects:]<layout type>=<layout parameters> <input file> <output file>
|
||
|
\endcode
|
||
|
|
||
|
If no object is specified, then everything in the input file will be written to the output file with the specified
|
||
|
layout type and parameters. If objects are specified then everything in the input file will be written to the
|
||
|
output file as is, except for those specified objects. They will be written to the output file with the given
|
||
|
layout type and parameters.
|
||
|
|
||
|
Following is a description of the dataset layouts and the <code style="background-color:whitesmoke;">h5repack</code>
|
||
|
options to use to change a dataset:
|
||
|
<table>
|
||
|
<tr>
|
||
|
<th>Storage Layout</th><th>h5repack Option</th><th>Description</th>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>Contiguous
|
||
|
</td>
|
||
|
<td>CONTI
|
||
|
</td>
|
||
|
<td>Data is stored physically together
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>Chunked
|
||
|
</td>
|
||
|
<td>CHUNK=DIM[xDIM...xDIM]
|
||
|
</td>
|
||
|
<td>Data is stored in DIM[xDIM...xDIM] chunks
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>Compact
|
||
|
</td>
|
||
|
<td>COMPA
|
||
|
</td>
|
||
|
<td>Data is stored in the header of the object (less I/O)
|
||
|
</td>
|
||
|
</tr>
|
||
|
</table>
|
||
|
|
||
|
If you type <code style="background-color:whitesmoke;">h5repack -h</code> on the command line, you will see
|
||
|
a detailed usage statement with examples of modifying the layout.
|
||
|
|
||
|
In the following example, the dataset <code style="background-color:whitesmoke;">/dset</code> in the file
|
||
|
dset.h5 is contiguous, as shown by the <code style="background-color:whitesmoke;">h5dump -pH</code> command.
|
||
|
The <code style="background-color:whitesmoke;">h5repack</code> utility writes dset.h5 to a new file, dsetrpk.h5,
|
||
|
where the dataset <code style="background-color:whitesmoke;">dset</code> is chunked. This can be seen by examining
|
||
|
the resulting dsetrpk.h5 file with <code style="background-color:whitesmoke;">h5dump</code>, as shown:
|
||
|
\code
|
||
|
$ h5dump -pH dset.h5
|
||
|
HDF5 "dset.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
|
||
|
STORAGE_LAYOUT {
|
||
|
CONTIGUOUS
|
||
|
SIZE 96
|
||
|
OFFSET 1400
|
||
|
}
|
||
|
FILTERS {
|
||
|
NONE
|
||
|
}
|
||
|
FILLVALUE {
|
||
|
FILL_TIME H5D_FILL_TIME_IFSET
|
||
|
VALUE 0
|
||
|
}
|
||
|
ALLOCATION_TIME {
|
||
|
H5D_ALLOC_TIME_LATE
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
|
||
|
$ h5repack -l dset:CHUNK=4x6 dset.h5 dsetrpk.h5
|
||
|
|
||
|
$ h5dump -pH dsetrpk.h5
|
||
|
HDF5 "dsetrpk.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
|
||
|
STORAGE_LAYOUT {
|
||
|
CHUNKED ( 4, 6 )
|
||
|
SIZE 96
|
||
|
}
|
||
|
FILTERS {
|
||
|
NONE
|
||
|
}
|
||
|
FILLVALUE {
|
||
|
FILL_TIME H5D_FILL_TIME_IFSET
|
||
|
VALUE 0
|
||
|
}
|
||
|
ALLOCATION_TIME {
|
||
|
H5D_ALLOC_TIME_INCR
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
There can be many reasons that the storage layout needs to be changed for a dataset. For example,
|
||
|
there may be a performance issue with a dataset due to a small chunk size.
|
||
|
|
||
|
\section secViewToolsEditApply Apply Compression Filter to a Dataset
|
||
|
The <code style="background-color:whitesmoke;">h5repack</code> utility can be used to compress or
|
||
|
remove compression from a dataset in a file. By default, compression cannot be added to or removed
|
||
|
from a dataset once it has been created. However, with <code style="background-color:whitesmoke;">h5repack</code>
|
||
|
you can write a file to a new file and specify a compression filter to apply to a dataset or datasets in the new file.
|
||
|
|
||
|
To apply a filter to an object in an HDF5 file, specify the <code style="background-color:whitesmoke;">-f</code> option,
|
||
|
where the string following the <code style="background-color:whitesmoke;">-f</code> option defines the filter and
|
||
|
its parameters (if there are any) to apply to a given object or objects:
|
||
|
\code
|
||
|
h5repack -f [list of objects:]<name of filter>=<filter parameters> <input file> <output file>
|
||
|
\endcode
|
||
|
|
||
|
If no objects are specified then everything in the input file will be written to the output file with
|
||
|
the filter and parameters specified. If objects are specified, then everything in the input file will
|
||
|
be written to the output file as is, except for the specified objects. They will be written to the
|
||
|
output file with the filter and parameters specified.
|
||
|
|
||
|
If you type <code style="background-color:whitesmoke;">h5repack --help</code> on the command line,
|
||
|
you will see a detailed usage statement with examples of modifying a filter. There are actually
|
||
|
numerous filters that you can apply to a dataset:
|
||
|
<table>
|
||
|
<tr>
|
||
|
<th>Filter<th></th>Options</th>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>GZIP compression (levels 1-9)
|
||
|
<td>GZIP=<deflation level>
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>SZIP compression
|
||
|
<td>SZIP=<pixels per block,coding>
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>Shuffle filter
|
||
|
<td>SHUF
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>Checksum filter
|
||
|
<td>FLET
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>NBIT compression
|
||
|
<td>NBIT
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>HDF5 Scale/Offset filter
|
||
|
<td>SOFF=<scale_factor,scale_type>
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>User defined filter
|
||
|
<td>UD=<filter_number,cd_value_count,value_1[,value_2,...,value_N]>
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>Remove ALL filters
|
||
|
</td>
|
||
|
<td>NONE
|
||
|
</td>
|
||
|
</tr>
|
||
|
</table>
|
||
|
|
||
|
Be aware that a dataset must be chunked to apply compression to it. If the dataset is not already chunked,
|
||
|
then <code style="background-color:whitesmoke;">h5repack</code> will apply chunking to it. Both chunking
|
||
|
and compression cannot be applied to a dataset at the same time with <code style="background-color:whitesmoke;">h5repack</code>.
|
||
|
|
||
|
In the following example,
|
||
|
\li <em>h5dump</em> lists the properties for the objects in <em>dset.h5</em>. Note that the dataset <em>dset</em> is contiguous.
|
||
|
\li <em>h5repack</em> writes dset.h5 into a new file <em>dsetrpk.h5</em>, applying GZIP Level 5 compression to the dataset <em>/dset</em> in dsetrpk.h5.
|
||
|
\li <em>h5dump</em> lists the properties for the new <em>dsetrpk.h5</em> file. Note that <em>/dset</em> is both compressed and chunked.
|
||
|
|
||
|
<em>Example</em>
|
||
|
\code
|
||
|
$ h5dump -pH dset.h5
|
||
|
HDF5 "dset.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 12, 18 ) / ( 12, 18 ) }
|
||
|
STORAGE_LAYOUT {
|
||
|
CONTIGUOUS
|
||
|
SIZE 864
|
||
|
OFFSET 1400
|
||
|
}
|
||
|
FILTERS {
|
||
|
NONE
|
||
|
}
|
||
|
FILLVALUE {
|
||
|
FILL_TIME H5D_FILL_TIME_IFSET
|
||
|
VALUE 0
|
||
|
}
|
||
|
ALLOCATION_TIME {
|
||
|
H5D_ALLOC_TIME_LATE
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
|
||
|
$ h5repack -f dset:GZIP=5 dset.h5 dsetrpk.h5
|
||
|
|
||
|
$ h5dump -pH dsetrpk.h5
|
||
|
HDF5 "dsetrpk.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 12, 18 ) / ( 12, 18 ) }
|
||
|
STORAGE_LAYOUT {
|
||
|
CHUNKED ( 12, 18 )
|
||
|
SIZE 160 (5.400:1 COMPRESSION)
|
||
|
}
|
||
|
FILTERS {
|
||
|
COMPRESSION DEFLATE { LEVEL 5 }
|
||
|
}
|
||
|
FILLVALUE {
|
||
|
FILL_TIME H5D_FILL_TIME_IFSET
|
||
|
VALUE 0
|
||
|
}
|
||
|
ALLOCATION_TIME {
|
||
|
H5D_ALLOC_TIME_INCR
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
\section secViewToolsEditCopy Copy Objects to Another File
|
||
|
The <code style="background-color:whitesmoke;">h5copy</code> utility can be used to copy an object or
|
||
|
objects from one HDF5 file to another or to a different location in the same file. It uses the
|
||
|
#H5Ocopy and #H5Lcopy APIs in HDF5.
|
||
|
|
||
|
Following are some of the options that can be used with <code style="background-color:whitesmoke;">h5copy</code>.
|
||
|
<table>
|
||
|
<tr>
|
||
|
<th>h5copy Options</th><th>Description</th>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-i, --input
|
||
|
</td>
|
||
|
<td>Input file name
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-o, --output
|
||
|
</td>
|
||
|
<td>Output file name
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-s, --source
|
||
|
</td>
|
||
|
<td>Source object name
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-d, --destination
|
||
|
</td>
|
||
|
<td>Destination object name
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-p, --parents
|
||
|
</td>
|
||
|
<td>Make parent groups as needed
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-v, --verbose
|
||
|
</td>
|
||
|
<td>Verbose mode
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-f, --flag
|
||
|
</td>
|
||
|
<td>Flag type
|
||
|
</td>
|
||
|
</tr>
|
||
|
</table>
|
||
|
|
||
|
For a complete list of options and information on using <code style="background-color:whitesmoke;">h5copy</code>, type:
|
||
|
\code
|
||
|
h5copy --help
|
||
|
\endcode
|
||
|
|
||
|
In the example below, the dataset <code style="background-color:whitesmoke;">/MyGroup/Group_A/dset2</code>
|
||
|
in <code style="background-color:whitesmoke;">groups.h5</code> gets copied to the root
|
||
|
("<code style="background-color:whitesmoke;">/</code>") group of a new file,
|
||
|
<code style="background-color:whitesmoke;">newgroup.h5</code>, with the name
|
||
|
<code style="background-color:whitesmoke;">dset3</code>:
|
||
|
\code
|
||
|
$h5dump -H groups.h5
|
||
|
HDF5 "groups.h5" {
|
||
|
GROUP "/" {
|
||
|
GROUP "MyGroup" {
|
||
|
GROUP "Group_A" {
|
||
|
DATASET "dset2" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 2, 10 ) / ( 2, 10 ) }
|
||
|
}
|
||
|
}
|
||
|
GROUP "Group_B" {
|
||
|
}
|
||
|
DATASET "dset1" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 3, 3 ) / ( 3, 3 ) }
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
|
||
|
$ h5copy -i groups.h5 -o newgroup.h5 -s /MyGroup/Group_A/dset2 -d /dset3
|
||
|
|
||
|
$ h5dump -H newgroup.h5
|
||
|
HDF5 "newgroup.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "dset3" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 2, 10 ) / ( 2, 10 ) }
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
There are also <code style="background-color:whitesmoke;">h5copy</code> flags that can be specified
|
||
|
with the <code style="background-color:whitesmoke;">-f</code> option. In the example below, the
|
||
|
<code style="background-color:whitesmoke;">-f shallow</code> option specifies to copy only the
|
||
|
immediate members of the group <code style="background-color:whitesmoke;">/MyGroup</code> from
|
||
|
the <code style="background-color:whitesmoke;">groups.h5</code> file mentioned above to a new
|
||
|
file <code style="background-color:whitesmoke;">mygrouponly.h5</code>:
|
||
|
\code
|
||
|
h5copy -v -i groups.h5 -o mygrouponly.h5 -s /MyGroup -d /MyGroup -f shallow
|
||
|
\endcode
|
||
|
|
||
|
The output of the above command is shown below. The verbose option <code style="background-color:whitesmoke;">-v</code>
|
||
|
describes the action that was taken, as shown in the highlighted text.
|
||
|
\code
|
||
|
Copying file <groups.h5> and object </MyGroup> to file <mygrouponly.h5> and object </MyGroup>
|
||
|
Using shallow flag
|
||
|
|
||
|
$ h5dump -H mygrouponly.h5
|
||
|
HDF5 "mygrouponly.h5" {
|
||
|
GROUP "/" {
|
||
|
GROUP "MyGroup" {
|
||
|
GROUP "Group_A" {
|
||
|
}
|
||
|
GROUP "Group_B" {
|
||
|
}
|
||
|
DATASET "dset1" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 3, 3 ) / ( 3, 3 ) }
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
\section secViewToolsEditAdd Add or Remove User Block from File
|
||
|
The user block is a space in an HDF5 file that is not interpreted by the HDF5 library. It is a property
|
||
|
list that can be added when creating a file. See the #H5Pset_userblock API in the \ref RM for more
|
||
|
information regarding this property.
|
||
|
|
||
|
Once created in a file, the user block cannot be removed. However, you can use the
|
||
|
<code style="background-color:whitesmoke;">h5jam</code> and <code style="background-color:whitesmoke;">h5unjam</code>
|
||
|
utilities to add or remove a user block from a file into a new file.
|
||
|
|
||
|
These two utilities work similarly, except that <code style="background-color:whitesmoke;">h5jam</code>
|
||
|
adds a user block to a file and <code style="background-color:whitesmoke;">h5unjam</code> removes the user
|
||
|
block. You can also overwrite or delete a user block in a file.
|
||
|
|
||
|
Specify the <code style="background-color:whitesmoke;">-h</code> option to see a complete list of options
|
||
|
that can be used with <code style="background-color:whitesmoke;">h5jam</code> and
|
||
|
<code style="background-color:whitesmoke;">h5unjam</code>. For example:
|
||
|
\code
|
||
|
h5jam -h
|
||
|
h5unjam -h
|
||
|
\endcode
|
||
|
|
||
|
Below are the basic options for adding or removing a user block with <code style="background-color:whitesmoke;">h5jam</code>
|
||
|
and <code style="background-color:whitesmoke;">h5unjam</code>:
|
||
|
|
||
|
<table>
|
||
|
<tr>
|
||
|
<th>h5copy Options</th><th>Description</th>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-i
|
||
|
</td>
|
||
|
<td>Input File
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-o
|
||
|
</td>
|
||
|
<td>Output File
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-u
|
||
|
</td>
|
||
|
<td>File to add or remove from user block
|
||
|
</td>
|
||
|
</tr>
|
||
|
</table>
|
||
|
|
||
|
Let's say you wanted to add the program that creates an HDF5 file to its user block. As an example, you
|
||
|
can take the <code style="background-color:whitesmoke;">h5_crtgrpar.c</code> program from the
|
||
|
\ref LBExamples
|
||
|
and add it to the file it creates, <code style="background-color:whitesmoke;">groups.h5</code>. This can
|
||
|
be done with <code style="background-color:whitesmoke;">h5jam</code>, as follows:
|
||
|
\code
|
||
|
h5jam -i groups.h5 -u h5_crtgrpar.c -o groupsub.h5
|
||
|
\endcode
|
||
|
|
||
|
You can actually view the file with more <code style="background-color:whitesmoke;">groupsub.h5</code>
|
||
|
to see that the <code style="background-color:whitesmoke;">h5_crtgrpar.c</code> file is indeed included.
|
||
|
|
||
|
To remove the user block that was just added, type:
|
||
|
\code
|
||
|
h5unjam -i groupsub.h5 -u h5_crtgrparNEW.c -o groups-noub.h5
|
||
|
\endcode
|
||
|
|
||
|
This writes the user block in the file <code style="background-color:whitesmoke;">groupsub.h5</code>
|
||
|
into <code style="background-color:whitesmoke;">h5_crtgrparNEW.c</code>. The new HDF5 file,
|
||
|
<code style="background-color:whitesmoke;">groups-noub.h5</code>, will not contain a user block.
|
||
|
|
||
|
<hr>
|
||
|
Navigate back: \ref index "Main" / \ref GettingStarted / \ref ViewToolsCommand
|
||
|
|
||
|
*/
|
||
|
|
||
|
/** @page ViewToolsConvert Command-line Tools For Converting HDF5 Files
|
||
|
Navigate back: \ref index "Main" / \ref GettingStarted / \ref ViewToolsCommand
|
||
|
<hr>
|
||
|
|
||
|
\section secViewToolsConvertTOC Contents
|
||
|
<ul>
|
||
|
<li>\ref secViewToolsConvertASCII</li>
|
||
|
<li>\ref secViewToolsConvertBinary</li>
|
||
|
<li>\ref secViewToolsConvertExport</li>
|
||
|
</ul>
|
||
|
|
||
|
\section secViewToolsConvertASCII Output HDF5 Dataset into an ASCII File (to Import into Excel and Other Applications)
|
||
|
The <code style="background-color:whitesmoke;">h5dump</code> utility can be used to convert an HDF5 dataset
|
||
|
into an ASCII file, which can then be imported into Excel and other applications. The following options are used:
|
||
|
<table>
|
||
|
<tr>
|
||
|
<th>Options</th><th>Description</th>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td> -d D, --dataset=D
|
||
|
</td>
|
||
|
<td>Display dataset D
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td> -o F, --output=F
|
||
|
</td>
|
||
|
<td>Output raw data into file F
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td> -y, --noindex
|
||
|
</td>
|
||
|
<td>Suppress printing of array indices with the data
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td> -w N, --width=N
|
||
|
</td>
|
||
|
<td>Set N number of columns of output. A value of 0
|
||
|
sets the number to 65535 (the maximum)
|
||
|
</td>
|
||
|
</tr>
|
||
|
</table>
|
||
|
|
||
|
As an example, <code style="background-color:whitesmoke;">h5_crtdat.c</code> from the \ref LBDsetCreate
|
||
|
HDF5 Tutorial topic, creates the file <code style="background-color:whitesmoke;">dset.h5</code> with
|
||
|
a dataset <code style="background-color:whitesmoke;">/dset</code> that is a 4 x 6 integer array. The
|
||
|
following is displayed when viewing <code style="background-color:whitesmoke;">dset.h5</code> with
|
||
|
<code style="background-color:whitesmoke;">h5dump</code>:
|
||
|
\code
|
||
|
$ h5dump dset.h5
|
||
|
HDF5 "dset.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
|
||
|
DATA {
|
||
|
(0,0): 1, 2, 3, 4, 5, 6,
|
||
|
(1,0): 7, 8, 9, 10, 11, 12,
|
||
|
(2,0): 13, 14, 15, 16, 17, 18,
|
||
|
(3,0): 19, 20, 21, 22, 23, 24
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
The following command will output the values of the <code style="background-color:whitesmoke;">/dset</code>
|
||
|
dataset to the ASCII file <code style="background-color:whitesmoke;">dset.asci</code>:
|
||
|
\code
|
||
|
h5dump -d /dset -o dset.asci -y -w 50 dset.h5
|
||
|
\endcode
|
||
|
|
||
|
In particular, note that:
|
||
|
\li The default behavior of <code style="background-color:whitesmoke;">h5dump</code> is to print indices,
|
||
|
and the <code style="background-color:whitesmoke;">-y</code> option suppresses this.
|
||
|
\li The <code style="background-color:whitesmoke;">-w 50</code> option tells
|
||
|
<code style="background-color:whitesmoke;">h5dump</code> to allow 50 columns for outputting the data. The
|
||
|
value specified must be large enough to accommodate the dimension size of the dataset multiplied by the
|
||
|
number of positions and spaces needed to print each value. If the value is not large enough, the output
|
||
|
will wrap to the next line, and the data will not display as expected in Excel or other applications. To
|
||
|
ensure that the output does not wrap to the next line, you can also specify 0 (zero) for the
|
||
|
<code style="background-color:whitesmoke;">-w</code> option.
|
||
|
|
||
|
In addition to creating the ASCII file <code style="background-color:whitesmoke;">dset.asci</code>, the
|
||
|
above command outputs the metadata of the specified dataset:
|
||
|
\code
|
||
|
HDF5 "dset.h5" {
|
||
|
DATASET "/dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
|
||
|
DATA {
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
The <code style="background-color:whitesmoke;">dset.asci</code> file will contain the values for the dataset:
|
||
|
\code
|
||
|
1, 2, 3, 4, 5, 6,
|
||
|
7, 8, 9, 10, 11, 12,
|
||
|
13, 14, 15, 16, 17, 18,
|
||
|
19, 20, 21, 22, 23, 24
|
||
|
\endcode
|
||
|
|
||
|
\section secViewToolsConvertBinary Output HDF5 Dataset into Binary File
|
||
|
The <code style="background-color:whitesmoke;">h5dump</code> utility can be used to convert an
|
||
|
HDF5 dataset to a binary file with the following options:
|
||
|
<table>
|
||
|
<tr>
|
||
|
<th>Options</th><th>Description</th>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-d D, --dataset=D
|
||
|
</td>
|
||
|
<td>Display dataset D
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-o F, --output=F
|
||
|
</td>
|
||
|
<td>Output raw data into file F
|
||
|
</td>
|
||
|
</tr>
|
||
|
<tr>
|
||
|
<td>-b B, --binary=B
|
||
|
</td>
|
||
|
<td>Binary file output of form B.
|
||
|
Valid values are: LE, BE, NATIVE, FILE
|
||
|
</td>
|
||
|
</tr>
|
||
|
</table>
|
||
|
|
||
|
As an example, <code style="background-color:whitesmoke;">h5_crtdat.c</code> from the
|
||
|
\ref LBDsetCreate HDF5 Tutorial topic, creates the file dset.h5 with a dataset
|
||
|
<code style="background-color:whitesmoke;">/dset</code> that is a 4 x 6 integer array. The
|
||
|
following is displayed when viewing <code style="background-color:whitesmoke;">dset.h5</code>
|
||
|
with <code style="background-color:whitesmoke;">h5dump</code>:
|
||
|
\code
|
||
|
$ h5dump -d /dset/ dset.h5
|
||
|
HDF5 "dset.h5" {
|
||
|
DATASET "/dset/" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
|
||
|
DATA {
|
||
|
(0,0): 1, 2, 3, 4, 5, 6,
|
||
|
(1,0): 7, 8, 9, 10, 11, 12,
|
||
|
(2,0): 13, 14, 15, 16, 17, 18,
|
||
|
(3,0): 19, 20, 21, 22, 23, 24
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
As specified by the <code style="background-color:whitesmoke;">-d</code> and
|
||
|
<code style="background-color:whitesmoke;">-o</code> options, the following
|
||
|
<code style="background-color:whitesmoke;">h5dump</code> command will output the values of the dataset
|
||
|
<code style="background-color:whitesmoke;">/dset </code>to a file called
|
||
|
<code style="background-color:whitesmoke;">dset.bin</code>. The <code style="background-color:whitesmoke;">-b</code>
|
||
|
option specifies that the output will be binary in Little Endian format (LE).
|
||
|
|
||
|
\code
|
||
|
h5dump -d /dset -b LE -o dset.bin dset.h5
|
||
|
\endcode
|
||
|
|
||
|
This command outputs the metadata for the dataset, as well as creating the binary file
|
||
|
<code style="background-color:whitesmoke;">dset.bin</code>:
|
||
|
\code
|
||
|
HDF5 "dset.h5" {
|
||
|
DATASET "/dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
|
||
|
DATA {
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
If you look at the resulting <code style="background-color:whitesmoke;">dset.bin</code> file with
|
||
|
a binary editor, you will see that it contains the dataset's values. For example (on Linux) you will see:
|
||
|
\code
|
||
|
$ od -t d dset.bin
|
||
|
0000000 1 2 3 4
|
||
|
0000020 5 6 7 8
|
||
|
0000040 9 10 11 12
|
||
|
0000060 13 14 15 16
|
||
|
0000100 17 18 19 20
|
||
|
0000120 21 22 23 24
|
||
|
0000140
|
||
|
\endcode
|
||
|
|
||
|
\section secViewToolsConvertExport Export from h5dump and Import into HDF5
|
||
|
The <code style="background-color:whitesmoke;">h5import</code> utility can use the output of
|
||
|
<code style="background-color:whitesmoke;">h5dump</code> as input to create a dataset or file.
|
||
|
|
||
|
The <code style="background-color:whitesmoke;">h5dump</code> utility must first create two files:
|
||
|
\li A DDL file, which will be used as an <code style="background-color:whitesmoke;">h5import</code> configuration file
|
||
|
\li A raw data file containing the data to be imported
|
||
|
|
||
|
The DDL file must be generated with the <code style="background-color:whitesmoke;">h5dump -p</code> option, to generate properties.
|
||
|
|
||
|
The raw data file that can be imported into HDF5 using this method may contain either numeric or string data with the following restrictions:
|
||
|
\li Numeric data requires the use of the <code style="background-color:whitesmoke;">h5dump -b</code> option to produce a binary data file.
|
||
|
\li String data must be written with the <code style="background-color:whitesmoke;">h5dump -y</code> and
|
||
|
<code style="background-color:whitesmoke;">--width=1</code> options, generating a single column of strings without indices.
|
||
|
|
||
|
Two examples follow: the first imports a dataset with a numeric datatype. Note that numeric data requires
|
||
|
the use of the <code style="background-color:whitesmoke;">h5dump -b</code> option to produce a binary data
|
||
|
file. The example program (<code style="background-color:whitesmoke;">h5_crtdat.c</code>) that creates this
|
||
|
file is included with the \ref IntroHDF5 tutorial and can be obtained from the \ref LBExamples page:
|
||
|
\code
|
||
|
h5dump -p -d "/dset" --ddl=dsetbin.dmp -o dset.bin -b dset.h5
|
||
|
h5import dset.bin -c dsetbin.dmp -o new-dset.h5
|
||
|
\endcode
|
||
|
|
||
|
The output before and after running these commands is shown below:
|
||
|
\code
|
||
|
$ h5dump dset.h5
|
||
|
HDF5 "dset.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
|
||
|
DATA {
|
||
|
(0,0): 1, 2, 3, 4, 5, 6,
|
||
|
(1,0): 7, 8, 9, 10, 11, 12,
|
||
|
(2,0): 13, 14, 15, 16, 17, 18,
|
||
|
(3,0): 19, 20, 21, 22, 23, 24
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
$ h5dump -p -d "/dset" --ddl=dsetbin.dmp -o dset.bin -b dset.h5
|
||
|
|
||
|
$ h5import dset.bin -c dsetbin.dmp -o new-dset.h5
|
||
|
|
||
|
$ h5dump new-dset.h5
|
||
|
HDF5 "new-dset.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "dset" {
|
||
|
DATATYPE H5T_STD_I32BE
|
||
|
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
|
||
|
DATA {
|
||
|
(0,0): 1, 2, 3, 4, 5, 6,
|
||
|
(1,0): 7, 8, 9, 10, 11, 12,
|
||
|
(2,0): 13, 14, 15, 16, 17, 18,
|
||
|
(3,0): 19, 20, 21, 22, 23, 24
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
The second example imports string data. The example program that creates this file can be downloaded
|
||
|
from the <a href="https://portal.hdfgroup.org/display/HDF5/Examples+by+API">Examples by API</a> page.
|
||
|
|
||
|
Note that string data requires use of the <code style="background-color:whitesmoke;">h5dump -y</code>
|
||
|
option to exclude indexes and the <code style="background-color:whitesmoke;">h5dump --width=1</code>
|
||
|
option to generate a single column of strings. The <code style="background-color:whitesmoke;">-o</code>
|
||
|
option outputs the data into an ASCII file.
|
||
|
\code
|
||
|
h5dump -p -d "/DS1" -O vlstring.dmp -o vlstring.ascii -y --width=1 h5ex_t_vlstring.h5
|
||
|
h5import vlstring.ascii -c vlstring.dmp -o new-vlstring.h5
|
||
|
\endcode
|
||
|
|
||
|
The output before and after running these commands is shown below:
|
||
|
\code
|
||
|
$ h5dump h5ex_t_vlstring.h5
|
||
|
HDF5 "h5ex_t_vlstring.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "DS1" {
|
||
|
DATATYPE H5T_STRING {
|
||
|
STRSIZE H5T_VARIABLE;
|
||
|
STRPAD H5T_STR_SPACEPAD;
|
||
|
CSET H5T_CSET_ASCII;
|
||
|
CTYPE H5T_C_S1;
|
||
|
}
|
||
|
DATASPACE SIMPLE { ( 4 ) / ( 4 ) }
|
||
|
DATA {
|
||
|
(0): "Parting", "is such", "sweet", "sorrow."
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
|
||
|
$ h5dump -p -d "/DS1" -O vlstring.dmp -o vlstring.ascii -y --width=1 h5ex_t_vlstring.h5
|
||
|
|
||
|
$ h5import vlstring.ascii -c vlstring.dmp -o new-vlstring.h5
|
||
|
|
||
|
$ h5dump new-vlstring.h5
|
||
|
HDF5 "new-vlstring.h5" {
|
||
|
GROUP "/" {
|
||
|
DATASET "DS1" {
|
||
|
DATATYPE H5T_STRING {
|
||
|
STRSIZE H5T_VARIABLE;
|
||
|
STRPAD H5T_STR_NULLTERM;
|
||
|
CSET H5T_CSET_ASCII;
|
||
|
CTYPE H5T_C_S1;
|
||
|
}
|
||
|
DATASPACE SIMPLE { ( 4 ) / ( 4 ) }
|
||
|
DATA {
|
||
|
(0): "Parting", "is such", "sweet", "sorrow."
|
||
|
}
|
||
|
}
|
||
|
}
|
||
|
\endcode
|
||
|
|
||
|
<hr>
|
||
|
Navigate back: \ref index "Main" / \ref GettingStarted / \ref ViewToolsCommand
|
||
|
|
||
|
*/
|