mirror/hdf5
2
0
Fork 0
mirror of https://github.com/HDFGroup/hdf5.git synced 2024-11-21 01:04:10 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add szip documentation to dataset section of user guide (#4997)

Browse Source
This commit is contained in:
Allen Byrne 2024-10-23 21:37:15 -05:00 committed by GitHub
parent 8efc084646
commit 26cf1640b2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 111 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

112
src/H5Dmodule.h
View File

@ -2733,7 +2733,117 @@ allocated if necessary.
* </li></ol>
*
* \subsubsection subsubsec_dataset_filters_szip Using the SZip Filter
* See The HDF Group website for further information regarding the SZip filter.
* Szip compression software, providing lossless compression of scientific data, has been provided with HDF
* software products as of HDF5 Release 1.6.0. Szip is an implementation of the extended-Rice lossless
* compression algorithm. The Consultative Committee on Space Data Systems (CCSDS) has adopted the
* extended-Rice algorithm for international standards for space applications[1,6,7]. Szip is reported
* to provide fast and effective compression, specifically for the EOS data generated by the NASA Earth
* Observatory System (EOS)[1].
* It was originally developed at University of New Mexico (UNM) and integrated with HDF by UNM researchers
* and developers.
*
* The primary gain with Szip compression is in speed of processing. Szip also provides some advantage in
* compression ratio over other compression methods.
*
* <h4>Szip and HDF5</h4>
* Using Szip compression in HDF5: Szip is a stand-alone library that is configured as an optional filter in
* HDF5.
* Depending on which Szip library is used (encoder enabled or decode-only), an HDF5 application can
* create, write, and read datasets compressed with Szip compression, or can only read datasets
* compressed with Szip.
* Applications use Szip by setting Szip as an optional filter when a dataset is created. If the Szip
* encoder is enabled with the HDF5 library, data is automatically compressed and decompressed with
* Szip during I/O.
* If only the decoder is present, the HDF5 library cannot create and write Szip-compressed datasets,
* but it automatically decompresses Szip-compressed data when data is read.
*
* This sample HDF5 program illustrates the use of Szip compression with HDF5.
* \code
* Example of Szip Usage in HDF5
* The following sample program illustrates the use of Szip compression in HDF5. This sample program is
* also available in the subdirectory HDF5Examples.
* #include "hdf5.h"
* #define NX 500
* #define NY 600
* #define CH_NX 100
* #define CH_NY 25
*
* int main(void)
* {
* hid_t file, data_space, dataset32, properties;
* float buf[NX][NY];
* float buf_r[NX][NY];
* hsize_t dims[2], chunk_size[2];
* int i, j;
* unsigned szip_options_mask;
* unsigned szip_pixels_per_block;
*
* // Initialize data buffer with some bogus data.
* for (i=0; i < NX; i++) {
* for (j=0; j < NY; j++)
* buf[i][j] = i + j;
* }
*
* // Describe the size of the array.
* dims[0] = NX;
* dims[1] = NY;
* data_space = H5Screate_simple (2, dims, NULL);
*
* // Create a new file using read/write access, default file
* // creation properties, and default file access properties.
* file = H5Fcreate ("test.h5", H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);
*
* // Set the dataset creation property list to specify that
* // the raw data is to be partitioned into 100x100 element
* // chunks and that each chunk is to be compressed.
* chunk_size[0] = CH_NX;
* chunk_size[1] = CH_NY;
* properties = H5Pcreate (H5P_DATASET_CREATE);
* H5Pset_chunk (properties, 2, chunk_size);
*
* // Set parameters for SZIP compression; check the description of
* // the H5Pset_szip function in the HDF5 Reference Manual for more
* // information.
* szip_options_mask=H5_SZIP_NN_OPTION_MASK;
* szip_pixels_per_block=32;
*
* H5Pset_szip (properties, szip_options_mask, szip_pixels_per_block);
*
* // Create a new dataset within the file. The datatype
* // and data space describe the data on disk, which may
* // be different from the format used in the application's
* // memory.
*
* dataset32 = H5Dcreate (file, "datasetF32", H5T_NATIVE_FLOAT, data_space, properties);
*
* // Write the array to the file. The datatype and dataspace
* // describe the format of the data in the `buf' buffer.
* // The raw data is translated to the format required on disk,
* // as defined above. We use default raw data transfer properties.
*
* H5Dwrite (dataset32, H5T_NATIVE_FLOAT, H5S_ALL, H5S_ALL, H5P_DEFAULT, buf);
*
* // Read the array. This is similar to writing data,
* // except the data flows in the opposite direction.
* // Note: Decompression is automatic.
*
* H5Dread (dataset32, H5T_NATIVE_FLOAT, H5S_ALL, H5S_ALL, H5P_DEFAULT, buf_r);
*
* H5Dclose (dataset32);
* H5Sclose (data_space);
* H5Pclose (properties);
* H5Fclose (file);
* }
* \endcode
*
* Details of the required and optional parameters are provided in the H5Pset_szip entry
* in the HDF5 Reference Manual.
*
* Software distribution: Starting with Release 1.6.0, HDF5 has been distributed with Szip enabled,
* making it easier to use Szip compression. The software is distributed as follows:
* - Pre-compiled HDF5 binaries are provided with the Szip encoder enabled.
* - Szip source code can be obtained at:
* <a href="https://github.com/MathisRosenhauer/libaec.git">LIBAEC(SZIP) repository</a>
*
* \subsubsection subsubsec_dataset_filters_dyn Using Dynamically-Loadable Filters
* see \ref sec_filter_plugins for further information regarding the dynamically-loadable filters.