mirror of
https://github.com/HDFGroup/hdf5.git
synced 2025-01-18 15:15:56 +08:00
a0445d806c
* Take a stab at using constructors to initialize instead of function-entry macros. This is a work in progress. It's good enough to run `many_dsets`. * Committing clang-format changes * Add the `many_dsets` benchmark and some scripts I used on jelly for setting up the build/test environment and for recording/flame-graphing profiles. * Committing clang-format changes * Change my Makefile and environment script to work both on jelly and on mayll (and probably on Summit). * Disable clang-format "fix." * Replace the `if (!H5_TERM_GLOBAL)` test in each FUNC_ENTER_ macro with `if (true)`. * Fix bad grammar in a comment. * Instead of labeling the H5*__init_package routines constructors, fold each into an initialization routine, H5*_init(), and call each of the H5*_init() routines. Call most of the H5*_init() routines from H5_init_library() in an explicit order that I found out earlier by instrumenting each __init_package routine and running the library tests. Roll H5FD*__init_package routines into H5FD*_init() routines. This change ends just-in-time initialization of package dependencies by package initializers. Don't track in per-package variables (H5_PKG_INIT_VAR) whether each package has been initialized. Instead, track in a single library variable whether the whole library is initialized or not. Drive the initialization of packages by H5_init_library() with a table of initializer routines. Also drive the termination of packages by H5_term_library() with a table. Perform initialization as needed from FUNC_ENTER_API_INIT(err). This basically restores the old behavior of that macro. Delete a bunch of #definitions in H5private.h that have fallen out of use with these changes. * Committing clang-format changes * Undo the bad auto-formatting that appears to have occurred in spite of my disabling it. Bracket some code in /* clang-format off */ /* clang-format on */ to prevent a recurrence. * Remove a diagnostic abort(). * Fix a logic error: print a comma between every package terminator run, and don't print an initial comma. * Complete the changes I started in H5_term_library() that undo the bad auto-formatting. Stop tracking whether package "tops" were initialized in per-package variables H5*_top_package_initialize_s. H5_term_library() takes care of that for them. Remove H5R_top_term_package() and H5R_term_package(), they don't do anything. * Committing clang-format changes * NFCI. Simplify macro text: replace `if (true) {` with `{`. * Fix formatting and suppress clang-format on a longer range. * Quiet some unused label, unused variable complaints that cropped up after I simplified the FUNC_ENTER_ macros for the sake of performance. * Committing clang-format changes * Delete some programs and scripts that don't belong in the pull request. * Use the right function-entry macro. * Use a sensible format and disable auto-formatting. * Stop calling do-nothing initializer H5FS_init(). Delete it. * Document what changes to make if the default VFD changes. * While I am here, change an `await_prior` flag on the terminator table to `true` to match the previous, non-table-driven code that was here. Found the oversight making the following changes: NFCI: insert an empty line and copy over slightly-edited comments from the previous version, where those comments still correctly explained how library termination operated. * NFCI: lower a staircase. * Replace every occurrence of FUNC_ENTER_NOAPI_INIT(...) with H5_PUSH_FUNC since that is all that that macro does any more. Quiet a bunch of new warnings by changing FUNC_ENTER_NOAPI(...) to FUNC_ENTER_NOAPI_NOERR and removing disused `done:` labels. * NFCI: add curly braces around a multiline statement. * Quiet a signed/unsigned comparison warning. * Add some documentation about library initialization and shutdown. * Make sure that the library is initialized, or else that initialization is already underway, before performing any VFD's initialization. * Committing clang-format changes * Committing clang-format changes * Reduce differences from `develop` branch. * Always initialize `tot_init`. * Committing clang-format changes * Fix typo: H5SL_init initializes skip lists, not VOL. * Remove H5_TERM_GLOBAL test in H5T_init. H5T_init was unusual in that it tested H5_TERM_GLOBAL and exited early if it was set. No other module initializers did that, and I cannot find any reason that should be necessary. Tests still pass when I remove it, so away it goes. * Use HD prefix. * Add function header comments. * Drop the intermediate variable, it's only used once. * Extract subroutine `H5FDperform_init(hid_t (*init)(void))` that initializes the library, if necessary, before calling its VFD-initializer argument. Use H5FDperform_init in the definition of the symbols H5FD_<vfd> (e.g., H5FD_SEC2), which may be evaluated before the library is initialized, like so: ``` ``` I implement H5FDperform_init in its own source file, H5FDperform.c, and exclude that file from trace processing because the `bin/trace` cannot deal with the function-pointer type. * Straggler from last: add new source file src/H5FDperform.c. * Committing clang-format changes * Add a missing file to the MANIFEST. * Switch to FUNC_ENTER_API_NOINIT in H5FDperform_init() and hbool_t in H5_term_library(). Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
257 lines
10 KiB
C
257 lines
10 KiB
C
/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
|
* Copyright by The HDF Group. *
|
|
* Copyright by the Board of Trustees of the University of Illinois. *
|
|
* All rights reserved. *
|
|
* *
|
|
* This file is part of HDF5. The full HDF5 copyright notice, including *
|
|
* terms governing use, modification, and redistribution, is contained in *
|
|
* the COPYING file, which can be found at the root of the source code *
|
|
* distribution tree, or in https://www.hdfgroup.org/licenses. *
|
|
* If you do not have access to either file, you may request a copy from *
|
|
* help@hdfgroup.org. *
|
|
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
|
|
|
|
/*
|
|
* Programmer: Robb Matzke
|
|
* Monday, August 2, 1999
|
|
*
|
|
* Purpose: The public header file for the "multi" driver.
|
|
*/
|
|
#ifndef H5FDmulti_H
|
|
#define H5FDmulti_H
|
|
|
|
#define H5FD_MULTI (H5FDperform_init(H5FD_multi_init))
|
|
|
|
#ifdef __cplusplus
|
|
extern "C" {
|
|
#endif
|
|
H5_DLL hid_t H5FD_multi_init(void);
|
|
|
|
/**
|
|
* \ingroup FAPL
|
|
*
|
|
* \brief Sets up use of the multi-file driver
|
|
*
|
|
* \fapl_id
|
|
* \param[in] memb_map Maps memory usage types to other memory usage types
|
|
* \param[in] memb_fapl Property list for each memory usage type
|
|
* \param[in] memb_name Name generator for names of member files
|
|
* \param[in] memb_addr The offsets within the virtual address space, from 0
|
|
* (zero) to #HADDR_MAX, at which each type of data storage begins
|
|
* \param[in] relax Allows read-only access to incomplete file sets when \c TRUE
|
|
* \returns \herr_t
|
|
*
|
|
* \details H5Pset_fapl_multi() sets the file access property list \p fapl_id to
|
|
* use the multi-file driver.
|
|
*
|
|
* The multi-file driver enables different types of HDF5 data and
|
|
* metadata to be written to separate files. These files are viewed by
|
|
* the HDF5 library and the application as a single virtual HDF5 file
|
|
* with a single HDF5 file address space. The types of data that can be
|
|
* broken out into separate files include raw data, the superblock,
|
|
* B-tree data, global heap data, local heap data, and object
|
|
* headers. At the programmer's discretion, two or more types of data
|
|
* can be written to the same file while other types of data are
|
|
* written to separate files.
|
|
*
|
|
* The array \p memb_map maps memory usage types to other memory usage
|
|
* types and is the mechanism that allows the caller to specify how
|
|
* many files are created. The array contains #H5FD_MEM_NTYPES entries,
|
|
* which are either the value #H5FD_MEM_DEFAULT or a memory usage
|
|
* type. The number of unique values determines the number of files
|
|
* that are opened.
|
|
*
|
|
* The array \p memb_fapl contains a property list for each memory
|
|
* usage type that will be associated with a file.
|
|
*
|
|
* The array \p memb_name should be a name generator (a
|
|
* \Code{printf}-style format with a \Code{%s} which will be replaced
|
|
* with the name passed to H5FDopen(), usually from H5Fcreate() or
|
|
* H5Fopen()).
|
|
*
|
|
* The array \p memb_addr specifies the offsets within the virtual
|
|
* address space, from 0 (zero) to #HADDR_MAX, at which each type of
|
|
* data storage begins.
|
|
*
|
|
* If \p relax is set to 1 (TRUE), then opening an existing file for
|
|
* read-only access will not fail if some file members are
|
|
* missing. This allows a file to be accessed in a limited sense if
|
|
* just the meta data is available.
|
|
*
|
|
* Default values for each of the optional arguments are as follows:
|
|
* <table>
|
|
* <tr>
|
|
* <td>\p memb_map</td>
|
|
* <td>The default member map contains the value #H5FD_MEM_DEFAULT for each element.</td>
|
|
* </tr>
|
|
* <tr>
|
|
* <td>
|
|
* \p memb_fapl
|
|
* </td>
|
|
* <td>
|
|
* The default value is #H5P_DEFAULT for each element.
|
|
* </td>
|
|
* </tr>
|
|
* <tr>
|
|
* <td>
|
|
* \p memb_name
|
|
* </td>
|
|
* <td>
|
|
* The default string is \Code{%s-X.h5} where \c X is one of the following letters:
|
|
* - \c s for #H5FD_MEM_SUPER
|
|
* - \c b for #H5FD_MEM_BTREE
|
|
* - \c r for #H5FD_MEM_DRAW
|
|
* - \c g for #H5FD_MEM_GHEAP
|
|
* - \c l for #H5FD_MEM_LHEAP
|
|
* - \c o for #H5FD_MEM_OHDR
|
|
* </td>
|
|
* </tr>
|
|
* <tr>
|
|
* <td>
|
|
* \p memb_addr
|
|
* </td>
|
|
* <td>
|
|
* The default setting is that the address space is equally divided
|
|
* among all of the elements:
|
|
* - #H5FD_MEM_SUPER \Code{-> 0 * (HADDR_MAX/6)}
|
|
* - #H5FD_MEM_BTREE \Code{-> 1 * (HADDR_MAX/6)}
|
|
* - #H5FD_MEM_DRAW \Code{-> 2 * (HADDR_MAX/6)}
|
|
* - #H5FD_MEM_GHEAP \Code{-> 3 * (HADDR_MAX/6)}
|
|
* - #H5FD_MEM_LHEAP \Code{-> 4 * (HADDR_MAX/6)}
|
|
* - #H5FD_MEM_OHDR \Code{-> 5 * (HADDR_MAX/6)}
|
|
* </td>
|
|
* </tr>
|
|
* </table>
|
|
*
|
|
* \par Example:
|
|
* The following code sample sets up a multi-file access property list that
|
|
* partitions data into meta and raw files, each being one-half of the address:\n
|
|
* \code
|
|
* H5FD_mem_t mt, memb_map[H5FD_MEM_NTYPES];
|
|
* hid_t memb_fapl[H5FD_MEM_NTYPES];
|
|
* const char *memb[H5FD_MEM_NTYPES];
|
|
* haddr_t memb_addr[H5FD_MEM_NTYPES];
|
|
*
|
|
* // The mapping...
|
|
* for (mt=0; mt<H5FD_MEM_NTYPES; mt++) {
|
|
* memb_map[mt] = H5FD_MEM_SUPER;
|
|
* }
|
|
* memb_map[H5FD_MEM_DRAW] = H5FD_MEM_DRAW;
|
|
*
|
|
* // Member information
|
|
* memb_fapl[H5FD_MEM_SUPER] = H5P_DEFAULT;
|
|
* memb_name[H5FD_MEM_SUPER] = "%s.meta";
|
|
* memb_addr[H5FD_MEM_SUPER] = 0;
|
|
*
|
|
* memb_fapl[H5FD_MEM_DRAW] = H5P_DEFAULT;
|
|
* memb_name[H5FD_MEM_DRAW] = "%s.raw";
|
|
* memb_addr[H5FD_MEM_DRAW] = HADDR_MAX/2;
|
|
*
|
|
* hid_t fapl = H5Pcreate(H5P_FILE_ACCESS);
|
|
* H5Pset_fapl_multi(fapl, memb_map, memb_fapl,
|
|
* memb_name, memb_addr, TRUE);
|
|
* \endcode
|
|
*
|
|
* \version 1.6.3 \p memb_name parameter type changed to \Code{const char* const*}.
|
|
* \since 1.4.0
|
|
*/
|
|
H5_DLL herr_t H5Pset_fapl_multi(hid_t fapl_id, const H5FD_mem_t *memb_map, const hid_t *memb_fapl,
|
|
const char *const *memb_name, const haddr_t *memb_addr, hbool_t relax);
|
|
|
|
/**
|
|
* \ingroup FAPL
|
|
*
|
|
* \brief Returns information about the multi-file access property list
|
|
*
|
|
* \fapl_id
|
|
* \param[out] memb_map Maps memory usage types to other memory usage types
|
|
* \param[out] memb_fapl Property list for each memory usage type
|
|
* \param[out] memb_name Name generator for names of member files
|
|
* \param[out] memb_addr The offsets within the virtual address space, from 0
|
|
* (zero) to #HADDR_MAX, at which each type of data storage begins
|
|
* \param[out] relax Allows read-only access to incomplete file sets when \c TRUE
|
|
* \returns \herr_t
|
|
*
|
|
* \details H5Pget_fapl_multi() returns information about the multi-file access
|
|
* property list.
|
|
*
|
|
* \since 1.4.0
|
|
*
|
|
*/
|
|
H5_DLL herr_t H5Pget_fapl_multi(hid_t fapl_id, H5FD_mem_t *memb_map /*out*/, hid_t *memb_fapl /*out*/,
|
|
char **memb_name /*out*/, haddr_t *memb_addr /*out*/, hbool_t *relax /*out*/);
|
|
|
|
/**
|
|
* \ingroup FAPL
|
|
*
|
|
* \brief Emulates the old split file driver
|
|
*
|
|
* \fapl_id{fapl}
|
|
* \param[in] meta_ext Metadata filename extension
|
|
* \param[in] meta_plist_id File access property list identifier for the metadata file
|
|
* \param[in] raw_ext Raw data filename extension
|
|
* \param[in] raw_plist_id
|
|
* \returns \herr_t
|
|
*
|
|
* \details H5Pset_fapl_split() is a compatibility function that enables the
|
|
* multi-file driver to emulate the split driver from HDF5 Releases 1.0
|
|
* and 1.2. The split file driver stored metadata and raw data in
|
|
* separate files but provided no mechanism for separating types of
|
|
* metadata.
|
|
*
|
|
* \p fapl is a file access property list identifier.
|
|
*
|
|
* \p meta_ext is the filename extension for the metadata file. The
|
|
* extension is appended to the name passed to H5FDopen(), usually from
|
|
* H5Fcreate() or H5Fopen(), to form the name of the metadata file. If
|
|
* the string \Code{%s} is used in the extension, it works like the
|
|
* name generator as in H5Pset_fapl_multi().
|
|
*
|
|
* \p meta_plist_id is the file access property list identifier for the
|
|
* metadata file.
|
|
*
|
|
* \p raw_ext is the filename extension for the raw data file. The
|
|
* extension is appended to the name passed to H5FDopen(), usually from
|
|
* H5Fcreate() or H5Fopen(), to form the name of the raw data file. If
|
|
* the string \Code{%s} is used in the extension, it works like the
|
|
* name generator as in H5Pset_fapl_multi().
|
|
*
|
|
* \p raw_plist_id is the file access property list identifier for the
|
|
* raw data file.
|
|
*
|
|
* If a user wishes to check to see whether this driver is in use, the
|
|
* user must call H5Pget_driver() and compare the returned value to the
|
|
* string #H5FD_MULTI. A positive match will confirm that the multi
|
|
* driver is in use; HDF5 provides no mechanism to determine whether it
|
|
* was called as the special case invoked by H5Pset_fapl_split().
|
|
*
|
|
* \par Example:
|
|
* \code
|
|
* // Example 1: Both metadata and rawdata files are in the same
|
|
* // directory. Use Station1-m.h5 and Station1-r.h5 as
|
|
* // the metadata and rawdata files.
|
|
* hid_t fapl, fid;
|
|
* fapl = H5Pcreate(H5P_FILE_ACCESS);
|
|
* H5Pset_fapl_split(fapl, "-m.h5", H5P_DEFAULT, "-r.h5", H5P_DEFAULT);
|
|
* fid=H5Fcreate("Station1",H5F_ACC_TRUNC,H5P_DEFAULT,fapl);
|
|
*
|
|
* // Example 2: metadata and rawdata files are in different
|
|
* // directories. Use PointA-m.h5 and /pfs/PointA-r.h5 as
|
|
* // the metadata and rawdata files.
|
|
* hid_t fapl, fid;
|
|
* fapl = H5Pcreate(H5P_FILE_ACCESS);
|
|
* H5Pset_fapl_split(fapl, "-m.h5", H5P_DEFAULT, "/pfs/%s-r.h5", H5P_DEFAULT);
|
|
* fid=H5Fcreate("PointA",H5F_ACC_TRUNC,H5P_DEFAULT,fapl);
|
|
* \endcode
|
|
*
|
|
* \since 1.4.0
|
|
*
|
|
*/
|
|
H5_DLL herr_t H5Pset_fapl_split(hid_t fapl, const char *meta_ext, hid_t meta_plist_id, const char *raw_ext,
|
|
hid_t raw_plist_id);
|
|
#ifdef __cplusplus
|
|
}
|
|
#endif
|
|
|
|
#endif
|