hdf5/src/H5FDmulti.h
David Young a0445d806c
Simplify function enter macros for performance benefits (#1024)
* Take a stab at using constructors to initialize instead of
function-entry macros.  This is a work in progress.  It's good enough to
run `many_dsets`.

* Committing clang-format changes

* Add the `many_dsets` benchmark and some scripts I used on jelly for
setting up the build/test environment and for recording/flame-graphing
profiles.

* Committing clang-format changes

* Change my Makefile and environment script to work both on jelly
and on mayll (and probably on Summit).

* Disable clang-format "fix."

* Replace the `if (!H5_TERM_GLOBAL)` test in each FUNC_ENTER_ macro with
`if (true)`.

* Fix bad grammar in a comment.

* Instead of labeling the H5*__init_package routines constructors, fold
each into an initialization routine, H5*_init(), and call each of
the H5*_init() routines.  Call most of the H5*_init() routines from
H5_init_library() in an explicit order that I found out earlier by
instrumenting each __init_package routine and running the library
tests.  Roll H5FD*__init_package routines into H5FD*_init() routines.
This change ends just-in-time initialization of package dependencies by
package initializers.

Don't track in per-package variables (H5_PKG_INIT_VAR) whether each
package has been initialized.  Instead, track in a single library
variable whether the whole library is initialized or not.

Drive the initialization of packages by H5_init_library() with a table
of initializer routines.  Also drive the termination of packages by
H5_term_library() with a table.

Perform initialization as needed from FUNC_ENTER_API_INIT(err).  This
basically restores the old behavior of that macro.

Delete a bunch of #definitions in H5private.h that have fallen out of
use with these changes.

* Committing clang-format changes

* Undo the bad auto-formatting that appears to have occurred in spite
of my disabling it.  Bracket some code in /* clang-format off */ /*
clang-format on */ to prevent a recurrence.

* Remove a diagnostic abort().

* Fix a logic error: print a comma between every package terminator run,
and don't print an initial comma.

* Complete the changes I started in H5_term_library() that undo the bad
auto-formatting.

Stop tracking whether package "tops" were initialized in per-package
variables H5*_top_package_initialize_s.  H5_term_library() takes care of
that for them.

Remove H5R_top_term_package() and H5R_term_package(), they don't do
anything.

* Committing clang-format changes

* NFCI.  Simplify macro text: replace `if (true) {` with `{`.

* Fix formatting and suppress clang-format on a longer range.

* Quiet some unused label, unused variable complaints that cropped up
after I simplified the FUNC_ENTER_ macros for the sake of performance.

* Committing clang-format changes

* Delete some programs and scripts that don't belong in the pull request.

* Use the right function-entry macro.

* Use a sensible format and disable auto-formatting.

* Stop calling do-nothing initializer H5FS_init().  Delete it.

* Document what changes to make if the default VFD changes.

* While I am here, change an `await_prior` flag on the terminator table
to `true` to match the previous, non-table-driven code that was here.
Found the oversight making the following changes:

NFCI: insert an empty line and copy over slightly-edited comments from
the previous version, where those comments still correctly explained how
library termination operated.

* NFCI: lower a staircase.

* Replace every occurrence of FUNC_ENTER_NOAPI_INIT(...) with H5_PUSH_FUNC
since that is all that that macro does any more.

Quiet a bunch of new warnings by changing FUNC_ENTER_NOAPI(...) to
FUNC_ENTER_NOAPI_NOERR and removing disused `done:` labels.

* NFCI: add curly braces around a multiline statement.

* Quiet a signed/unsigned comparison warning.

* Add some documentation about library initialization and shutdown.

* Make sure that the library is initialized, or else that initialization
is already underway, before performing any VFD's initialization.

* Committing clang-format changes

* Committing clang-format changes

* Reduce differences from `develop` branch.

* Always initialize `tot_init`.

* Committing clang-format changes

* Fix typo: H5SL_init initializes skip lists, not VOL.

* Remove H5_TERM_GLOBAL test in H5T_init.  H5T_init was unusual in that
it tested H5_TERM_GLOBAL and exited early if it was set.  No other
module initializers did that, and I cannot find any reason that should
be necessary.  Tests still pass when I remove it, so away it goes.

* Use HD prefix.

* Add function header comments.

* Drop the intermediate variable, it's only used once.

* Extract subroutine `H5FDperform_init(hid_t (*init)(void))` that
initializes the library, if necessary, before calling its VFD-initializer
argument.  Use H5FDperform_init in the definition of the symbols
H5FD_<vfd> (e.g., H5FD_SEC2), which may be evaluated before the library
is initialized, like so:

```
```

I implement H5FDperform_init in its own source file, H5FDperform.c,
and exclude that file from trace processing because the `bin/trace`
cannot deal with the function-pointer type.

* Straggler from last: add new source file src/H5FDperform.c.

* Committing clang-format changes

* Add a missing file to the MANIFEST.

* Switch to FUNC_ENTER_API_NOINIT in H5FDperform_init() and hbool_t in
H5_term_library().

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2021-11-08 15:08:07 -06:00

257 lines
10 KiB
C

/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* Copyright by the Board of Trustees of the University of Illinois. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the COPYING file, which can be found at the root of the source code *
* distribution tree, or in https://www.hdfgroup.org/licenses. *
* If you do not have access to either file, you may request a copy from *
* help@hdfgroup.org. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
/*
* Programmer: Robb Matzke
* Monday, August 2, 1999
*
* Purpose: The public header file for the "multi" driver.
*/
#ifndef H5FDmulti_H
#define H5FDmulti_H
#define H5FD_MULTI (H5FDperform_init(H5FD_multi_init))
#ifdef __cplusplus
extern "C" {
#endif
H5_DLL hid_t H5FD_multi_init(void);
/**
* \ingroup FAPL
*
* \brief Sets up use of the multi-file driver
*
* \fapl_id
* \param[in] memb_map Maps memory usage types to other memory usage types
* \param[in] memb_fapl Property list for each memory usage type
* \param[in] memb_name Name generator for names of member files
* \param[in] memb_addr The offsets within the virtual address space, from 0
* (zero) to #HADDR_MAX, at which each type of data storage begins
* \param[in] relax Allows read-only access to incomplete file sets when \c TRUE
* \returns \herr_t
*
* \details H5Pset_fapl_multi() sets the file access property list \p fapl_id to
* use the multi-file driver.
*
* The multi-file driver enables different types of HDF5 data and
* metadata to be written to separate files. These files are viewed by
* the HDF5 library and the application as a single virtual HDF5 file
* with a single HDF5 file address space. The types of data that can be
* broken out into separate files include raw data, the superblock,
* B-tree data, global heap data, local heap data, and object
* headers. At the programmer's discretion, two or more types of data
* can be written to the same file while other types of data are
* written to separate files.
*
* The array \p memb_map maps memory usage types to other memory usage
* types and is the mechanism that allows the caller to specify how
* many files are created. The array contains #H5FD_MEM_NTYPES entries,
* which are either the value #H5FD_MEM_DEFAULT or a memory usage
* type. The number of unique values determines the number of files
* that are opened.
*
* The array \p memb_fapl contains a property list for each memory
* usage type that will be associated with a file.
*
* The array \p memb_name should be a name generator (a
* \Code{printf}-style format with a \Code{%s} which will be replaced
* with the name passed to H5FDopen(), usually from H5Fcreate() or
* H5Fopen()).
*
* The array \p memb_addr specifies the offsets within the virtual
* address space, from 0 (zero) to #HADDR_MAX, at which each type of
* data storage begins.
*
* If \p relax is set to 1 (TRUE), then opening an existing file for
* read-only access will not fail if some file members are
* missing. This allows a file to be accessed in a limited sense if
* just the meta data is available.
*
* Default values for each of the optional arguments are as follows:
* <table>
* <tr>
* <td>\p memb_map</td>
* <td>The default member map contains the value #H5FD_MEM_DEFAULT for each element.</td>
* </tr>
* <tr>
* <td>
* \p memb_fapl
* </td>
* <td>
* The default value is #H5P_DEFAULT for each element.
* </td>
* </tr>
* <tr>
* <td>
* \p memb_name
* </td>
* <td>
* The default string is \Code{%s-X.h5} where \c X is one of the following letters:
* - \c s for #H5FD_MEM_SUPER
* - \c b for #H5FD_MEM_BTREE
* - \c r for #H5FD_MEM_DRAW
* - \c g for #H5FD_MEM_GHEAP
* - \c l for #H5FD_MEM_LHEAP
* - \c o for #H5FD_MEM_OHDR
* </td>
* </tr>
* <tr>
* <td>
* \p memb_addr
* </td>
* <td>
* The default setting is that the address space is equally divided
* among all of the elements:
* - #H5FD_MEM_SUPER \Code{-> 0 * (HADDR_MAX/6)}
* - #H5FD_MEM_BTREE \Code{-> 1 * (HADDR_MAX/6)}
* - #H5FD_MEM_DRAW \Code{-> 2 * (HADDR_MAX/6)}
* - #H5FD_MEM_GHEAP \Code{-> 3 * (HADDR_MAX/6)}
* - #H5FD_MEM_LHEAP \Code{-> 4 * (HADDR_MAX/6)}
* - #H5FD_MEM_OHDR \Code{-> 5 * (HADDR_MAX/6)}
* </td>
* </tr>
* </table>
*
* \par Example:
* The following code sample sets up a multi-file access property list that
* partitions data into meta and raw files, each being one-half of the address:\n
* \code
* H5FD_mem_t mt, memb_map[H5FD_MEM_NTYPES];
* hid_t memb_fapl[H5FD_MEM_NTYPES];
* const char *memb[H5FD_MEM_NTYPES];
* haddr_t memb_addr[H5FD_MEM_NTYPES];
*
* // The mapping...
* for (mt=0; mt<H5FD_MEM_NTYPES; mt++) {
* memb_map[mt] = H5FD_MEM_SUPER;
* }
* memb_map[H5FD_MEM_DRAW] = H5FD_MEM_DRAW;
*
* // Member information
* memb_fapl[H5FD_MEM_SUPER] = H5P_DEFAULT;
* memb_name[H5FD_MEM_SUPER] = "%s.meta";
* memb_addr[H5FD_MEM_SUPER] = 0;
*
* memb_fapl[H5FD_MEM_DRAW] = H5P_DEFAULT;
* memb_name[H5FD_MEM_DRAW] = "%s.raw";
* memb_addr[H5FD_MEM_DRAW] = HADDR_MAX/2;
*
* hid_t fapl = H5Pcreate(H5P_FILE_ACCESS);
* H5Pset_fapl_multi(fapl, memb_map, memb_fapl,
* memb_name, memb_addr, TRUE);
* \endcode
*
* \version 1.6.3 \p memb_name parameter type changed to \Code{const char* const*}.
* \since 1.4.0
*/
H5_DLL herr_t H5Pset_fapl_multi(hid_t fapl_id, const H5FD_mem_t *memb_map, const hid_t *memb_fapl,
const char *const *memb_name, const haddr_t *memb_addr, hbool_t relax);
/**
* \ingroup FAPL
*
* \brief Returns information about the multi-file access property list
*
* \fapl_id
* \param[out] memb_map Maps memory usage types to other memory usage types
* \param[out] memb_fapl Property list for each memory usage type
* \param[out] memb_name Name generator for names of member files
* \param[out] memb_addr The offsets within the virtual address space, from 0
* (zero) to #HADDR_MAX, at which each type of data storage begins
* \param[out] relax Allows read-only access to incomplete file sets when \c TRUE
* \returns \herr_t
*
* \details H5Pget_fapl_multi() returns information about the multi-file access
* property list.
*
* \since 1.4.0
*
*/
H5_DLL herr_t H5Pget_fapl_multi(hid_t fapl_id, H5FD_mem_t *memb_map /*out*/, hid_t *memb_fapl /*out*/,
char **memb_name /*out*/, haddr_t *memb_addr /*out*/, hbool_t *relax /*out*/);
/**
* \ingroup FAPL
*
* \brief Emulates the old split file driver
*
* \fapl_id{fapl}
* \param[in] meta_ext Metadata filename extension
* \param[in] meta_plist_id File access property list identifier for the metadata file
* \param[in] raw_ext Raw data filename extension
* \param[in] raw_plist_id
* \returns \herr_t
*
* \details H5Pset_fapl_split() is a compatibility function that enables the
* multi-file driver to emulate the split driver from HDF5 Releases 1.0
* and 1.2. The split file driver stored metadata and raw data in
* separate files but provided no mechanism for separating types of
* metadata.
*
* \p fapl is a file access property list identifier.
*
* \p meta_ext is the filename extension for the metadata file. The
* extension is appended to the name passed to H5FDopen(), usually from
* H5Fcreate() or H5Fopen(), to form the name of the metadata file. If
* the string \Code{%s} is used in the extension, it works like the
* name generator as in H5Pset_fapl_multi().
*
* \p meta_plist_id is the file access property list identifier for the
* metadata file.
*
* \p raw_ext is the filename extension for the raw data file. The
* extension is appended to the name passed to H5FDopen(), usually from
* H5Fcreate() or H5Fopen(), to form the name of the raw data file. If
* the string \Code{%s} is used in the extension, it works like the
* name generator as in H5Pset_fapl_multi().
*
* \p raw_plist_id is the file access property list identifier for the
* raw data file.
*
* If a user wishes to check to see whether this driver is in use, the
* user must call H5Pget_driver() and compare the returned value to the
* string #H5FD_MULTI. A positive match will confirm that the multi
* driver is in use; HDF5 provides no mechanism to determine whether it
* was called as the special case invoked by H5Pset_fapl_split().
*
* \par Example:
* \code
* // Example 1: Both metadata and rawdata files are in the same
* // directory. Use Station1-m.h5 and Station1-r.h5 as
* // the metadata and rawdata files.
* hid_t fapl, fid;
* fapl = H5Pcreate(H5P_FILE_ACCESS);
* H5Pset_fapl_split(fapl, "-m.h5", H5P_DEFAULT, "-r.h5", H5P_DEFAULT);
* fid=H5Fcreate("Station1",H5F_ACC_TRUNC,H5P_DEFAULT,fapl);
*
* // Example 2: metadata and rawdata files are in different
* // directories. Use PointA-m.h5 and /pfs/PointA-r.h5 as
* // the metadata and rawdata files.
* hid_t fapl, fid;
* fapl = H5Pcreate(H5P_FILE_ACCESS);
* H5Pset_fapl_split(fapl, "-m.h5", H5P_DEFAULT, "/pfs/%s-r.h5", H5P_DEFAULT);
* fid=H5Fcreate("PointA",H5F_ACC_TRUNC,H5P_DEFAULT,fapl);
* \endcode
*
* \since 1.4.0
*
*/
H5_DLL herr_t H5Pset_fapl_split(hid_t fapl, const char *meta_ext, hid_t meta_plist_id, const char *raw_ext,
hid_t raw_plist_id);
#ifdef __cplusplus
}
#endif
#endif