mirror/postgresql
2
0
Fork 0
mirror of https://git.postgresql.org/git/postgresql.git synced 2024-12-21 08:29:39 +08:00
Code Issues Packages Projects Releases Wiki Activity
27b77ecf9f
postgresql/contrib/bloom/bloom.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

217 lines
7.0 KiB
C
Raw Normal View History

Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
/*-------------------------------------------------------------------------
*
* bloom.h
* Header for bloom index.
*
Update copyright for 2022 Backpatch-through: 10
2022-01-08 08:04:57 +08:00
* Copyright (c) 2016-2022, PostgreSQL Global Development Group
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
*
* IDENTIFICATION
* contrib/bloom/bloom.h
*
*-------------------------------------------------------------------------
*/
#ifndef _BLOOM_H_
#define _BLOOM_H_
Revert "Rename files and headers related to index AM" This follows multiple complains from Peter Geoghegan, Andres Freund and Alvaro Herrera that this issue ought to be dug more before actually happening, if it happens. Discussion: https://postgr.es/m/20191226144606.GA5659@alvherre.pgsql
2019-12-27 07:09:00 +08:00
#include "access/amapi.h"
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
#include "access/generic_xlog.h"
#include "access/itup.h"
#include "access/xlog.h"
#include "fmgr.h"
Make the order of the header file includes consistent. Similar to commits 14aec03502, 7e735035f2 and dddf4cdc33, this commit makes the order of header file inclusion consistent in more places. Author: Vignesh C Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CALDaNm2Sznv8RR6Ex-iJO6xAdsxgWhCoETkaYX=+9DW3q0QCfA@mail.gmail.com
2019-11-25 10:38:57 +08:00
#include "nodes/pathnodes.h"
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
/* Support procedures numbers */
#define BLOOM_HASH_PROC 1
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
#define BLOOM_OPTIONS_PROC 2
#define BLOOM_NPROC 2
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
/* Scan strategies */
#define BLOOM_EQUAL_STRATEGY 1
#define BLOOM_NSTRATEGIES 1
/* Opaque for bloom pages */
typedef struct BloomPageOpaqueData
{
Add page id to bloom index Added to ensure that bloom index pages can be distinguished from other pages by pg_filedump. Because there wasn't any public/production versions before, it doesn't pay attention to any compatibility issues. Per notice from Tom Lane
2016-04-12 23:03:01 +08:00
OffsetNumber maxoff; /* number of index tuples on page */
uint16 flags; /* see bit definitions below */
uint16 unused; /* placeholder to force maxaligning of size of
* BloomPageOpaqueData and to place
* bloom_page_id exactly at the end of page */
uint16 bloom_page_id; /* for identification of BLOOM indexes */
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
} BloomPageOpaqueData;
typedef BloomPageOpaqueData *BloomPageOpaque;
/* Bloom page flags */
#define BLOOM_META (1<<0)
#define BLOOM_DELETED (2<<0)
Add page id to bloom index Added to ensure that bloom index pages can be distinguished from other pages by pg_filedump. Because there wasn't any public/production versions before, it doesn't pay attention to any compatibility issues. Per notice from Tom Lane
2016-04-12 23:03:01 +08:00
/*
* The page ID is for the convenience of pg_filedump and similar utilities,
* which otherwise would have a hard time telling pages of different index
* types apart. It should be the last 2 bytes on the page. This is more or
* less "free" due to alignment considerations.
*
* See comments above GinPageOpaqueData.
*/
#define BLOOM_PAGE_ID 0xFF83
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
/* Macros for accessing bloom page structures */
#define BloomPageGetOpaque(page) ((BloomPageOpaque) PageGetSpecialPointer(page))
#define BloomPageGetMaxOffset(page) (BloomPageGetOpaque(page)->maxoff)
Fixes in bloom contrib module missed during review - macroses llike (var & FLAG) are changed to ((var & FLAG) != 0) - do not copy uninitialized part of notFullPage array to page
2016-04-02 01:09:13 +08:00
#define BloomPageIsMeta(page) \
((BloomPageGetOpaque(page)->flags & BLOOM_META) != 0)
#define BloomPageIsDeleted(page) \
((BloomPageGetOpaque(page)->flags & BLOOM_DELETED) != 0)
#define BloomPageSetDeleted(page) \
(BloomPageGetOpaque(page)->flags |= BLOOM_DELETED)
#define BloomPageSetNonDeleted(page) \
(BloomPageGetOpaque(page)->flags &= ~BLOOM_DELETED)
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
#define BloomPageGetData(page) ((BloomTuple *)PageGetContents(page))
#define BloomPageGetTuple(state, page, offset) \
((BloomTuple *)(PageGetContents(page) \
+ (state)->sizeOfBloomTuple * ((offset) - 1)))
#define BloomPageGetNextTuple(state, tuple) \
((BloomTuple *)((Pointer)(tuple) + (state)->sizeOfBloomTuple))
/* Preserved page numbers */
#define BLOOM_METAPAGE_BLKNO (0)
#define BLOOM_HEAD_BLKNO (1) /* first data page */
Fixes in bloom contrib module Looking at result of buildfarm member jaguarundi it seems to me that BloomOptions isn't inited sometime, but I don't see yet how it's possible. Nevertheless, check of signature length's is missed, so, add a limit of it. Also add missed GenericXLogAbort() in case of already deleted page in vacuum + minor code refactoring.
2016-04-02 18:47:04 +08:00
/*
Measure Bloom index signature-length reloption in bits, not words. Per discussion, this is a more understandable and future-proof way of exposing the setting to users. On-disk, we can still store it in words, so as to not break on-disk compatibility with beta1. Along the way, clean up the code associated with Bloom reloptions. Provide explicit macros for default and maximum lengths rather than having magic numbers buried in multiple places in the code. Drop the adjustBloomOptions() code altogether: it was useless in view of the fact that reloptions.c already performed default-substitution and range checking for the options. Rename a couple of macros and types for more clarity. Discussion: <23767.1464926580@sss.pgh.pa.us>
2016-06-03 22:52:36 +08:00
* We store Bloom signatures as arrays of uint16 words.
Fixes in bloom contrib module Looking at result of buildfarm member jaguarundi it seems to me that BloomOptions isn't inited sometime, but I don't see yet how it's possible. Nevertheless, check of signature length's is missed, so, add a limit of it. Also add missed GenericXLogAbort() in case of already deleted page in vacuum + minor code refactoring.
2016-04-02 18:47:04 +08:00
*/
Measure Bloom index signature-length reloption in bits, not words. Per discussion, this is a more understandable and future-proof way of exposing the setting to users. On-disk, we can still store it in words, so as to not break on-disk compatibility with beta1. Along the way, clean up the code associated with Bloom reloptions. Provide explicit macros for default and maximum lengths rather than having magic numbers buried in multiple places in the code. Drop the adjustBloomOptions() code altogether: it was useless in view of the fact that reloptions.c already performed default-substitution and range checking for the options. Rename a couple of macros and types for more clarity. Discussion: <23767.1464926580@sss.pgh.pa.us>
2016-06-03 22:52:36 +08:00
typedef uint16 BloomSignatureWord;
#define SIGNWORDBITS ((int) (BITS_PER_BYTE * sizeof(BloomSignatureWord)))
/*
* Default and maximum Bloom signature length in bits.
*/
#define DEFAULT_BLOOM_LENGTH (5 * SIGNWORDBITS)
#define MAX_BLOOM_LENGTH (256 * SIGNWORDBITS)
/*
* Default and maximum signature bits generated per index key.
*/
#define DEFAULT_BLOOM_BITS 2
#define MAX_BLOOM_BITS (MAX_BLOOM_LENGTH - 1)
Fixes in bloom contrib module Looking at result of buildfarm member jaguarundi it seems to me that BloomOptions isn't inited sometime, but I don't see yet how it's possible. Nevertheless, check of signature length's is missed, so, add a limit of it. Also add missed GenericXLogAbort() in case of already deleted page in vacuum + minor code refactoring.
2016-04-02 18:47:04 +08:00
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
/* Bloom index options */
typedef struct BloomOptions
{
int32 vl_len_; /* varlena header (do not touch directly!) */
Measure Bloom index signature-length reloption in bits, not words. Per discussion, this is a more understandable and future-proof way of exposing the setting to users. On-disk, we can still store it in words, so as to not break on-disk compatibility with beta1. Along the way, clean up the code associated with Bloom reloptions. Provide explicit macros for default and maximum lengths rather than having magic numbers buried in multiple places in the code. Drop the adjustBloomOptions() code altogether: it was useless in view of the fact that reloptions.c already performed default-substitution and range checking for the options. Rename a couple of macros and types for more clarity. Discussion: <23767.1464926580@sss.pgh.pa.us>
2016-06-03 22:52:36 +08:00
int bloomLength; /* length of signature in words (not bits!) */
int bitSize[INDEX_MAX_KEYS]; /* # of bits generated for each
* index key */
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
} BloomOptions;
/*
* FreeBlockNumberArray - array of block numbers sized so that metadata fill
* all space in metapage.
*/
typedef BlockNumber FreeBlockNumberArray[
MAXALIGN_DOWN(
BLCKSZ - SizeOfPageHeaderData - MAXALIGN(sizeof(BloomPageOpaqueData))
- MAXALIGN(sizeof(uint16) * 2 + sizeof(uint32) + sizeof(BloomOptions))
) / sizeof(BlockNumber)
];
/* Metadata of bloom index */
typedef struct BloomMetaPageData
{
uint32 magickNumber;
uint16 nStart;
uint16 nEnd;
BloomOptions opts;
FreeBlockNumberArray notFullPage;
} BloomMetaPageData;
/* Magic number to distinguish bloom pages among anothers */
#define BLOOM_MAGICK_NUMBER (0xDBAC0DED)
/* Number of blocks numbers fit in BloomMetaPageData */
#define BloomMetaBlockN (sizeof(FreeBlockNumberArray) / sizeof(BlockNumber))
#define BloomPageGetMeta(page) ((BloomMetaPageData *) PageGetContents(page))
typedef struct BloomState
{
FmgrInfo hashFn[INDEX_MAX_KEYS];
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 19:09:32 +08:00
Oid collations[INDEX_MAX_KEYS];
Fix contrib/bloom to not fail under CLOBBER_CACHE_ALWAYS. The code was supposing that rd_amcache wouldn't disappear from under it during a scan; which is wrong. Copy the data out of the relcache rather than trying to reference it there.
2016-04-04 03:16:07 +08:00
BloomOptions opts; /* copy of options on index's metapage */
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
int32 nColumns;
/*
Fix contrib/bloom to not fail under CLOBBER_CACHE_ALWAYS. The code was supposing that rd_amcache wouldn't disappear from under it during a scan; which is wrong. Copy the data out of the relcache rather than trying to reference it there.
2016-04-04 03:16:07 +08:00
* sizeOfBloomTuple is index-specific, and it depends on reloptions, so
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
* precompute it
*/
Clean up some stuff in new contrib/bloom module. Coverity complained about implicit sign-extension in the BloomPageGetFreeSpace macro, probably because sizeOfBloomTuple isn't wide enough for size calculations. No overflow is really possible as long as maxoff and sizeOfBloomTuple are small enough to represent a realistic situation, but it seems like a good idea to declare sizeOfBloomTuple as Size not int32. Add missing check on BloomPageAddItem() result, again from Coverity. Avoid core dump due to not allocating so->sign array when scan->numberOfKeys is zero. Also thanks to Coverity. Use FLEXIBLE_ARRAY_MEMBER rather than declaring an array as size 1 when it isn't necessarily. Very minor beautification of related code. Unfortunately, none of the Coverity-detected mistakes look like they could account for the remaining buildfarm unhappiness with this module. It's barely possible that the FLEXIBLE_ARRAY_MEMBER mistake does account for that, if it's enabling bogus compiler optimizations; but I'm not terribly optimistic. We probably still have bugs to find here.
2016-04-04 02:17:20 +08:00
Size sizeOfBloomTuple;
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
} BloomState;
#define BloomPageGetFreeSpace(state, page) \
(BLCKSZ - MAXALIGN(SizeOfPageHeaderData) \
- BloomPageGetMaxOffset(page) * (state)->sizeOfBloomTuple \
- MAXALIGN(sizeof(BloomPageOpaqueData)))
/*
* Tuples are very different from all other relations
*/
typedef struct BloomTuple
{
ItemPointerData heapPtr;
Measure Bloom index signature-length reloption in bits, not words. Per discussion, this is a more understandable and future-proof way of exposing the setting to users. On-disk, we can still store it in words, so as to not break on-disk compatibility with beta1. Along the way, clean up the code associated with Bloom reloptions. Provide explicit macros for default and maximum lengths rather than having magic numbers buried in multiple places in the code. Drop the adjustBloomOptions() code altogether: it was useless in view of the fact that reloptions.c already performed default-substitution and range checking for the options. Rename a couple of macros and types for more clarity. Discussion: <23767.1464926580@sss.pgh.pa.us>
2016-06-03 22:52:36 +08:00
BloomSignatureWord sign[FLEXIBLE_ARRAY_MEMBER];
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
} BloomTuple;
#define BLOOMTUPLEHDRSZ offsetof(BloomTuple, sign)
/* Opaque data structure for bloom index scan */
typedef struct BloomScanOpaqueData
{
Measure Bloom index signature-length reloption in bits, not words. Per discussion, this is a more understandable and future-proof way of exposing the setting to users. On-disk, we can still store it in words, so as to not break on-disk compatibility with beta1. Along the way, clean up the code associated with Bloom reloptions. Provide explicit macros for default and maximum lengths rather than having magic numbers buried in multiple places in the code. Drop the adjustBloomOptions() code altogether: it was useless in view of the fact that reloptions.c already performed default-substitution and range checking for the options. Rename a couple of macros and types for more clarity. Discussion: <23767.1464926580@sss.pgh.pa.us>
2016-06-03 22:52:36 +08:00
BloomSignatureWord *sign; /* Scan signature */
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
BloomState state;
} BloomScanOpaqueData;
typedef BloomScanOpaqueData *BloomScanOpaque;
/* blutils.c */
extern void _PG_init(void);
extern void initBloomState(BloomState *state, Relation index);
Fix contrib/bloom to work for unlogged indexes. blbuildempty did not do even approximately the right thing: it tried to add a metapage to the relation's regular data fork, which already has one at that point. It should look like the ambuildempty methods for all the standard index types, ie, initialize a metapage image in some transient storage and then write it directly to the init fork. To support that, refactor BloomInitMetapage into two functions. In passing, fix BloomInitMetapage so it doesn't leave the rd_options field of the index's relcache entry pointing at transient storage. I'm not sure this had any visible consequence, since nothing much else is likely to look at a bloom index's rd_options, but it's certainly poor practice. Per bug #14155 from Zhou Digoal. Report: <20160524144146.22598.42558@wrigleys.postgresql.org>
2016-05-25 09:04:23 +08:00
extern void BloomFillMetapage(Relation index, Page metaPage);
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
extern void BloomInitMetapage(Relation index);
extern void BloomInitPage(Page page, uint16 flags);
extern Buffer BloomNewBuffer(Relation index);
Measure Bloom index signature-length reloption in bits, not words. Per discussion, this is a more understandable and future-proof way of exposing the setting to users. On-disk, we can still store it in words, so as to not break on-disk compatibility with beta1. Along the way, clean up the code associated with Bloom reloptions. Provide explicit macros for default and maximum lengths rather than having magic numbers buried in multiple places in the code. Drop the adjustBloomOptions() code altogether: it was useless in view of the fact that reloptions.c already performed default-substitution and range checking for the options. Rename a couple of macros and types for more clarity. Discussion: <23767.1464926580@sss.pgh.pa.us>
2016-06-03 22:52:36 +08:00
extern void signValue(BloomState *state, BloomSignatureWord *sign, Datum value, int attno);
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
extern BloomTuple *BloomFormTuple(BloomState *state, ItemPointer iptr, Datum *values, bool *isnull);
extern bool BloomPageAddItem(BloomState *state, Page page, BloomTuple *tuple);
/* blvalidate.c */
extern bool blvalidate(Oid opclassoid);
/* index access method interface functions */
extern bool blinsert(Relation index, Datum *values, bool *isnull,
ItemPointer ht_ctid, Relation heapRel,
Allow index AMs to cache data across aminsert calls within a SQL command. It's always been possible for index AMs to cache data across successive amgettuple calls within a single SQL command: the IndexScanDesc.opaque field is meant for precisely that. However, no comparable facility exists for amortizing setup work across successive aminsert calls. This patch adds such a feature and teaches GIN, GIST, and BRIN to use it to amortize catalog lookups they'd previously been doing on every call. (The other standard index AMs keep everything they need in the relcache, so there's little to improve there.) For GIN, the overall improvement in a statement that inserts many rows can be as much as 10%, though it seems a bit less for the other two. In addition, this makes a really significant difference in runtime for CLOBBER_CACHE_ALWAYS tests, since in those builds the repeated catalog lookups are vastly more expensive. The reason this has been hard up to now is that the aminsert function is not passed any useful place to cache per-statement data. What I chose to do is to add suitable fields to struct IndexInfo and pass that to aminsert. That's not widening the index AM API very much because IndexInfo is already within the ken of ambuild; in fact, by passing the same info to aminsert as to ambuild, this is really removing an inconsistency in the AM API. Discussion: https://postgr.es/m/27568.1486508680@sss.pgh.pa.us
2017-02-10 00:52:12 +08:00
IndexUniqueCheck checkUnique,
Pass down "logically unchanged index" hint. Add an executor aminsert() hint mechanism that informs index AMs that the incoming index tuple (the tuple that accompanies the hint) is not being inserted by execution of an SQL statement that logically modifies any of the index's key columns. The hint is received by indexes when an UPDATE takes place that does not apply an optimization like heapam's HOT (though only for indexes where all key columns are logically unchanged). Any index tuple that receives the hint on insert is expected to be a duplicate of at least one existing older version that is needed for the same logical row. Related versions will typically be stored on the same index page, at least within index AMs that apply the hint. Recognizing the difference between MVCC version churn duplicates and true logical row duplicates at the index AM level can help with cleanup of garbage index tuples. Cleanup can intelligently target tuples that are likely to be garbage, without wasting too many cycles on less promising tuples/pages (index pages with little or no version churn). This is infrastructure for an upcoming commit that will teach nbtree to perform bottom-up index deletion. No index AM actually applies the hint just yet. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Victor Yegorov <vyegorov@gmail.com> Discussion: https://postgr.es/m/CAH2-Wz=CEKFa74EScx_hFVshCOn6AA5T-ajFASTdzipdkLTNQQ@mail.gmail.com
2021-01-14 00:11:00 +08:00
bool indexUnchanged,
Allow index AMs to cache data across aminsert calls within a SQL command. It's always been possible for index AMs to cache data across successive amgettuple calls within a single SQL command: the IndexScanDesc.opaque field is meant for precisely that. However, no comparable facility exists for amortizing setup work across successive aminsert calls. This patch adds such a feature and teaches GIN, GIST, and BRIN to use it to amortize catalog lookups they'd previously been doing on every call. (The other standard index AMs keep everything they need in the relcache, so there's little to improve there.) For GIN, the overall improvement in a statement that inserts many rows can be as much as 10%, though it seems a bit less for the other two. In addition, this makes a really significant difference in runtime for CLOBBER_CACHE_ALWAYS tests, since in those builds the repeated catalog lookups are vastly more expensive. The reason this has been hard up to now is that the aminsert function is not passed any useful place to cache per-statement data. What I chose to do is to add suitable fields to struct IndexInfo and pass that to aminsert. That's not widening the index AM API very much because IndexInfo is already within the ken of ambuild; in fact, by passing the same info to aminsert as to ambuild, this is really removing an inconsistency in the AM API. Discussion: https://postgr.es/m/27568.1486508680@sss.pgh.pa.us
2017-02-10 00:52:12 +08:00
struct IndexInfo *indexInfo);
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
extern IndexScanDesc blbeginscan(Relation r, int nkeys, int norderbys);
extern int64 blgetbitmap(IndexScanDesc scan, TIDBitmap *tbm);
extern void blrescan(IndexScanDesc scan, ScanKey scankey, int nscankeys,
ScanKey orderbys, int norderbys);
extern void blendscan(IndexScanDesc scan);
extern IndexBuildResult *blbuild(Relation heap, Relation index,
struct IndexInfo *indexInfo);
extern void blbuildempty(Relation index);
extern IndexBulkDeleteResult *blbulkdelete(IndexVacuumInfo *info,
IndexBulkDeleteResult *stats, IndexBulkDeleteCallback callback,
void *callback_state);
extern IndexBulkDeleteResult *blvacuumcleanup(IndexVacuumInfo *info,
IndexBulkDeleteResult *stats);
extern bytea *bloptions(Datum reloptions, bool validate);
extern void blcostestimate(PlannerInfo *root, IndexPath *path,
double loop_count, Cost *indexStartupCost,
Cost *indexTotalCost, Selectivity *indexSelectivity,
Add optimizer and executor support for parallel index scans. In combination with 569174f1be92be93f5366212cc46960d28a5c5cd, which taught the btree AM how to perform parallel index scans, this allows parallel index scan plans on btree indexes. This infrastructure should be general enough to support parallel index scans for other index AMs as well, if someone updates them to support parallel scans. Amit Kapila, reviewed and tested by Anastasia Lubennikova, Tushar Ahuja, and Haribabu Kommi, and me.
2017-02-16 02:53:24 +08:00
double *indexCorrelation, double *indexPages);
Bloom index contrib module Module provides new access method. It is actually a simple Bloom filter implemented as pgsql's index. It could give some benefits on search with large number of columns. Module is a single way to test generic WAL interface committed earlier. Author: Teodor Sigaev, Alexander Korotkov Reviewers: Aleksander Alekseev, Michael Paquier, Jim Nasby
2016-04-01 21:42:24 +08:00
#endif
Reference in New Issue Copy Permalink