mirror/postgresql
2
0
Fork 0
mirror of https://git.postgresql.org/git/postgresql.git synced 2025-01-12 18:34:36 +08:00
Code Issues Packages Projects Releases Wiki Activity
db9f287711
postgresql/contrib/pageinspect/btreefuncs.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

776 lines
22 KiB
C
Raw Normal View History

Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
/*
Remove cvs keywords from all files.
2010-09-21 04:08:53 +08:00
* contrib/pageinspect/btreefuncs.c
Add $PostgreSQL$ markers to a lot of files that were missing them. This particular batch was just for *.c and *.h file. The changes were made with the following 2 commands: find . \( \( -name 'libstemmer' -o -name 'expected' -o -name 'ppport.h' \) -prune \) -o \( -name '*.[ch]' \) \( -exec grep -q '\$PostgreSQL' {} \; -o -print \) | while read file ; do head -n 1 < $file | grep -q '^/\*' && echo $file; done | xargs -l sed -i -e '1s/^\// /' -e '1i/*\n * $PostgreSQL:$ \n *' find . \( \( -name 'libstemmer' -o -name 'expected' -o -name 'ppport.h' \) -prune \) -o \( -name '*.[ch]' \) \( -exec grep -q '\$PostgreSQL' {} \; -o -print \) | xargs -l sed -i -e '1i/*\n * $PostgreSQL:$ \n */'
2008-05-17 09:28:26 +08:00
*
*
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
* btreefuncs.c
*
* Copyright (c) 2006 Satoshi Nagayasu <nagayasus@nttdata.co.jp>
*
* Permission to use, copy, modify, and distribute this software and
* its documentation for any purpose, without fee, and without a
* written agreement is hereby granted, provided that the above
* copyright notice and this paragraph and the following two
* paragraphs appear in all copies.
*
* IN NO EVENT SHALL THE AUTHOR BE LIABLE TO ANY PARTY FOR DIRECT,
* INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
* LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS
* DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED
* OF THE POSSIBILITY OF SUCH DAMAGE.
*
* THE AUTHOR SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS
* IS" BASIS, AND THE AUTHOR HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE,
* SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
*/
#include "postgres.h"
#include "access/nbtree.h"
Replace heapam.h includes with {table, relation}.h where applicable. A lot of files only included heapam.h for relation_open, heap_open etc - replace the heapam.h include in those files with the narrower header. Author: Andres Freund Discussion: https://postgr.es/m/20190111000539.xbv7s6w7ilcvm7dp@alap3.anarazel.de
2019-01-22 02:18:20 +08:00
#include "access/relation.h"
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
#include "catalog/namespace.h"
Restructure index access method API to hide most of it at the C level. This patch reduces pg_am to just two columns, a name and a handler function. All the data formerly obtained from pg_am is now provided in a C struct returned by the handler function. This is similar to the designs we've adopted for FDWs and tablesample methods. There are multiple advantages. For one, the index AM's support functions are now simple C functions, making them faster to call and much less error-prone, since the C compiler can now check function signatures. For another, this will make it far more practical to define index access methods in installable extensions. A disadvantage is that SQL-level code can no longer see attributes of index AMs; in particular, some of the crosschecks in the opr_sanity regression test are no longer possible from SQL. We've addressed that by adding a facility for the index AM to perform such checks instead. (Much more could be done in that line, but for now we're content if the amvalidate functions more or less replace what opr_sanity used to do.) We might also want to expose some sort of reporting functionality, but this patch doesn't do that. Alexander Korotkov, reviewed by Petr Jelínek, and rather heavily editorialized on by me.
2016-01-18 08:36:59 +08:00
#include "catalog/pg_am.h"
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
#include "catalog/pg_type.h"
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
#include "funcapi.h"
#include "miscadmin.h"
Make the order of the header file includes consistent in contrib modules. The basic rule we follow here is to always first include 'postgres.h' or 'postgres_fe.h' whichever is applicable, then system header includes and then Postgres header includes.  In this, we also follow that all the Postgres header includes are in order based on their ASCII value.  We generally follow these rules, but the code has deviated in many places. This commit makes it consistent just for contrib modules. The later commits will enforce similar rules in other parts of code. Author: Vignesh C Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CALDaNm2Sznv8RR6Ex-iJO6xAdsxgWhCoETkaYX=+9DW3q0QCfA@mail.gmail.com
2019-10-23 11:56:22 +08:00
#include "pageinspect.h"
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
#include "utils/array.h"
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
#include "utils/builtins.h"
Move Trigger and TriggerDesc structs out of rel.h into a new reltrigger.h This lets us stop including rel.h into execnodes.h, which is a widely used header.
2011-02-24 01:18:09 +08:00
#include "utils/rel.h"
Move some things from builtins.h to new header files This avoids that builtins.h has to include additional header files.
2017-01-21 09:29:53 +08:00
#include "utils/varlena.h"
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
PG_FUNCTION_INFO_V1(bt_metap);
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
PG_FUNCTION_INFO_V1(bt_page_items_1_9);
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
PG_FUNCTION_INFO_V1(bt_page_items);
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
PG_FUNCTION_INFO_V1(bt_page_items_bytea);
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
PG_FUNCTION_INFO_V1(bt_page_stats_1_9);
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
PG_FUNCTION_INFO_V1(bt_page_stats);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
#define IS_INDEX(r) ((r)->rd_rel->relkind == RELKIND_INDEX)
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
#define IS_BTREE(r) ((r)->rd_rel->relam == BTREE_AM_OID)
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
#define DatumGetItemPointer(X) ((ItemPointer) DatumGetPointer(X))
#define ItemPointerGetDatum(X) PointerGetDatum(X)
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Fix CHECK_RELATION_BLOCK_RANGE macro, which was not merely producing a warning but was outright wrong.
2007-07-16 07:46:20 +08:00
/* note: BlockNumber is unsigned, hence can't be negative */
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
#define CHECK_RELATION_BLOCK_RANGE(rel, blkno) { \
Fix CHECK_RELATION_BLOCK_RANGE macro, which was not merely producing a warning but was outright wrong.
2007-07-16 07:46:20 +08:00
if ( RelationGetNumberOfBlocks(rel) <= (BlockNumber) (blkno) ) \
Silence a rather odd compiler warning. In passing, make this file's error messages look at least a little bit like the message style guidelines say.
2007-07-16 07:09:26 +08:00
elog(ERROR, "block number out of range"); }
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
/* ------------------------------------------------
* structure for single btree page statistics
* ------------------------------------------------
*/
typedef struct BTPageStat
{
uint32 blkno;
uint32 live_items;
uint32 dead_items;
uint32 page_size;
uint32 max_avail;
uint32 free_size;
uint32 avg_item_size;
char type;
/* opaque data */
BlockNumber btpo_prev;
BlockNumber btpo_next;
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
uint32 btpo_level;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
uint16 btpo_flags;
BTCycleId btpo_cycleid;
} BTPageStat;
/* -------------------------------------------------
* GetBTPageStatistics()
*
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
* Collect statistics of single b-tree page
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
* -------------------------------------------------
*/
static void
GetBTPageStatistics(BlockNumber blkno, Buffer buffer, BTPageStat *stat)
{
Revert no-op changes to BufferGetPage() The reverted changes were intended to force a choice of whether any newly-added BufferGetPage() calls needed to be accompanied by a test of the snapshot age, to support the "snapshot too old" feature. Such an accompanying test is needed in about 7% of the cases, where the page is being used as part of a scan rather than positioning for other purposes (such as DML or vacuuming). The additional effort required for back-patching, and the doubt whether the intended benefit would really be there, have indicated it is best just to rely on developers to do the right thing based on comments and existing usage, as we do with many other conventions. This change should have little or no effect on generated executable code. Motivated by the back-patching pain of Tom Lane and Robert Haas
2016-04-20 21:31:19 +08:00
Page page = BufferGetPage(buffer);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
PageHeader phdr = (PageHeader) page;
OffsetNumber maxoff = PageGetMaxOffsetNumber(page);
BTPageOpaque opaque = (BTPageOpaque) PageGetSpecialPointer(page);
int item_size = 0;
int off;
stat->blkno = blkno;
stat->max_avail = BLCKSZ - (BLCKSZ - phdr->pd_special + SizeOfPageHeaderData);
stat->dead_items = stat->live_items = 0;
stat->page_size = PageGetPageSize(page);
/* page type (flags) */
if (P_ISDELETED(opaque))
{
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
/* We divide deleted pages into leaf ('d') or internal ('D') */
if (P_ISLEAF(opaque) || !P_HAS_FULLXID(opaque))
stat->type = 'd';
else
stat->type = 'D';
/*
* Report safexid in a deleted page.
*
* Handle pg_upgrade'd deleted pages that used the previous safexid
* representation in btpo_level field (this used to be a union type
* called "bpto").
*/
if (P_HAS_FULLXID(opaque))
{
FullTransactionId safexid = BTPageGetDeleteXid(page);
pageinspect: Make page deletion elog less chatty. An elog that reports the value of a transaction ID stored on a deleted nbtree page was added by commit e5d8a999, which taught page deletion to store full 64-bit XIDs. It seems very chatty on further reflection, so lower its elevel from NOTICE to DEBUG2. Author: Peter Geoghegan <pg@bowt.ie> Backpatch: 14-, just like the nbtree XID enhancement.
2021-09-18 05:19:51 +08:00
elog(DEBUG2, "deleted page from block %u has safexid %u:%u",
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
blkno, EpochFromFullTransactionId(safexid),
XidFromFullTransactionId(safexid));
}
else
pageinspect: Make page deletion elog less chatty. An elog that reports the value of a transaction ID stored on a deleted nbtree page was added by commit e5d8a999, which taught page deletion to store full 64-bit XIDs. It seems very chatty on further reflection, so lower its elevel from NOTICE to DEBUG2. Author: Peter Geoghegan <pg@bowt.ie> Backpatch: 14-, just like the nbtree XID enhancement.
2021-09-18 05:19:51 +08:00
elog(DEBUG2, "deleted page from block %u has safexid %u",
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
blkno, opaque->btpo_level);
/* Don't interpret BTDeletedPageData as index tuples */
maxoff = InvalidOffsetNumber;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
}
else if (P_IGNORE(opaque))
stat->type = 'e';
else if (P_ISLEAF(opaque))
stat->type = 'l';
else if (P_ISROOT(opaque))
stat->type = 'r';
else
stat->type = 'i';
/* btpage opaque data */
stat->btpo_prev = opaque->btpo_prev;
stat->btpo_next = opaque->btpo_next;
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
stat->btpo_level = opaque->btpo_level;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
stat->btpo_flags = opaque->btpo_flags;
stat->btpo_cycleid = opaque->btpo_cycleid;
/* count live and dead tuples, and free space */
for (off = FirstOffsetNumber; off <= maxoff; off++)
{
IndexTuple itup;
ItemId id = PageGetItemId(page, off);
itup = (IndexTuple) PageGetItem(page, id);
item_size += IndexTupleSize(itup);
Redefine the lp_flags field of item pointers as having four states, rather than two independent bits (one of which was never used in heap pages anyway, or at least hadn't been in a very long time). This gives us flexibility to add the HOT notions of redirected and dead item pointers without requiring anything so klugy as magic values of lp_off and lp_len. The state values are chosen so that for the states currently in use (pre-HOT) there is no change in the physical representation.
2007-09-13 06:10:26 +08:00
if (!ItemIdIsDead(id))
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
stat->live_items++;
else
stat->dead_items++;
}
stat->free_size = PageGetFreeSpace(page);
if ((stat->live_items + stat->dead_items) > 0)
stat->avg_item_size = item_size / (stat->live_items + stat->dead_items);
else
stat->avg_item_size = 0;
}
/* -----------------------------------------------
Take buffer lock while inspecting btree index pages in contrib/pageinspect. It's not safe to examine a shared buffer without any lock.
2012-12-01 06:02:29 +08:00
* bt_page_stats()
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
*
Take buffer lock while inspecting btree index pages in contrib/pageinspect. It's not safe to examine a shared buffer without any lock.
2012-12-01 06:02:29 +08:00
* Usage: SELECT * FROM bt_page_stats('t1_pkey', 1);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
* -----------------------------------------------
*/
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
static Datum
bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
{
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 07:35:34 +08:00
text *relname = PG_GETARG_TEXT_PP(0);
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
int64 blkno = (ext_version == PAGEINSPECT_V1_8 ? PG_GETARG_UINT32(1) : PG_GETARG_INT64(1));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Buffer buffer;
Relation rel;
RangeVar *relrv;
Datum result;
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
HeapTuple tuple;
TupleDesc tupleDesc;
int j;
char *values[11];
BTPageStat stat;
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
Remove excess parens in ereport() calls Cosmetic cleanup, not worth backpatching. Discussion: https://postgr.es/m/20200129200401.GA6303@alvherre.pgsql Reviewed-by: Tom Lane, Michael Paquier
2020-01-31 00:32:04 +08:00
errmsg("must be superuser to use pageinspect functions")));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
relrv = makeRangeVarFromNameList(textToQualifiedNameList(relname));
rel = relation_openrv(relrv, AccessShareLock);
if (!IS_INDEX(rel) || !IS_BTREE(rel))
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
elog(ERROR, "relation \"%s\" is not a btree index",
RelationGetRelationName(rel));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Fix contrib/pgstattuple and contrib/pageinspect to prevent attempts to read temporary tables of other sessions; that is unsafe because of the way our buffer management works. Per report from Stuart Bishop. This is redundant with the bufmgr.c checks in HEAD, but not at all redundant in the back branches.
2009-04-01 06:54:31 +08:00
/*
* Reject attempts to read non-local temporary relations; we would be
* likely to get wrong data since we have no visibility into the owning
* session's local buffers.
*/
if (RELATION_IS_OTHER_TEMP(rel))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
if (blkno < 0 || blkno > MaxBlockNumber)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("invalid block number")));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
if (blkno == 0)
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("block 0 is a meta page")));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
CHECK_RELATION_BLOCK_RANGE(rel, blkno);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
buffer = ReadBuffer(rel, blkno);
Take buffer lock while inspecting btree index pages in contrib/pageinspect. It's not safe to examine a shared buffer without any lock.
2012-12-01 06:02:29 +08:00
LockBuffer(buffer, BUFFER_LOCK_SHARE);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
/* keep compiler quiet */
stat.btpo_prev = stat.btpo_next = InvalidBlockNumber;
stat.btpo_flags = stat.free_size = stat.avg_item_size = 0;
GetBTPageStatistics(blkno, buffer, &stat);
Take buffer lock while inspecting btree index pages in contrib/pageinspect. It's not safe to examine a shared buffer without any lock.
2012-12-01 06:02:29 +08:00
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupleDesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
j = 0;
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
values[j++] = psprintf("%u", stat.blkno);
Add more use of psprintf()
2014-01-07 10:30:26 +08:00
values[j++] = psprintf("%c", stat.type);
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
values[j++] = psprintf("%u", stat.live_items);
values[j++] = psprintf("%u", stat.dead_items);
values[j++] = psprintf("%u", stat.avg_item_size);
values[j++] = psprintf("%u", stat.page_size);
values[j++] = psprintf("%u", stat.free_size);
values[j++] = psprintf("%u", stat.btpo_prev);
values[j++] = psprintf("%u", stat.btpo_next);
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
values[j++] = psprintf("%u", stat.btpo_level);
Add more use of psprintf()
2014-01-07 10:30:26 +08:00
values[j++] = psprintf("%d", stat.btpo_flags);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
result = HeapTupleGetDatum(tuple);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
PG_RETURN_DATUM(result);
}
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
Datum
bt_page_stats_1_9(PG_FUNCTION_ARGS)
{
return bt_page_stats_internal(fcinfo, PAGEINSPECT_V1_9);
}
/* entry point for old extension version */
Datum
bt_page_stats(PG_FUNCTION_ARGS)
{
return bt_page_stats_internal(fcinfo, PAGEINSPECT_V1_8);
}
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
/*
* cross-call data structure for SRF
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
*/
struct user_args
{
Page page;
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
OffsetNumber offset;
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
bool leafpage;
bool rightmost;
TupleDesc tupd;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
};
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
/*-------------------------------------------------------
* bt_page_print_tuples()
*
* Form a tuple describing index tuple at a given offset
* ------------------------------------------------------
*/
static Datum
Remove unused parameter unused since 93ee38eade1b2b4964354b95b01b09e17d6f098d Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
2020-09-04 14:02:58 +08:00
bt_page_print_tuples(struct user_args *uargs)
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
{
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
Page page = uargs->page;
OffsetNumber offset = uargs->offset;
bool leafpage = uargs->leafpage;
bool rightmost = uargs->rightmost;
bool ispivottuple;
Datum values[9];
bool nulls[9];
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
HeapTuple tuple;
ItemId id;
IndexTuple itup;
int j;
int off;
int dlen;
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
char *dump,
*datacstring;
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
char *ptr;
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
ItemPointer htid;
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
id = PageGetItemId(page, offset);
if (!ItemIdIsValid(id))
elog(ERROR, "invalid ItemId");
itup = (IndexTuple) PageGetItem(page, id);
j = 0;
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
memset(nulls, 0, sizeof(nulls));
values[j++] = DatumGetInt16(offset);
values[j++] = ItemPointerGetDatum(&itup->t_tid);
values[j++] = Int32GetDatum((int) IndexTupleSize(itup));
values[j++] = BoolGetDatum(IndexTupleHasNulls(itup));
values[j++] = BoolGetDatum(IndexTupleHasVarwidths(itup));
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
ptr = (char *) itup + IndexInfoFindDataOffset(itup->t_info);
dlen = IndexTupleSize(itup) - IndexInfoFindDataOffset(itup->t_info);
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
/*
* Make sure that "data" column does not include posting list or pivot
* tuple representation of heap TID(s).
*
* Note: BTreeTupleIsPivot() won't work reliably on !heapkeyspace indexes
* (those built before BTREE_VERSION 4), but we have no way of determining
* if this page came from a !heapkeyspace index. We may only have a bytea
* nbtree page image to go on, so in general there is no metapage that we
* can check.
*
* That's okay here because BTreeTupleIsPivot() can only return false for
* a !heapkeyspace pivot, never true for a !heapkeyspace non-pivot. Since
* heap TID isn't part of the keyspace in a !heapkeyspace index anyway,
* there cannot possibly be a pivot tuple heap TID representation that we
* fail to make an adjustment for. A !heapkeyspace index can have
* BTreeTupleIsPivot() return true (due to things like suffix truncation
* for INCLUDE indexes in Postgres v11), but when that happens
* BTreeTupleGetHeapTID() can be trusted to work reliably (i.e. return
* NULL).
*
* Note: BTreeTupleIsPosting() always works reliably, even with
* !heapkeyspace indexes.
*/
if (BTreeTupleIsPosting(itup))
dlen -= IndexTupleSize(itup) - BTreeTupleGetPostingOffset(itup);
else if (BTreeTupleIsPivot(itup) && BTreeTupleGetHeapTID(itup) != NULL)
dlen -= MAXALIGN(sizeof(ItemPointerData));
if (dlen < 0 || dlen > INDEX_SIZE_MASK)
elog(ERROR, "invalid tuple length %d for tuple at offset number %u",
dlen, offset);
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
dump = palloc0(dlen * 3 + 1);
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
datacstring = dump;
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
for (off = 0; off < dlen; off++)
{
if (off > 0)
*dump++ = ' ';
sprintf(dump, "%02x", *(ptr + off) & 0xff);
dump += 2;
}
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
values[j++] = CStringGetTextDatum(datacstring);
pfree(datacstring);
/*
* We need to work around the BTreeTupleIsPivot() !heapkeyspace limitation
* again. Deduce whether or not tuple must be a pivot tuple based on
* whether or not the page is a leaf page, as well as the page offset
* number of the tuple.
*/
ispivottuple = (!leafpage || (!rightmost && offset == P_HIKEY));
/* LP_DEAD bit can never be set for pivot tuples, so show a NULL there */
if (!ispivottuple)
values[j++] = BoolGetDatum(ItemIdIsDead(id));
else
{
Assert(!ItemIdIsDead(id));
nulls[j++] = true;
}
htid = BTreeTupleGetHeapTID(itup);
if (ispivottuple && !BTreeTupleIsPivot(itup))
{
/* Don't show bogus heap TID in !heapkeyspace pivot tuple */
htid = NULL;
}
if (htid)
values[j++] = ItemPointerGetDatum(htid);
else
nulls[j++] = true;
if (BTreeTupleIsPosting(itup))
{
/* Build an array of item pointers */
ItemPointer tids;
Datum *tids_datum;
int nposting;
tids = BTreeTupleGetPosting(itup);
nposting = BTreeTupleGetNPosting(itup);
tids_datum = (Datum *) palloc(nposting * sizeof(Datum));
for (int i = 0; i < nposting; i++)
tids_datum[i] = ItemPointerGetDatum(&tids[i]);
values[j++] = PointerGetDatum(construct_array(tids_datum,
nposting,
TIDOID,
sizeof(ItemPointerData),
Introduce macros for typalign and typstorage constants. Our usual practice for "poor man's enum" catalog columns is to define macros for the possible values and use those, not literal constants, in C code. But for some reason lost in the mists of time, this was never done for typalign/attalign or typstorage/attstorage. It's never too late to make it better though, so let's do that. The reason I got interested in this right now is the need to duplicate some uses of the TYPSTORAGE constants in an upcoming ALTER TYPE patch. But in general, this sort of change aids greppability and readability, so it's a good idea even without any specific motivation. I may have missed a few places that could be converted, and it's even more likely that pending patches will re-introduce some hard-coded references. But that's not fatal --- there's no expectation that we'd actually change any of these values. We can clean up stragglers over time. Discussion: https://postgr.es/m/16457.1583189537@sss.pgh.pa.us
2020-03-04 23:34:25 +08:00
false, TYPALIGN_SHORT));
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
pfree(tids_datum);
}
else
nulls[j++] = true;
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
return HeapTupleGetDatum(tuple);
}
/*-------------------------------------------------------
* bt_page_items()
*
* Get IndexTupleData set in a btree page
*
* Usage: SELECT * FROM bt_page_items('t1_pkey', 1);
*-------------------------------------------------------
*/
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
static Datum
bt_page_items_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
{
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 07:35:34 +08:00
text *relname = PG_GETARG_TEXT_PP(0);
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
int64 blkno = (ext_version == PAGEINSPECT_V1_8 ? PG_GETARG_UINT32(1) : PG_GETARG_INT64(1));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Datum result;
FuncCallContext *fctx;
MemoryContext mctx;
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
struct user_args *uargs;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
Remove excess parens in ereport() calls Cosmetic cleanup, not worth backpatching. Discussion: https://postgr.es/m/20200129200401.GA6303@alvherre.pgsql Reviewed-by: Tom Lane, Michael Paquier
2020-01-31 00:32:04 +08:00
errmsg("must be superuser to use pageinspect functions")));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
if (SRF_IS_FIRSTCALL())
{
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
RangeVar *relrv;
Relation rel;
Buffer buffer;
BTPageOpaque opaque;
TupleDesc tupleDesc;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
fctx = SRF_FIRSTCALL_INIT();
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
relrv = makeRangeVarFromNameList(textToQualifiedNameList(relname));
rel = relation_openrv(relrv, AccessShareLock);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
if (!IS_INDEX(rel) || !IS_BTREE(rel))
elog(ERROR, "relation \"%s\" is not a btree index",
RelationGetRelationName(rel));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Fix contrib/pgstattuple and contrib/pageinspect to prevent attempts to read temporary tables of other sessions; that is unsafe because of the way our buffer management works. Per report from Stuart Bishop. This is redundant with the bufmgr.c checks in HEAD, but not at all redundant in the back branches.
2009-04-01 06:54:31 +08:00
/*
* Reject attempts to read non-local temporary relations; we would be
* likely to get wrong data since we have no visibility into the
* owning session's local buffers.
*/
if (RELATION_IS_OTHER_TEMP(rel))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
if (blkno < 0 || blkno > MaxBlockNumber)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("invalid block number")));
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
if (blkno == 0)
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("block 0 is a meta page")));
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
CHECK_RELATION_BLOCK_RANGE(rel, blkno);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
buffer = ReadBuffer(rel, blkno);
Take buffer lock while inspecting btree index pages in contrib/pageinspect. It's not safe to examine a shared buffer without any lock.
2012-12-01 06:02:29 +08:00
LockBuffer(buffer, BUFFER_LOCK_SHARE);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
/*
* We copy the page into local storage to avoid holding pin on the
* buffer longer than we must, and possibly failing to release it at
* all if the calling query doesn't fetch all rows.
*/
mctx = MemoryContextSwitchTo(fctx->multi_call_memory_ctx);
uargs = palloc(sizeof(struct user_args));
uargs->page = palloc(BLCKSZ);
Revert no-op changes to BufferGetPage() The reverted changes were intended to force a choice of whether any newly-added BufferGetPage() calls needed to be accompanied by a test of the snapshot age, to support the "snapshot too old" feature. Such an accompanying test is needed in about 7% of the cases, where the page is being used as part of a scan rather than positioning for other purposes (such as DML or vacuuming). The additional effort required for back-patching, and the doubt whether the intended benefit would really be there, have indicated it is best just to rely on developers to do the right thing based on comments and existing usage, as we do with many other conventions. This change should have little or no effect on generated executable code. Motivated by the back-patching pain of Tom Lane and Robert Haas
2016-04-20 21:31:19 +08:00
memcpy(uargs->page, BufferGetPage(buffer), BLCKSZ);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Take buffer lock while inspecting btree index pages in contrib/pageinspect. It's not safe to examine a shared buffer without any lock.
2012-12-01 06:02:29 +08:00
UnlockReleaseBuffer(buffer);
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
relation_close(rel, AccessShareLock);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
uargs->offset = FirstOffsetNumber;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
opaque = (BTPageOpaque) PageGetSpecialPointer(uargs->page);
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
if (!P_ISDELETED(opaque))
fctx->max_calls = PageGetMaxOffsetNumber(uargs->page);
else
{
/* Don't interpret BTDeletedPageData as index tuples */
elog(NOTICE, "page from block " INT64_FORMAT " is deleted", blkno);
fctx->max_calls = 0;
}
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
uargs->leafpage = P_ISLEAF(opaque);
uargs->rightmost = P_RIGHTMOST(opaque);
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupleDesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
tupleDesc = BlessTupleDesc(tupleDesc);
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
uargs->tupd = tupleDesc;
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
fctx->user_fctx = uargs;
MemoryContextSwitchTo(mctx);
}
fctx = SRF_PERCALL_SETUP();
uargs = fctx->user_fctx;
if (fctx->call_cntr < fctx->max_calls)
{
Remove unused parameter unused since 93ee38eade1b2b4964354b95b01b09e17d6f098d Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
2020-09-04 14:02:58 +08:00
result = bt_page_print_tuples(uargs);
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
uargs->offset++;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
SRF_RETURN_NEXT(fctx, result);
}
Remove useless pfree()s at the ends of various ValuePerCall SRFs. We don't need to manually clean up allocations in a SRF's multi_call_memory_ctx, because the SRF_RETURN_DONE infrastructure takes care of that (and also ensures that it will happen even if the function never gets a final call, which simple manual cleanup cannot do). Hence, the code removed by this patch is a waste of code and cycles. Worse, it gives the impression that cleaning up manually is a thing, which can lead to more serious errors such as those fixed in commits 085b6b667 and b4570d33a. So we should get rid of it. These are not quite actual bugs though, so I couldn't muster the enthusiasm to back-patch. Fix in HEAD only. Justin Pryzby Discussion: https://postgr.es/m/20200308173103.GC1357@telsasoft.com
2020-03-17 09:36:53 +08:00
SRF_RETURN_DONE(fctx);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
}
pageinspect: Change block number arguments to bigint Block numbers are 32-bit unsigned integers. Therefore, the smallest SQL integer type that they can fit in is bigint. However, in the pageinspect module, most input and output parameters dealing with block numbers were declared as int. The behavior with block numbers larger than a signed 32-bit integer was therefore dubious. Change these arguments to type bigint and add some more explicit error checking on the block range. (Other contrib modules appear to do this correctly already.) Since we are changing argument types of existing functions, in order to not misbehave if the binary is updated before the extension is updated, we need to create new C symbols for the entry points, similar to how it's done in other extensions as well. Reported-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Michael Paquier <michael@paquier.xyz> Discussion: https://www.postgresql.org/message-id/flat/d8f6bdd536df403b9b33816e9f7e0b9d@G08CNEXMBPEKD05.g08.fujitsu.local
2021-01-19 17:28:05 +08:00
Datum
bt_page_items_1_9(PG_FUNCTION_ARGS)
{
return bt_page_items_internal(fcinfo, PAGEINSPECT_V1_9);
}
/* entry point for old extension version */
Datum
bt_page_items(PG_FUNCTION_ARGS)
{
return bt_page_items_internal(fcinfo, PAGEINSPECT_V1_8);
}
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
/*-------------------------------------------------------
* bt_page_items_bytea()
*
* Get IndexTupleData set in a btree page
*
* Usage: SELECT * FROM bt_page_items(get_raw_page('t1_pkey', 1));
*-------------------------------------------------------
*/
Datum
bt_page_items_bytea(PG_FUNCTION_ARGS)
{
bytea *raw_page = PG_GETARG_BYTEA_P(0);
Datum result;
FuncCallContext *fctx;
struct user_args *uargs;
int raw_page_size;
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
Remove excess parens in ereport() calls Cosmetic cleanup, not worth backpatching. Discussion: https://postgr.es/m/20200129200401.GA6303@alvherre.pgsql Reviewed-by: Tom Lane, Michael Paquier
2020-01-31 00:32:04 +08:00
errmsg("must be superuser to use raw page functions")));
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
if (SRF_IS_FIRSTCALL())
{
BTPageOpaque opaque;
MemoryContext mctx;
TupleDesc tupleDesc;
raw_page_size = VARSIZE(raw_page) - VARHDRSZ;
if (raw_page_size < SizeOfPageHeaderData)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("input page too small (%d bytes)", raw_page_size)));
fctx = SRF_FIRSTCALL_INIT();
mctx = MemoryContextSwitchTo(fctx->multi_call_memory_ctx);
uargs = palloc(sizeof(struct user_args));
uargs->page = VARDATA(raw_page);
uargs->offset = FirstOffsetNumber;
opaque = (BTPageOpaque) PageGetSpecialPointer(uargs->page);
if (P_ISMETA(opaque))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("block is a meta page")));
if (P_ISDELETED(opaque))
elog(NOTICE, "page is deleted");
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
if (!P_ISDELETED(opaque))
fctx->max_calls = PageGetMaxOffsetNumber(uargs->page);
else
{
/* Don't interpret BTDeletedPageData as index tuples */
elog(NOTICE, "page from block is deleted");
fctx->max_calls = 0;
}
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
uargs->leafpage = P_ISLEAF(opaque);
uargs->rightmost = P_RIGHTMOST(opaque);
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupleDesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
tupleDesc = BlessTupleDesc(tupleDesc);
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
uargs->tupd = tupleDesc;
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
fctx->user_fctx = uargs;
MemoryContextSwitchTo(mctx);
}
fctx = SRF_PERCALL_SETUP();
uargs = fctx->user_fctx;
if (fctx->call_cntr < fctx->max_calls)
{
Remove unused parameter unused since 93ee38eade1b2b4964354b95b01b09e17d6f098d Discussion: https://www.postgresql.org/message-id/flat/511bb100-f829-ba21-2f10-9f952ec06ead%402ndquadrant.com
2020-09-04 14:02:58 +08:00
result = bt_page_print_tuples(uargs);
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
uargs->offset++;
SRF_RETURN_NEXT(fctx, result);
}
Remove useless pfree()s at the ends of various ValuePerCall SRFs. We don't need to manually clean up allocations in a SRF's multi_call_memory_ctx, because the SRF_RETURN_DONE infrastructure takes care of that (and also ensures that it will happen even if the function never gets a final call, which simple manual cleanup cannot do). Hence, the code removed by this patch is a waste of code and cycles. Worse, it gives the impression that cleaning up manually is a thing, which can lead to more serious errors such as those fixed in commits 085b6b667 and b4570d33a. So we should get rid of it. These are not quite actual bugs though, so I couldn't muster the enthusiasm to back-patch. Fix in HEAD only. Justin Pryzby Discussion: https://postgr.es/m/20200308173103.GC1357@telsasoft.com
2020-03-17 09:36:53 +08:00
SRF_RETURN_DONE(fctx);
pageinspect: Add bt_page_items function with bytea argument Author: Tomas Vondra <tomas.vondra@2ndquadrant.com> Reviewed-by: Ashutosh Sharma <ashu.coek88@gmail.com>
2017-04-05 11:48:49 +08:00
}
pageinspect: Fix types used for bt_metap() columns. The data types that contrib/pageinspect's bt_metap() function were declared to return as OUT arguments were wrong in some cases. For example, the oldest_xact column (a TransactionId/xid field) was declared integer/int4 within the pageinspect extension's sql file. This led to errors when an oldest_xact value that exceeded 2^31-1 was encountered. Some of the other columns were defined incorrectly ever since pageinspect was first introduced, though they were far less likely to produce problems in practice. Fix these issues by changing the declaration of bt_metap() to consistently use data types that can reliably represent all possible values. This fixes things on HEAD only. No backpatch, since it doesn't seem like there is a safe way to fix the issue without including a new version of the pageinspect extension (HEAD/Postgres 13 already introduced a new version of the extension). Besides, the oldest_xact issue has been around since the release of Postgres 11, and we haven't heard any complaints about it before now. Also, throw an error when we detect a bt_metap() declaration that must be from an old version of the pageinspect extension by examining the number of attributes from the tuple descriptor for the return tuples. It seems better to throw an error in a reliable and obvious way following a Postgres upgrade, rather than letting bt_metap() fail unpredictably. The problem is fundamentally with the CREATE FUNCTION declared data types themselves, so I see no sensible alternative. Reported-By: Victor Yegorov Bug: #16285 Discussion: https://postgr.es/m/16285-df8fc1000ab3d5fc@postgresql.org
2020-03-08 08:44:53 +08:00
/* Number of output arguments (columns) for bt_metap() */
#define BT_METAP_COLS_V1_8 9
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
/* ------------------------------------------------
* bt_metap()
*
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
* Get a btree's meta-page information
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
*
* Usage: SELECT * FROM bt_metap('t1_pkey')
* ------------------------------------------------
*/
Datum
bt_metap(PG_FUNCTION_ARGS)
{
Use wrappers of PG_DETOAST_DATUM_PACKED() more. This makes almost all core code follow the policy introduced in the previous commit. Specific decisions: - Text search support functions with char* and length arguments, such as prsstart and lexize, may receive unaligned strings. I doubt maintainers of non-core text search code will notice. - Use plain VARDATA() on values detoasted or synthesized earlier in the same function. Use VARDATA_ANY() on varlenas sourced outside the function, even if they happen to always have four-byte headers. As an exception, retain the universal practice of using VARDATA() on return values of SendFunctionCall(). - Retain PG_GETARG_BYTEA_P() in pageinspect. (Page images are too large for a one-byte header, so this misses no optimization.) Sites that do not call get_page_from_raw() typically need the four-byte alignment. - For now, do not change btree_gist. Its use of four-byte headers in memory is partly entangled with storage of 4-byte headers inside GBT_VARKEY, on disk. - For now, do not change gtrgm_consistent() or gtrgm_distance(). They incorporate the varlena header into a cache, and there are multiple credible implementation strategies to consider.
2017-03-13 07:35:34 +08:00
text *relname = PG_GETARG_TEXT_PP(0);
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
Datum result;
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Relation rel;
RangeVar *relrv;
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
BTMetaPageData *metad;
TupleDesc tupleDesc;
int j;
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
char *values[9];
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
Buffer buffer;
Page page;
HeapTuple tuple;
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
Remove excess parens in ereport() calls Cosmetic cleanup, not worth backpatching. Discussion: https://postgr.es/m/20200129200401.GA6303@alvherre.pgsql Reviewed-by: Tom Lane, Michael Paquier
2020-01-31 00:32:04 +08:00
errmsg("must be superuser to use pageinspect functions")));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
relrv = makeRangeVarFromNameList(textToQualifiedNameList(relname));
rel = relation_openrv(relrv, AccessShareLock);
if (!IS_INDEX(rel) || !IS_BTREE(rel))
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
elog(ERROR, "relation \"%s\" is not a btree index",
RelationGetRelationName(rel));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Fix contrib/pgstattuple and contrib/pageinspect to prevent attempts to read temporary tables of other sessions; that is unsafe because of the way our buffer management works. Per report from Stuart Bishop. This is redundant with the bufmgr.c checks in HEAD, but not at all redundant in the back branches.
2009-04-01 06:54:31 +08:00
/*
* Reject attempts to read non-local temporary relations; we would be
* likely to get wrong data since we have no visibility into the owning
* session's local buffers.
*/
if (RELATION_IS_OTHER_TEMP(rel))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot access temporary tables of other sessions")));
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
buffer = ReadBuffer(rel, 0);
Take buffer lock while inspecting btree index pages in contrib/pageinspect. It's not safe to examine a shared buffer without any lock.
2012-12-01 06:02:29 +08:00
LockBuffer(buffer, BUFFER_LOCK_SHARE);
Revert no-op changes to BufferGetPage() The reverted changes were intended to force a choice of whether any newly-added BufferGetPage() calls needed to be accompanied by a test of the snapshot age, to support the "snapshot too old" feature. Such an accompanying test is needed in about 7% of the cases, where the page is being used as part of a scan rather than positioning for other purposes (such as DML or vacuuming). The additional effort required for back-patching, and the doubt whether the intended benefit would really be there, have indicated it is best just to rely on developers to do the right thing based on comments and existing usage, as we do with many other conventions. This change should have little or no effect on generated executable code. Motivated by the back-patching pain of Tom Lane and Robert Haas
2016-04-20 21:31:19 +08:00
page = BufferGetPage(buffer);
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
metad = BTPageGetMeta(page);
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupleDesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
pageinspect: Fix types used for bt_metap() columns. The data types that contrib/pageinspect's bt_metap() function were declared to return as OUT arguments were wrong in some cases. For example, the oldest_xact column (a TransactionId/xid field) was declared integer/int4 within the pageinspect extension's sql file. This led to errors when an oldest_xact value that exceeded 2^31-1 was encountered. Some of the other columns were defined incorrectly ever since pageinspect was first introduced, though they were far less likely to produce problems in practice. Fix these issues by changing the declaration of bt_metap() to consistently use data types that can reliably represent all possible values. This fixes things on HEAD only. No backpatch, since it doesn't seem like there is a safe way to fix the issue without including a new version of the pageinspect extension (HEAD/Postgres 13 already introduced a new version of the extension). Besides, the oldest_xact issue has been around since the release of Postgres 11, and we haven't heard any complaints about it before now. Also, throw an error when we detect a bt_metap() declaration that must be from an old version of the pageinspect extension by examining the number of attributes from the tuple descriptor for the return tuples. It seems better to throw an error in a reliable and obvious way following a Postgres upgrade, rather than letting bt_metap() fail unpredictably. The problem is fundamentally with the CREATE FUNCTION declared data types themselves, so I see no sensible alternative. Reported-By: Victor Yegorov Bug: #16285 Discussion: https://postgr.es/m/16285-df8fc1000ab3d5fc@postgresql.org
2020-03-08 08:44:53 +08:00
/*
* We need a kluge here to detect API versions prior to 1.8. Earlier
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
* versions incorrectly used int4 for certain columns.
pageinspect: Fix types used for bt_metap() columns. The data types that contrib/pageinspect's bt_metap() function were declared to return as OUT arguments were wrong in some cases. For example, the oldest_xact column (a TransactionId/xid field) was declared integer/int4 within the pageinspect extension's sql file. This led to errors when an oldest_xact value that exceeded 2^31-1 was encountered. Some of the other columns were defined incorrectly ever since pageinspect was first introduced, though they were far less likely to produce problems in practice. Fix these issues by changing the declaration of bt_metap() to consistently use data types that can reliably represent all possible values. This fixes things on HEAD only. No backpatch, since it doesn't seem like there is a safe way to fix the issue without including a new version of the pageinspect extension (HEAD/Postgres 13 already introduced a new version of the extension). Besides, the oldest_xact issue has been around since the release of Postgres 11, and we haven't heard any complaints about it before now. Also, throw an error when we detect a bt_metap() declaration that must be from an old version of the pageinspect extension by examining the number of attributes from the tuple descriptor for the return tuples. It seems better to throw an error in a reliable and obvious way following a Postgres upgrade, rather than letting bt_metap() fail unpredictably. The problem is fundamentally with the CREATE FUNCTION declared data types themselves, so I see no sensible alternative. Reported-By: Victor Yegorov Bug: #16285 Discussion: https://postgr.es/m/16285-df8fc1000ab3d5fc@postgresql.org
2020-03-08 08:44:53 +08:00
*
* There is no way to reliably avoid the problems created by the old
* function definition at this point, so insist that the user update the
* extension.
*/
if (tupleDesc->natts < BT_METAP_COLS_V1_8)
ereport(ERROR,
(errcode(ERRCODE_INVALID_FUNCTION_DEFINITION),
errmsg("function has wrong number of declared columns"),
errhint("To resolve the problem, update the \"pageinspect\" extension to the latest version.")));
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
j = 0;
Add more use of psprintf()
2014-01-07 10:30:26 +08:00
values[j++] = psprintf("%d", metad->btm_magic);
values[j++] = psprintf("%d", metad->btm_version);
pageinspect: Fix types used for bt_metap() columns. The data types that contrib/pageinspect's bt_metap() function were declared to return as OUT arguments were wrong in some cases. For example, the oldest_xact column (a TransactionId/xid field) was declared integer/int4 within the pageinspect extension's sql file. This led to errors when an oldest_xact value that exceeded 2^31-1 was encountered. Some of the other columns were defined incorrectly ever since pageinspect was first introduced, though they were far less likely to produce problems in practice. Fix these issues by changing the declaration of bt_metap() to consistently use data types that can reliably represent all possible values. This fixes things on HEAD only. No backpatch, since it doesn't seem like there is a safe way to fix the issue without including a new version of the pageinspect extension (HEAD/Postgres 13 already introduced a new version of the extension). Besides, the oldest_xact issue has been around since the release of Postgres 11, and we haven't heard any complaints about it before now. Also, throw an error when we detect a bt_metap() declaration that must be from an old version of the pageinspect extension by examining the number of attributes from the tuple descriptor for the return tuples. It seems better to throw an error in a reliable and obvious way following a Postgres upgrade, rather than letting bt_metap() fail unpredictably. The problem is fundamentally with the CREATE FUNCTION declared data types themselves, so I see no sensible alternative. Reported-By: Victor Yegorov Bug: #16285 Discussion: https://postgr.es/m/16285-df8fc1000ab3d5fc@postgresql.org
2020-03-08 08:44:53 +08:00
values[j++] = psprintf(INT64_FORMAT, (int64) metad->btm_root);
values[j++] = psprintf(INT64_FORMAT, (int64) metad->btm_level);
values[j++] = psprintf(INT64_FORMAT, (int64) metad->btm_fastroot);
values[j++] = psprintf(INT64_FORMAT, (int64) metad->btm_fastlevel);
Fix handling of non-upgraded B-tree metapages 857f9c36 bumps B-tree metapage version while upgrade is performed "on the fly" when needed. However, some asserts fired when old version metapage was cached to rel->rd_amcache. Despite new metadata fields are never used from rel->rd_amcache, that needs to be fixed. This patch introduces metadata upgrade during its caching, which fills unavailable fields with their default values. contrib/pageinspect is also patched to handle non-upgraded metapages in the same way. Author: Alexander Korotkov
2018-04-05 22:56:00 +08:00
/*
* Get values of extended metadata if available, use default values
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
* otherwise. Note that we rely on the assumption that btm_allequalimage
* is initialized to zero with indexes that were built on versions prior
* to Postgres 13 (just like _bt_metaversion()).
Fix handling of non-upgraded B-tree metapages 857f9c36 bumps B-tree metapage version while upgrade is performed "on the fly" when needed. However, some asserts fired when old version metapage was cached to rel->rd_amcache. Despite new metadata fields are never used from rel->rd_amcache, that needs to be fixed. This patch introduces metadata upgrade during its caching, which fills unavailable fields with their default values. contrib/pageinspect is also patched to handle non-upgraded metapages in the same way. Author: Alexander Korotkov
2018-04-05 22:56:00 +08:00
*/
Make heap TID a tiebreaker nbtree index column. Make nbtree treat all index tuples as having a heap TID attribute. Index searches can distinguish duplicates by heap TID, since heap TID is always guaranteed to be unique. This general approach has numerous benefits for performance, and is prerequisite to teaching VACUUM to perform "retail index tuple deletion". Naively adding a new attribute to every pivot tuple has unacceptable overhead (it bloats internal pages), so suffix truncation of pivot tuples is added. This will usually truncate away the "extra" heap TID attribute from pivot tuples during a leaf page split, and may also truncate away additional user attributes. This can increase fan-out, especially in a multi-column index. Truncation can only occur at the attribute granularity, which isn't particularly effective, but works well enough for now. A future patch may add support for truncating "within" text attributes by generating truncated key values using new opclass infrastructure. Only new indexes (BTREE_VERSION 4 indexes) will have insertions that treat heap TID as a tiebreaker attribute, or will have pivot tuples undergo suffix truncation during a leaf page split (on-disk compatibility with versions 2 and 3 is preserved). Upgrades to version 4 cannot be performed on-the-fly, unlike upgrades from version 2 to version 3. contrib/amcheck continues to work with version 2 and 3 indexes, while also enforcing stricter invariants when verifying version 4 indexes. These stricter invariants are the same invariants described by "3.1.12 Sequencing" from the Lehman and Yao paper. A later patch will enhance the logic used by nbtree to pick a split point. This patch is likely to negatively impact performance without smarter choices around the precise point to split leaf pages at. Making these two mostly-distinct sets of enhancements into distinct commits seems like it might clarify their design, even though neither commit is particularly useful on its own. The maximum allowed size of new tuples is reduced by an amount equal to the space required to store an extra MAXALIGN()'d TID in a new high key during leaf page splits. The user-facing definition of the "1/3 of a page" restriction is already imprecise, and so does not need to be revised. However, there should be a compatibility note in the v12 release notes. Author: Peter Geoghegan Reviewed-By: Heikki Linnakangas, Alexander Korotkov Discussion: https://postgr.es/m/CAH2-WzkVb0Kom=R+88fDFb=JSxZMFvbHVC6Mn9LJ2n=X=kS-Uw@mail.gmail.com
2019-03-21 01:04:01 +08:00
if (metad->btm_version >= BTREE_NOVAC_VERSION)
Fix handling of non-upgraded B-tree metapages 857f9c36 bumps B-tree metapage version while upgrade is performed "on the fly" when needed. However, some asserts fired when old version metapage was cached to rel->rd_amcache. Despite new metadata fields are never used from rel->rd_amcache, that needs to be fixed. This patch introduces metadata upgrade during its caching, which fills unavailable fields with their default values. contrib/pageinspect is also patched to handle non-upgraded metapages in the same way. Author: Alexander Korotkov
2018-04-05 22:56:00 +08:00
{
Use full 64-bit XIDs in deleted nbtree pages. Otherwise we risk "leaking" deleted pages by making them non-recyclable indefinitely. Commit 6655a729 did the same thing for deleted pages in GiST indexes. That work was used as a starting point here. Stop storing an XID indicating the oldest bpto.xact across all deleted though unrecycled pages in nbtree metapages. There is no longer any reason to care about that condition/the oldest XID. It only ever made sense when wraparound was something _bt_vacuum_needs_cleanup() had to consider. The btm_oldest_btpo_xact metapage field has been repurposed and renamed. It is now btm_last_cleanup_num_delpages, which is used to remember how many non-recycled deleted pages remain from the last VACUUM (in practice its value is usually the precise number of pages that were _newly deleted_ during the specific VACUUM operation that last set the field). The general idea behind storing btm_last_cleanup_num_delpages is to use it to give _some_ consideration to non-recycled deleted pages inside _bt_vacuum_needs_cleanup() -- though never too much. We only really need to avoid leaving a truly excessive number of deleted pages in an unrecycled state forever. We only do this to cover certain narrow cases where no other factor makes VACUUM do a full scan, and yet the index continues to grow (and so actually misses out on recycling existing deleted pages). These metapage changes result in a clear user-visible benefit: We no longer trigger full index scans during VACUUM operations solely due to the presence of only 1 or 2 known deleted (though unrecycled) blocks from a very large index. All that matters now is keeping the costs and benefits in balance over time. Fix an issue that has been around since commit 857f9c36, which added the "skip full scan of index" mechanism (i.e. the _bt_vacuum_needs_cleanup() logic). The accuracy of btm_last_cleanup_num_heap_tuples accidentally hinged upon _when_ the source value gets stored. We now always store btm_last_cleanup_num_heap_tuples in btvacuumcleanup(). This fixes the issue because IndexVacuumInfo.num_heap_tuples (the source field) is expected to accurately indicate the state of the table _after_ the VACUUM completes inside btvacuumcleanup(). A backpatchable fix cannot easily be extracted from this commit. A targeted fix for the issue will follow in a later commit, though that won't happen today. I (pgeoghegan) have chosen to remove any mention of deleted pages in the documentation of the vacuum_cleanup_index_scale_factor GUC/param, since the presence of deleted (though unrecycled) pages is no longer of much concern to users. The vacuum_cleanup_index_scale_factor description in the docs now seems rather unclear in any case, and it should probably be rewritten in the near future. Perhaps some passing mention of page deletion will be added back at the same time. Bump XLOG_PAGE_MAGIC due to nbtree WAL records using full XIDs now. Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Masahiko Sawada <sawada.mshk@gmail.com> Discussion: https://postgr.es/m/CAH2-WznpdHvujGUwYZ8sihX=d5u-tRYhi-F4wnV2uN2zHpMUXw@mail.gmail.com
2021-02-25 10:41:34 +08:00
values[j++] = psprintf(INT64_FORMAT,
(int64) metad->btm_last_cleanup_num_delpages);
printf("%lf") is not portable, so omit the "l". The "l" (ell) width spec means something in the corresponding scanf usage, but not here. While modern POSIX says that applying "l" to "f" and other floating format specs is a no-op, SUSv2 says it's undefined. Buildfarm experience says that some old compilers emit warnings about it, and at least one old stdio implementation (mingw's "ANSI" option) actually produces wrong answers and/or crashes. Discussion: https://postgr.es/m/21670.1526769114@sss.pgh.pa.us Discussion: https://postgr.es/m/c085e1da-0d64-1c15-242d-c921f32e0d5c@dunslane.net
2018-05-20 23:40:54 +08:00
values[j++] = psprintf("%f", metad->btm_last_cleanup_num_heap_tuples);
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
values[j++] = metad->btm_allequalimage ? "t" : "f";
Fix handling of non-upgraded B-tree metapages 857f9c36 bumps B-tree metapage version while upgrade is performed "on the fly" when needed. However, some asserts fired when old version metapage was cached to rel->rd_amcache. Despite new metadata fields are never used from rel->rd_amcache, that needs to be fixed. This patch introduces metadata upgrade during its caching, which fills unavailable fields with their default values. contrib/pageinspect is also patched to handle non-upgraded metapages in the same way. Author: Alexander Korotkov
2018-04-05 22:56:00 +08:00
}
else
{
values[j++] = "0";
values[j++] = "-1";
Teach pageinspect about nbtree deduplication. Add a new bt_metap() column to display the metapage's allequalimage field. Also add three new columns to contrib/pageinspect's bt_page_items() function: * Add a boolean column ("dead") that displays the LP_DEAD bit value for each non-pivot tuple. * Add a TID column ("htid") that displays a single heap TID value for each tuple. This is the TID that is returned by BTreeTupleGetHeapTID(), so comparable values are shown for pivot tuples, plain non-pivot tuples, and posting list tuples. * Add a TID array column ("tids") that displays TIDs from each tuple's posting list, if any. This works just like the "tids" column from pageinspect's gin_leafpage_items() function. No version bump for the pageinspect extension, since there hasn't been a stable Postgres release since the last version bump (the last bump was part of commit 58b4cb30). Author: Peter Geoghegan Discussion: https://postgr.es/m/CAH2-WzmSMmU2eNvY9+a4MNP+z02h6sa-uxZvN3un6jY02ZVBSw@mail.gmail.com
2020-03-01 04:10:17 +08:00
values[j++] = "f";
Fix handling of non-upgraded B-tree metapages 857f9c36 bumps B-tree metapage version while upgrade is performed "on the fly" when needed. However, some asserts fired when old version metapage was cached to rel->rd_amcache. Despite new metadata fields are never used from rel->rd_amcache, that needs to be fixed. This patch introduces metadata upgrade during its caching, which fills unavailable fields with their default values. contrib/pageinspect is also patched to handle non-upgraded metapages in the same way. Author: Alexander Korotkov
2018-04-05 22:56:00 +08:00
}
Code review for btreefuncs additions: restrict to superusers to avoid exposing user data to others, and clean up usage of deprecated APIs.
2007-08-27 07:22:49 +08:00
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
result = HeapTupleGetDatum(tuple);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
Take buffer lock while inspecting btree index pages in contrib/pageinspect. It's not safe to examine a shared buffer without any lock.
2012-12-01 06:02:29 +08:00
UnlockReleaseBuffer(buffer);
Add database page inspection /contrib module. Simon and Heikki
2007-05-18 03:11:25 +08:00
relation_close(rel, AccessShareLock);
PG_RETURN_DATUM(result);
}
Reference in New Issue Copy Permalink