postgresql/contrib/pageinspect/heapfuncs.c

623 lines
17 KiB
C
Raw Normal View History

/*-------------------------------------------------------------------------
*
* heapfuncs.c
* Functions to investigate heap pages
*
* We check the input to these functions for corrupt pointers etc. that
* might cause crashes, but at the same time we try to print out as much
* information as possible, even if it's nonsense. That's because if a
* page is corrupt, we don't know why and how exactly it is corrupt, so we
* let the user judge it.
2007-11-16 05:14:46 +08:00
*
* These functions are restricted to superusers for the fear of introducing
* security holes if the input checking isn't as water-tight as it should be.
2007-11-16 05:14:46 +08:00
* You'd need to be superuser to obtain a raw page image anyway, so
* there's hardly any use case for using these without superuser-rights
* anyway.
*
* Copyright (c) 2007-2021, PostgreSQL Global Development Group
*
* IDENTIFICATION
2010-09-21 04:08:53 +08:00
* contrib/pageinspect/heapfuncs.c
*
*-------------------------------------------------------------------------
*/
#include "postgres.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_type.h"
#include "funcapi.h"
Avoid using %c printf format for potentially non-ASCII characters. Since %c only passes a C "char" to printf, it's incapable of dealing with multibyte characters. Passing just the first byte of such a character leads to an output string that is visibly not correctly encoded, resulting in undesirable behavior such as encoding conversion failures while sending error messages to clients. We've lived with this issue for a long time because it was inconvenient to avoid in a portable fashion. However, now that we always use our own snprintf code, it's reasonable to use the %.*s format to print just one possibly-multibyte character in a string. (We previously avoided that obvious-looking answer in order to work around glibc's bug #6530, cf commits 54cd4f045 and ed437e2b2.) Hence, run around and fix a bunch of places that used %c to report a character found in a user-supplied string. For simplicity, I did not touch places that were emitting non-user-facing debug messages, or reporting catalog data that should always be ASCII. (It's also unclear how useful this approach could be in frontend code, where it's less certain that we know what encoding we're dealing with.) In passing, improve a couple of poorly-written error messages in pageinspect/heapfuncs.c. This is a longstanding issue, but I'm hesitant to back-patch because of the impact on translatable message strings. In any case this fix would not work reliably before v12. Tom Lane and Quan Zongliang Discussion: https://postgr.es/m/a120087c-4c88-d9d4-1ec5-808d7a7f133d@gmail.com
2020-06-29 23:41:19 +08:00
#include "mb/pg_wchar.h"
#include "miscadmin.h"
#include "pageinspect.h"
#include "port/pg_bitutils.h"
#include "utils/array.h"
#include "utils/builtins.h"
#include "utils/rel.h"
Remove WITH OIDS support, change oid catalog column visibility. Previously tables declared WITH OIDS, including a significant fraction of the catalog tables, stored the oid column not as a normal column, but as part of the tuple header. This special column was not shown by default, which was somewhat odd, as it's often (consider e.g. pg_class.oid) one of the more important parts of a row. Neither pg_dump nor COPY included the contents of the oid column by default. The fact that the oid column was not an ordinary column necessitated a significant amount of special case code to support oid columns. That already was painful for the existing, but upcoming work aiming to make table storage pluggable, would have required expanding and duplicating that "specialness" significantly. WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0). Remove it. Removing includes: - CREATE TABLE and ALTER TABLE syntax for declaring the table to be WITH OIDS has been removed (WITH (oids[ = true]) will error out) - pg_dump does not support dumping tables declared WITH OIDS and will issue a warning when dumping one (and ignore the oid column). - restoring an pg_dump archive with pg_restore will warn when restoring a table with oid contents (and ignore the oid column) - COPY will refuse to load binary dump that includes oids. - pg_upgrade will error out when encountering tables declared WITH OIDS, they have to be altered to remove the oid column first. - Functionality to access the oid of the last inserted row (like plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed. The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false) for CREATE TABLE) is still supported. While that requires a bit of support code, it seems unnecessary to break applications / dumps that do not use oids, and are explicit about not using them. The biggest user of WITH OID columns was postgres' catalog. This commit changes all 'magic' oid columns to be columns that are normally declared and stored. To reduce unnecessary query breakage all the newly added columns are still named 'oid', even if a table's column naming scheme would indicate 'reloid' or such. This obviously requires adapting a lot code, mostly replacing oid access via HeapTupleGetOid() with access to the underlying Form_pg_*->oid column. The bootstrap process now assigns oids for all oid columns in genbki.pl that do not have an explicit value (starting at the largest oid previously used), only oids assigned later by oids will be above FirstBootstrapObjectId. As the oid column now is a normal column the special bootstrap syntax for oids has been removed. Oids are not automatically assigned during insertion anymore, all backend code explicitly assigns oids with GetNewOidWithIndex(). For the rare case that insertions into the catalog via SQL are called for the new pg_nextoid() function can be used (which only works on catalog tables). The fact that oid columns on system tables are now normal columns means that they will be included in the set of columns expanded by * (i.e. SELECT * FROM pg_class will now include the table's oid, previously it did not). It'd not technically be hard to hide oid column by default, but that'd mean confusing behavior would either have to be carried forward forever, or it'd cause breakage down the line. While it's not unlikely that further adjustments are needed, the scope/invasiveness of the patch makes it worthwhile to get merge this now. It's painful to maintain externally, too complicated to commit after the code code freeze, and a dependency of a number of other patches. Catversion bump, for obvious reasons. Author: Andres Freund, with contributions by John Naylor Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
2018-11-21 07:36:57 +08:00
/*
* It's not supported to create tuples with oids anymore, but when pg_upgrade
* was used to upgrade from an older version, tuples might still have an
* oid. Seems worthwhile to display that.
*/
#define HeapTupleHeaderGetOidOld(tup) \
( \
((tup)->t_infomask & HEAP_HASOID_OLD) ? \
*((Oid *) ((char *)(tup) + (tup)->t_hoff - sizeof(Oid))) \
: \
InvalidOid \
)
/*
* bits_to_text
*
* Converts a bits8-array of 'len' bits to a human-readable
* c-string representation.
*/
static char *
bits_to_text(bits8 *bits, int len)
{
2007-11-16 05:14:46 +08:00
int i;
char *str;
str = palloc(len + 1);
2007-11-16 05:14:46 +08:00
for (i = 0; i < len; i++)
str[i] = (bits[(i / 8)] & (1 << (i % 8))) ? '1' : '0';
str[i] = '\0';
return str;
}
/*
* text_to_bits
*
* Converts a c-string representation of bits into a bits8-array. This is
* the reverse operation of previous routine.
*/
static bits8 *
text_to_bits(char *str, int len)
{
bits8 *bits;
int off = 0;
char byte = 0;
bits = palloc(len + 1);
while (off < len)
{
if (off % 8 == 0)
byte = 0;
if ((str[off] == '0') || (str[off] == '1'))
byte = byte | ((str[off] - '0') << off % 8);
else
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
Avoid using %c printf format for potentially non-ASCII characters. Since %c only passes a C "char" to printf, it's incapable of dealing with multibyte characters. Passing just the first byte of such a character leads to an output string that is visibly not correctly encoded, resulting in undesirable behavior such as encoding conversion failures while sending error messages to clients. We've lived with this issue for a long time because it was inconvenient to avoid in a portable fashion. However, now that we always use our own snprintf code, it's reasonable to use the %.*s format to print just one possibly-multibyte character in a string. (We previously avoided that obvious-looking answer in order to work around glibc's bug #6530, cf commits 54cd4f045 and ed437e2b2.) Hence, run around and fix a bunch of places that used %c to report a character found in a user-supplied string. For simplicity, I did not touch places that were emitting non-user-facing debug messages, or reporting catalog data that should always be ASCII. (It's also unclear how useful this approach could be in frontend code, where it's less certain that we know what encoding we're dealing with.) In passing, improve a couple of poorly-written error messages in pageinspect/heapfuncs.c. This is a longstanding issue, but I'm hesitant to back-patch because of the impact on translatable message strings. In any case this fix would not work reliably before v12. Tom Lane and Quan Zongliang Discussion: https://postgr.es/m/a120087c-4c88-d9d4-1ec5-808d7a7f133d@gmail.com
2020-06-29 23:41:19 +08:00
errmsg("invalid character \"%.*s\" in t_bits string",
pg_mblen(str + off), str + off)));
if (off % 8 == 7)
bits[off / 8] = byte;
off++;
}
return bits;
}
/*
* heap_page_items
*
* Allows inspection of line pointers and tuple headers of a heap page.
*/
PG_FUNCTION_INFO_V1(heap_page_items);
typedef struct heap_page_items_state
{
TupleDesc tupd;
Page page;
uint16 offset;
} heap_page_items_state;
Datum
heap_page_items(PG_FUNCTION_ARGS)
{
2007-11-16 05:14:46 +08:00
bytea *raw_page = PG_GETARG_BYTEA_P(0);
heap_page_items_state *inter_call_data = NULL;
FuncCallContext *fctx;
2007-11-16 05:14:46 +08:00
int raw_page_size;
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser to use raw page functions")));
raw_page_size = VARSIZE(raw_page) - VARHDRSZ;
if (SRF_IS_FIRSTCALL())
{
TupleDesc tupdesc;
MemoryContext mctx;
2007-11-16 05:14:46 +08:00
if (raw_page_size < SizeOfPageHeaderData)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("input page too small (%d bytes)", raw_page_size)));
fctx = SRF_FIRSTCALL_INIT();
mctx = MemoryContextSwitchTo(fctx->multi_call_memory_ctx);
inter_call_data = palloc(sizeof(heap_page_items_state));
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
inter_call_data->tupd = tupdesc;
inter_call_data->offset = FirstOffsetNumber;
inter_call_data->page = VARDATA(raw_page);
fctx->max_calls = PageGetMaxOffsetNumber(inter_call_data->page);
fctx->user_fctx = inter_call_data;
MemoryContextSwitchTo(mctx);
}
fctx = SRF_PERCALL_SETUP();
inter_call_data = fctx->user_fctx;
if (fctx->call_cntr < fctx->max_calls)
{
Page page = inter_call_data->page;
HeapTuple resultTuple;
Datum result;
ItemId id;
Datum values[14];
bool nulls[14];
2007-11-16 05:14:46 +08:00
uint16 lp_offset;
uint16 lp_flags;
uint16 lp_len;
memset(nulls, 0, sizeof(nulls));
/* Extract information from the line pointer */
2007-11-16 05:14:46 +08:00
id = PageGetItemId(page, inter_call_data->offset);
2007-11-16 05:14:46 +08:00
lp_offset = ItemIdGetOffset(id);
lp_flags = ItemIdGetFlags(id);
lp_len = ItemIdGetLength(id);
values[0] = UInt16GetDatum(inter_call_data->offset);
values[1] = UInt16GetDatum(lp_offset);
values[2] = UInt16GetDatum(lp_flags);
values[3] = UInt16GetDatum(lp_len);
2007-11-16 05:14:46 +08:00
/*
* We do just enough validity checking to make sure we don't reference
* data outside the page passed to us. The page could be corrupt in
* many other ways, but at least we won't crash.
*/
if (ItemIdHasStorage(id) &&
lp_len >= MinHeapTupleSize &&
lp_offset == MAXALIGN(lp_offset) &&
lp_offset + lp_len <= raw_page_size)
{
2016-06-10 06:02:36 +08:00
HeapTupleHeader tuphdr;
bytea *tuple_data_bytea;
int tuple_data_len;
/* Extract information from the tuple header */
tuphdr = (HeapTupleHeader) PageGetItem(page, id);
2007-11-16 05:14:46 +08:00
values[4] = UInt32GetDatum(HeapTupleHeaderGetRawXmin(tuphdr));
Improve concurrency of foreign key locking This patch introduces two additional lock modes for tuples: "SELECT FOR KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each other, in contrast with already existing "SELECT FOR SHARE" and "SELECT FOR UPDATE". UPDATE commands that do not modify the values stored in the columns that are part of the key of the tuple now grab a SELECT FOR NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently with tuple locks of the FOR KEY SHARE variety. Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this means the concurrency improvement applies to them, which is the whole point of this patch. The added tuple lock semantics require some rejiggering of the multixact module, so that the locking level that each transaction is holding can be stored alongside its Xid. Also, multixacts now need to persist across server restarts and crashes, because they can now represent not only tuple locks, but also tuple updates. This means we need more careful tracking of lifetime of pg_multixact SLRU files; since they now persist longer, we require more infrastructure to figure out when they can be removed. pg_upgrade also needs to be careful to copy pg_multixact files over from the old server to the new, or at least part of multixact.c state, depending on the versions of the old and new servers. Tuple time qualification rules (HeapTupleSatisfies routines) need to be careful not to consider tuples with the "is multi" infomask bit set as being only locked; they might need to look up MultiXact values (i.e. possibly do pg_multixact I/O) to find out the Xid that updated a tuple, whereas they previously were assured to only use information readily available from the tuple header. This is considered acceptable, because the extra I/O would involve cases that would previously cause some commands to block waiting for concurrent transactions to finish. Another important change is the fact that locking tuples that have previously been updated causes the future versions to be marked as locked, too; this is essential for correctness of foreign key checks. This causes additional WAL-logging, also (there was previously a single WAL record for a locked tuple; now there are as many as updated copies of the tuple there exist.) With all this in place, contention related to tuples being checked by foreign key rules should be much reduced. As a bonus, the old behavior that a subtransaction grabbing a stronger tuple lock than the parent (sub)transaction held on a given tuple and later aborting caused the weaker lock to be lost, has been fixed. Many new spec files were added for isolation tester framework, to ensure overall behavior is sane. There's probably room for several more tests. There were several reviewers of this patch; in particular, Noah Misch and Andres Freund spent considerable time in it. Original idea for the patch came from Simon Riggs, after a problem report by Joel Jacobson. Most code is from me, with contributions from Marti Raudsepp, Alexander Shulgin, Noah Misch and Andres Freund. This patch was discussed in several pgsql-hackers threads; the most important start at the following message-ids: AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com 1290721684-sup-3951@alvh.no-ip.org 1294953201-sup-2099@alvh.no-ip.org 1320343602-sup-2290@alvh.no-ip.org 1339690386-sup-8927@alvh.no-ip.org 4FE5FF020200002500048A3D@gw.wicourts.gov 4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 23:04:59 +08:00
values[5] = UInt32GetDatum(HeapTupleHeaderGetRawXmax(tuphdr));
/* shared with xvac */
values[6] = UInt32GetDatum(HeapTupleHeaderGetRawCommandId(tuphdr));
values[7] = PointerGetDatum(&tuphdr->t_ctid);
values[8] = UInt32GetDatum(tuphdr->t_infomask2);
values[9] = UInt32GetDatum(tuphdr->t_infomask);
values[10] = UInt8GetDatum(tuphdr->t_hoff);
/* Copy raw tuple data into bytea attribute */
tuple_data_len = lp_len - tuphdr->t_hoff;
tuple_data_bytea = (bytea *) palloc(tuple_data_len + VARHDRSZ);
SET_VARSIZE(tuple_data_bytea, tuple_data_len + VARHDRSZ);
memcpy(VARDATA(tuple_data_bytea), (char *) tuphdr + tuphdr->t_hoff,
2016-06-10 06:02:36 +08:00
tuple_data_len);
values[13] = PointerGetDatum(tuple_data_bytea);
2007-11-16 05:14:46 +08:00
/*
* We already checked that the item is completely within the raw
* page passed to us, with the length given in the line pointer.
* Let's check that t_hoff doesn't point over lp_len, before using
* it to access t_bits and oid.
*/
if (tuphdr->t_hoff >= SizeofHeapTupleHeader &&
tuphdr->t_hoff <= lp_len &&
tuphdr->t_hoff == MAXALIGN(tuphdr->t_hoff))
{
if (tuphdr->t_infomask & HEAP_HASNULL)
{
int bits_len;
bits_len =
BITMAPLEN(HeapTupleHeaderGetNatts(tuphdr)) * BITS_PER_BYTE;
values[11] = CStringGetTextDatum(bits_to_text(tuphdr->t_bits, bits_len));
2007-11-16 05:14:46 +08:00
}
else
nulls[11] = true;
Remove WITH OIDS support, change oid catalog column visibility. Previously tables declared WITH OIDS, including a significant fraction of the catalog tables, stored the oid column not as a normal column, but as part of the tuple header. This special column was not shown by default, which was somewhat odd, as it's often (consider e.g. pg_class.oid) one of the more important parts of a row. Neither pg_dump nor COPY included the contents of the oid column by default. The fact that the oid column was not an ordinary column necessitated a significant amount of special case code to support oid columns. That already was painful for the existing, but upcoming work aiming to make table storage pluggable, would have required expanding and duplicating that "specialness" significantly. WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0). Remove it. Removing includes: - CREATE TABLE and ALTER TABLE syntax for declaring the table to be WITH OIDS has been removed (WITH (oids[ = true]) will error out) - pg_dump does not support dumping tables declared WITH OIDS and will issue a warning when dumping one (and ignore the oid column). - restoring an pg_dump archive with pg_restore will warn when restoring a table with oid contents (and ignore the oid column) - COPY will refuse to load binary dump that includes oids. - pg_upgrade will error out when encountering tables declared WITH OIDS, they have to be altered to remove the oid column first. - Functionality to access the oid of the last inserted row (like plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed. The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false) for CREATE TABLE) is still supported. While that requires a bit of support code, it seems unnecessary to break applications / dumps that do not use oids, and are explicit about not using them. The biggest user of WITH OID columns was postgres' catalog. This commit changes all 'magic' oid columns to be columns that are normally declared and stored. To reduce unnecessary query breakage all the newly added columns are still named 'oid', even if a table's column naming scheme would indicate 'reloid' or such. This obviously requires adapting a lot code, mostly replacing oid access via HeapTupleGetOid() with access to the underlying Form_pg_*->oid column. The bootstrap process now assigns oids for all oid columns in genbki.pl that do not have an explicit value (starting at the largest oid previously used), only oids assigned later by oids will be above FirstBootstrapObjectId. As the oid column now is a normal column the special bootstrap syntax for oids has been removed. Oids are not automatically assigned during insertion anymore, all backend code explicitly assigns oids with GetNewOidWithIndex(). For the rare case that insertions into the catalog via SQL are called for the new pg_nextoid() function can be used (which only works on catalog tables). The fact that oid columns on system tables are now normal columns means that they will be included in the set of columns expanded by * (i.e. SELECT * FROM pg_class will now include the table's oid, previously it did not). It'd not technically be hard to hide oid column by default, but that'd mean confusing behavior would either have to be carried forward forever, or it'd cause breakage down the line. While it's not unlikely that further adjustments are needed, the scope/invasiveness of the patch makes it worthwhile to get merge this now. It's painful to maintain externally, too complicated to commit after the code code freeze, and a dependency of a number of other patches. Catversion bump, for obvious reasons. Author: Andres Freund, with contributions by John Naylor Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
2018-11-21 07:36:57 +08:00
if (tuphdr->t_infomask & HEAP_HASOID_OLD)
values[12] = HeapTupleHeaderGetOidOld(tuphdr);
else
nulls[12] = true;
}
else
{
nulls[11] = true;
nulls[12] = true;
}
}
else
{
2007-11-16 05:14:46 +08:00
/*
* The line pointer is not used, or it's invalid. Set the rest of
* the fields to NULL
*/
int i;
for (i = 4; i <= 13; i++)
nulls[i] = true;
}
2007-11-16 05:14:46 +08:00
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
result = HeapTupleGetDatum(resultTuple);
inter_call_data->offset++;
SRF_RETURN_NEXT(fctx, result);
}
else
SRF_RETURN_DONE(fctx);
}
/*
* tuple_data_split_internal
*
* Split raw tuple data taken directly from a page into an array of bytea
* elements. This routine does a lookup on NULL values and creates array
2016-10-26 16:10:13 +08:00
* elements accordingly. This is a reimplementation of nocachegetattr()
* in heaptuple.c simplified for educational purposes.
*/
static Datum
tuple_data_split_internal(Oid relid, char *tupdata,
2016-06-10 06:02:36 +08:00
uint16 tupdata_len, uint16 t_infomask,
uint16 t_infomask2, bits8 *t_bits,
bool do_detoast)
{
2016-06-10 06:02:36 +08:00
ArrayBuildState *raw_attrs;
int nattrs;
int i;
int off = 0;
Relation rel;
TupleDesc tupdesc;
/* Get tuple descriptor from relation OID */
rel = relation_open(relid, AccessShareLock);
tupdesc = RelationGetDescr(rel);
raw_attrs = initArrayResult(BYTEAOID, CurrentMemoryContext, false);
nattrs = tupdesc->natts;
if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("only heap AM is supported")));
if (nattrs < (t_infomask2 & HEAP_NATTS_MASK))
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg("number of attributes in tuple header is greater than number of attributes in tuple descriptor")));
for (i = 0; i < nattrs; i++)
{
2016-06-10 06:02:36 +08:00
Form_pg_attribute attr;
bool is_null;
bytea *attr_data = NULL;
attr = TupleDescAttr(tupdesc, i);
/*
2016-06-10 06:02:36 +08:00
* Tuple header can specify less attributes than tuple descriptor as
* ALTER TABLE ADD COLUMN without DEFAULT keyword does not actually
* change tuples in pages, so attributes with numbers greater than
* (t_infomask2 & HEAP_NATTS_MASK) should be treated as NULL.
*/
if (i >= (t_infomask2 & HEAP_NATTS_MASK))
is_null = true;
else
is_null = (t_infomask & HEAP_HASNULL) && att_isnull(i, t_bits);
if (!is_null)
{
2016-06-10 06:02:36 +08:00
int len;
if (attr->attlen == -1)
{
off = att_align_pointer(off, attr->attalign, -1,
tupdata + off);
2016-06-10 06:02:36 +08:00
/*
* As VARSIZE_ANY throws an exception if it can't properly
* detect the type of external storage in macros VARTAG_SIZE,
* this check is repeated to have a nicer error handling.
*/
if (VARATT_IS_EXTERNAL(tupdata + off) &&
!VARATT_IS_EXTERNAL_ONDISK(tupdata + off) &&
!VARATT_IS_EXTERNAL_INDIRECT(tupdata + off))
ereport(ERROR,
2016-06-10 06:02:36 +08:00
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg("first byte of varlena attribute is incorrect for attribute %d", i)));
len = VARSIZE_ANY(tupdata + off);
}
else
{
off = att_align_nominal(off, attr->attalign);
len = attr->attlen;
}
if (tupdata_len < off + len)
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg("unexpected end of tuple data")));
if (attr->attlen == -1 && do_detoast)
attr_data = DatumGetByteaPCopy(tupdata + off);
else
{
attr_data = (bytea *) palloc(len + VARHDRSZ);
SET_VARSIZE(attr_data, len + VARHDRSZ);
memcpy(VARDATA(attr_data), tupdata + off, len);
}
off = att_addlength_pointer(off, attr->attlen,
tupdata + off);
}
raw_attrs = accumArrayResult(raw_attrs, PointerGetDatum(attr_data),
is_null, BYTEAOID, CurrentMemoryContext);
if (attr_data)
pfree(attr_data);
}
if (tupdata_len != off)
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
errmsg("end of tuple reached without looking at all its data")));
relation_close(rel, AccessShareLock);
return makeArrayResult(raw_attrs, CurrentMemoryContext);
}
/*
* tuple_data_split
*
* Split raw tuple data taken directly from page into distinct elements
* taking into account null values.
*/
PG_FUNCTION_INFO_V1(tuple_data_split);
Datum
tuple_data_split(PG_FUNCTION_ARGS)
{
2016-06-10 06:02:36 +08:00
Oid relid;
bytea *raw_data;
uint16 t_infomask;
uint16 t_infomask2;
char *t_bits_str;
bool do_detoast = false;
bits8 *t_bits = NULL;
Datum res;
relid = PG_GETARG_OID(0);
raw_data = PG_ARGISNULL(1) ? NULL : PG_GETARG_BYTEA_P(1);
t_infomask = PG_GETARG_INT16(2);
t_infomask2 = PG_GETARG_INT16(3);
t_bits_str = PG_ARGISNULL(4) ? NULL :
text_to_cstring(PG_GETARG_TEXT_PP(4));
if (PG_NARGS() >= 6)
do_detoast = PG_GETARG_BOOL(5);
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser to use raw page functions")));
if (!raw_data)
PG_RETURN_NULL();
/*
* Convert t_bits string back to the bits8 array as represented in the
* tuple header.
*/
if (t_infomask & HEAP_HASNULL)
{
2016-06-10 06:02:36 +08:00
int bits_str_len;
int bits_len;
bits_len = BITMAPLEN(t_infomask2 & HEAP_NATTS_MASK) * BITS_PER_BYTE;
if (!t_bits_str)
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
Avoid using %c printf format for potentially non-ASCII characters. Since %c only passes a C "char" to printf, it's incapable of dealing with multibyte characters. Passing just the first byte of such a character leads to an output string that is visibly not correctly encoded, resulting in undesirable behavior such as encoding conversion failures while sending error messages to clients. We've lived with this issue for a long time because it was inconvenient to avoid in a portable fashion. However, now that we always use our own snprintf code, it's reasonable to use the %.*s format to print just one possibly-multibyte character in a string. (We previously avoided that obvious-looking answer in order to work around glibc's bug #6530, cf commits 54cd4f045 and ed437e2b2.) Hence, run around and fix a bunch of places that used %c to report a character found in a user-supplied string. For simplicity, I did not touch places that were emitting non-user-facing debug messages, or reporting catalog data that should always be ASCII. (It's also unclear how useful this approach could be in frontend code, where it's less certain that we know what encoding we're dealing with.) In passing, improve a couple of poorly-written error messages in pageinspect/heapfuncs.c. This is a longstanding issue, but I'm hesitant to back-patch because of the impact on translatable message strings. In any case this fix would not work reliably before v12. Tom Lane and Quan Zongliang Discussion: https://postgr.es/m/a120087c-4c88-d9d4-1ec5-808d7a7f133d@gmail.com
2020-06-29 23:41:19 +08:00
errmsg("t_bits string must not be NULL")));
bits_str_len = strlen(t_bits_str);
if (bits_len != bits_str_len)
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
Avoid using %c printf format for potentially non-ASCII characters. Since %c only passes a C "char" to printf, it's incapable of dealing with multibyte characters. Passing just the first byte of such a character leads to an output string that is visibly not correctly encoded, resulting in undesirable behavior such as encoding conversion failures while sending error messages to clients. We've lived with this issue for a long time because it was inconvenient to avoid in a portable fashion. However, now that we always use our own snprintf code, it's reasonable to use the %.*s format to print just one possibly-multibyte character in a string. (We previously avoided that obvious-looking answer in order to work around glibc's bug #6530, cf commits 54cd4f045 and ed437e2b2.) Hence, run around and fix a bunch of places that used %c to report a character found in a user-supplied string. For simplicity, I did not touch places that were emitting non-user-facing debug messages, or reporting catalog data that should always be ASCII. (It's also unclear how useful this approach could be in frontend code, where it's less certain that we know what encoding we're dealing with.) In passing, improve a couple of poorly-written error messages in pageinspect/heapfuncs.c. This is a longstanding issue, but I'm hesitant to back-patch because of the impact on translatable message strings. In any case this fix would not work reliably before v12. Tom Lane and Quan Zongliang Discussion: https://postgr.es/m/a120087c-4c88-d9d4-1ec5-808d7a7f133d@gmail.com
2020-06-29 23:41:19 +08:00
errmsg("unexpected length of t_bits string: %u, expected %u",
bits_str_len, bits_len)));
/* do the conversion */
t_bits = text_to_bits(t_bits_str, bits_str_len);
}
else
{
if (t_bits_str)
ereport(ERROR,
(errcode(ERRCODE_DATA_CORRUPTED),
Avoid using %c printf format for potentially non-ASCII characters. Since %c only passes a C "char" to printf, it's incapable of dealing with multibyte characters. Passing just the first byte of such a character leads to an output string that is visibly not correctly encoded, resulting in undesirable behavior such as encoding conversion failures while sending error messages to clients. We've lived with this issue for a long time because it was inconvenient to avoid in a portable fashion. However, now that we always use our own snprintf code, it's reasonable to use the %.*s format to print just one possibly-multibyte character in a string. (We previously avoided that obvious-looking answer in order to work around glibc's bug #6530, cf commits 54cd4f045 and ed437e2b2.) Hence, run around and fix a bunch of places that used %c to report a character found in a user-supplied string. For simplicity, I did not touch places that were emitting non-user-facing debug messages, or reporting catalog data that should always be ASCII. (It's also unclear how useful this approach could be in frontend code, where it's less certain that we know what encoding we're dealing with.) In passing, improve a couple of poorly-written error messages in pageinspect/heapfuncs.c. This is a longstanding issue, but I'm hesitant to back-patch because of the impact on translatable message strings. In any case this fix would not work reliably before v12. Tom Lane and Quan Zongliang Discussion: https://postgr.es/m/a120087c-4c88-d9d4-1ec5-808d7a7f133d@gmail.com
2020-06-29 23:41:19 +08:00
errmsg("t_bits string is expected to be NULL, but instead it is %zu bytes long",
strlen(t_bits_str))));
}
/* Split tuple data */
res = tuple_data_split_internal(relid, (char *) raw_data + VARHDRSZ,
VARSIZE(raw_data) - VARHDRSZ,
t_infomask, t_infomask2, t_bits,
do_detoast);
if (t_bits)
pfree(t_bits);
PG_RETURN_ARRAYTYPE_P(res);
}
/*
* heap_tuple_infomask_flags
*
* Decode into a human-readable format t_infomask and t_infomask2 associated
* to a tuple. All the flags are described in access/htup_details.h.
*/
PG_FUNCTION_INFO_V1(heap_tuple_infomask_flags);
Datum
heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
{
#define HEAP_TUPLE_INFOMASK_COLS 2
Datum values[HEAP_TUPLE_INFOMASK_COLS];
bool nulls[HEAP_TUPLE_INFOMASK_COLS];
uint16 t_infomask = PG_GETARG_INT16(0);
uint16 t_infomask2 = PG_GETARG_INT16(1);
int cnt = 0;
ArrayType *a;
int bitcnt;
Datum *flags;
TupleDesc tupdesc;
HeapTuple tuple;
if (!superuser())
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser to use raw page functions")));
/* Build a tuple descriptor for our result type */
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
elog(ERROR, "return type must be a row type");
bitcnt = pg_popcount((const char *) &t_infomask, sizeof(uint16)) +
pg_popcount((const char *) &t_infomask2, sizeof(uint16));
/* Initialize values and NULL flags arrays */
MemSet(values, 0, sizeof(values));
MemSet(nulls, 0, sizeof(nulls));
/* If no flags, return a set of empty arrays */
if (bitcnt <= 0)
{
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
}
/* build set of raw flags */
flags = (Datum *) palloc0(sizeof(Datum) * bitcnt);
/* decode t_infomask */
if ((t_infomask & HEAP_HASNULL) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_HASNULL");
if ((t_infomask & HEAP_HASVARWIDTH) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_HASVARWIDTH");
if ((t_infomask & HEAP_HASEXTERNAL) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_HASEXTERNAL");
if ((t_infomask & HEAP_HASOID_OLD) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_HASOID_OLD");
if ((t_infomask & HEAP_XMAX_KEYSHR_LOCK) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_XMAX_KEYSHR_LOCK");
if ((t_infomask & HEAP_COMBOCID) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_COMBOCID");
if ((t_infomask & HEAP_XMAX_EXCL_LOCK) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_XMAX_EXCL_LOCK");
if ((t_infomask & HEAP_XMAX_LOCK_ONLY) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_XMAX_LOCK_ONLY");
if ((t_infomask & HEAP_XMIN_COMMITTED) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_XMIN_COMMITTED");
if ((t_infomask & HEAP_XMIN_INVALID) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_XMIN_INVALID");
if ((t_infomask & HEAP_XMAX_COMMITTED) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_XMAX_COMMITTED");
if ((t_infomask & HEAP_XMAX_INVALID) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_XMAX_INVALID");
if ((t_infomask & HEAP_XMAX_IS_MULTI) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_XMAX_IS_MULTI");
if ((t_infomask & HEAP_UPDATED) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_UPDATED");
if ((t_infomask & HEAP_MOVED_OFF) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_MOVED_OFF");
if ((t_infomask & HEAP_MOVED_IN) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_MOVED_IN");
/* decode t_infomask2 */
if ((t_infomask2 & HEAP_KEYS_UPDATED) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_KEYS_UPDATED");
if ((t_infomask2 & HEAP_HOT_UPDATED) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_HOT_UPDATED");
if ((t_infomask2 & HEAP_ONLY_TUPLE) != 0)
flags[cnt++] = CStringGetTextDatum("HEAP_ONLY_TUPLE");
/* build value */
Assert(cnt <= bitcnt);
a = construct_array(flags, cnt, TEXTOID, -1, false, TYPALIGN_INT);
values[0] = PointerGetDatum(a);
/*
* Build set of combined flags. Use the same array as previously, this
* keeps the code simple.
*/
cnt = 0;
MemSet(flags, 0, sizeof(Datum) * bitcnt);
/* decode combined masks of t_infomask */
if ((t_infomask & HEAP_XMAX_SHR_LOCK) == HEAP_XMAX_SHR_LOCK)
flags[cnt++] = CStringGetTextDatum("HEAP_XMAX_SHR_LOCK");
if ((t_infomask & HEAP_XMIN_FROZEN) == HEAP_XMIN_FROZEN)
flags[cnt++] = CStringGetTextDatum("HEAP_XMIN_FROZEN");
if ((t_infomask & HEAP_MOVED) == HEAP_MOVED)
flags[cnt++] = CStringGetTextDatum("HEAP_MOVED");
/* Build an empty array if there are no combined flags */
if (cnt == 0)
a = construct_empty_array(TEXTOID);
else
a = construct_array(flags, cnt, TEXTOID, -1, false, TYPALIGN_INT);
pfree(flags);
values[1] = PointerGetDatum(a);
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
}