mirror/postgresql
2
0
Fork 0
mirror of https://git.postgresql.org/git/postgresql.git synced 2024-12-15 08:20:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
014f9f34d2
postgresql/contrib/intarray/_int_gist.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

638 lines
15 KiB
C
Raw Normal View History

Add $PostgreSQL$ markers to a lot of files that were missing them. This particular batch was just for *.c and *.h file. The changes were made with the following 2 commands: find . \( \( -name 'libstemmer' -o -name 'expected' -o -name 'ppport.h' \) -prune \) -o \( -name '*.[ch]' \) \( -exec grep -q '\$PostgreSQL' {} \; -o -print \) | while read file ; do head -n 1 < $file | grep -q '^/\*' && echo $file; done | xargs -l sed -i -e '1s/^\// /' -e '1i/*\n * $PostgreSQL:$ \n *' find . \( \( -name 'libstemmer' -o -name 'expected' -o -name 'ppport.h' \) -prune \) -o \( -name '*.[ch]' \) \( -exec grep -q '\$PostgreSQL' {} \; -o -print \) | xargs -l sed -i -e '1i/*\n * $PostgreSQL:$ \n */'
2008-05-17 09:28:26 +08:00
/*
Remove cvs keywords from all files.
2010-09-21 04:08:53 +08:00
* contrib/intarray/_int_gist.c
Add $PostgreSQL$ markers to a lot of files that were missing them. This particular batch was just for *.c and *.h file. The changes were made with the following 2 commands: find . \( \( -name 'libstemmer' -o -name 'expected' -o -name 'ppport.h' \) -prune \) -o \( -name '*.[ch]' \) \( -exec grep -q '\$PostgreSQL' {} \; -o -print \) | while read file ; do head -n 1 < $file | grep -q '^/\*' && echo $file; done | xargs -l sed -i -e '1s/^\// /' -e '1i/*\n * $PostgreSQL:$ \n *' find . \( \( -name 'libstemmer' -o -name 'expected' -o -name 'ppport.h' \) -prune \) -o \( -name '*.[ch]' \) \( -exec grep -q '\$PostgreSQL' {} \; -o -print \) | xargs -l sed -i -e '1i/*\n * $PostgreSQL:$ \n */'
2008-05-17 09:28:26 +08:00
*/
Restructure some header files a bit, in particular heapam.h, by removing some unnecessary #include lines in it. Also, move some tuple routine prototypes and macros to htup.h, which allows removal of heapam.h inclusion from some .c files. For this to work, a new header file access/sysattr.h needed to be created, initially containing attribute numbers of system columns, for pg_dump usage. While at it, make contrib ltree, intarray and hstore header files more consistent with our header style.
2008-05-12 08:00:54 +08:00
#include "postgres.h"
Centralize definition of integer limits. Several submitted and even committed patches have run into the problem that C89, our baseline, does not provide minimum/maximum values for various integer datatypes. C99's stdint.h does, but we can't rely on it. Several parts of the code defined limits locally, so instead centralize the definitions to c.h. This patch also changes the more obvious usages of literal limit values; there's more places that could be changed, but it's less clear whether it's beneficial to change those. Author: Andrew Gierth Discussion: 87619tc5wc.fsf@news-spur.riddles.org.uk
2015-03-26 05:39:42 +08:00
#include <limits.h>
Remove unnecessary uses of Abs() Use C standard abs() or fabs() instead. Reviewed-by: Zhang Mingli <zmlpostgres@gmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/4beb42b5-216b-bce8-d452-d924d5794c63%40enterprisedb.com
2022-10-07 19:28:38 +08:00
#include <math.h>
Centralize definition of integer limits. Several submitted and even committed patches have run into the problem that C89, our baseline, does not provide minimum/maximum values for various integer datatypes. C99's stdint.h does, but we can't rely on it. Several parts of the code defined limits locally, so instead centralize the definitions to c.h. This patch also changes the more obvious usages of literal limit values; there's more places that could be changed, but it's less clear whether it's beneficial to change those. Author: Andrew Gierth Discussion: 87619tc5wc.fsf@news-spur.riddles.org.uk
2015-03-26 05:39:42 +08:00
Make the order of the header file includes consistent in contrib modules. The basic rule we follow here is to always first include 'postgres.h' or 'postgres_fe.h' whichever is applicable, then system header includes and then Postgres header includes.  In this, we also follow that all the Postgres header includes are in order based on their ASCII value.  We generally follow these rules, but the code has deviated in many places. This commit makes it consistent just for contrib modules. The later commits will enforce similar rules in other parts of code. Author: Vignesh C Reviewed-by: Amit Kapila Discussion: https://postgr.es/m/CALDaNm2Sznv8RR6Ex-iJO6xAdsxgWhCoETkaYX=+9DW3q0QCfA@mail.gmail.com
2019-10-23 11:56:22 +08:00
#include "_int.h"
Restructure some header files a bit, in particular heapam.h, by removing some unnecessary #include lines in it. Also, move some tuple routine prototypes and macros to htup.h, which allows removal of heapam.h inclusion from some .c files. For this to work, a new header file access/sysattr.h needed to be created, initially containing attribute numbers of system columns, for pg_dump usage. While at it, make contrib ltree, intarray and hstore header files more consistent with our header style.
2008-05-12 08:00:54 +08:00
#include "access/gist.h"
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
#include "access/reloptions.h"
Move strategy numbers to include/access/stratnum.h For upcoming BRIN opclasses, it's convenient to have strategy numbers defined in a single place. Since there's nothing appropriate, create it. The StrategyNumber typedef now lives there, as well as existing strategy numbers for B-trees (from skey.h) and R-tree-and-friends (from gist.h). skey.h is forced to include stratnum.h because of the StrategyNumber typedef, but gist.h is not; extensions that currently rely on gist.h for rtree strategy numbers might need to add a new A few .c files can stop including skey.h and/or gist.h, which is a nice side benefit. Per discussion: https://www.postgresql.org/message-id/20150514232132.GZ2523@alvh.no-ip.org Authored by Emre Hasegeli and Álvaro. (It's not clear to me why bootscanner.l has any #include lines at all.)
2015-05-16 04:03:16 +08:00
#include "access/stratnum.h"
Restructure some header files a bit, in particular heapam.h, by removing some unnecessary #include lines in it. Also, move some tuple routine prototypes and macros to htup.h, which allows removal of heapam.h inclusion from some .c files. For this to work, a new header file access/sysattr.h needed to be created, initially containing attribute numbers of system columns, for pg_dump usage. While at it, make contrib ltree, intarray and hstore header files more consistent with our header style.
2008-05-12 08:00:54 +08:00
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
#define GETENTRY(vec,pos) ((ArrayType *) DatumGetPointer((vec)->vector[(pos)].key))
Add missing intarray files.
2003-06-12 03:31:05 +08:00
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
/*
* Control the maximum sparseness of compressed keys.
*
* The upper safe bound for this limit is half the maximum allocatable array
* size. A lower bound would give more guarantees that pathological data
* wouldn't eat excessive CPU and memory, but at the expense of breaking
* possibly working (after a fashion) indexes.
*/
#define MAXNUMELTS (Min((MaxAllocSize / sizeof(Datum)),((MaxAllocSize - ARR_OVERHEAD_NONULLS(1)) / sizeof(int)))/2)
/* or: #define MAXNUMELTS 1000000 */
Add missing intarray files.
2003-06-12 03:31:05 +08:00
/*
** GiST support methods
*/
PG_FUNCTION_INFO_V1(g_int_consistent);
PG_FUNCTION_INFO_V1(g_int_compress);
PG_FUNCTION_INFO_V1(g_int_decompress);
PG_FUNCTION_INFO_V1(g_int_penalty);
PG_FUNCTION_INFO_V1(g_int_picksplit);
PG_FUNCTION_INFO_V1(g_int_union);
PG_FUNCTION_INFO_V1(g_int_same);
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
PG_FUNCTION_INFO_V1(g_int_options);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
/*
** The GiST Consistent method for _intments
** Should return false if for all data items x below entry,
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
** the predicate x op query == false, where op is the oper
Add missing intarray files.
2003-06-12 03:31:05 +08:00
** corresponding to strategy in the pg_amop table.
*/
Datum
g_int_consistent(PG_FUNCTION_ARGS)
{
GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
ArrayType *query = PG_GETARG_ARRAYTYPE_P_COPY(1);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
StrategyNumber strategy = (StrategyNumber) PG_GETARG_UINT16(2);
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 22:49:15 +08:00
Push index operator lossiness determination down to GIST/GIN opclass "consistent" functions, and remove pg_amop.opreqcheck, as per recent discussion. The main immediate benefit of this is that we no longer need 8.3's ugly hack of requiring @@@ rather than @@ to test weight-using tsquery searches on GIN indexes. In future it should be possible to optimize some other queries better than is done now, by detecting at runtime whether the index match is exact or not. Tom Lane, after an idea of Heikki's, and with some help from Teodor.
2008-04-15 01:05:34 +08:00
/* Oid subtype = PG_GETARG_OID(3); */
bool *recheck = (bool *) PG_GETARG_POINTER(4);
Convert *GetDatum() and DatumGet*() macros to inline functions The previous macro implementations just cast the argument to a target type but did not check whether the input type was appropriate. The function implementation can do better type checking of the input type. For the *GetDatumFast() macros, converting to an inline function doesn't work in the !USE_FLOAT8_BYVAL case, but we can use AssertVariableIsOfTypeMacro() to get a similar level of type checking. Reviewed-by: Aleksander Alekseev <aleksander@timescale.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/8528fb7e-0aa2-6b54-85fb-0c0886dbd6ed%40enterprisedb.com
2022-09-28 02:47:07 +08:00
bool retval = false; /* silence compiler warning */
Add missing intarray files.
2003-06-12 03:31:05 +08:00
Push index operator lossiness determination down to GIST/GIN opclass "consistent" functions, and remove pg_amop.opreqcheck, as per recent discussion. The main immediate benefit of this is that we no longer need 8.3's ugly hack of requiring @@@ rather than @@ to test weight-using tsquery searches on GIN indexes. In future it should be possible to optimize some other queries better than is done now, by detecting at runtime whether the index match is exact or not. Tom Lane, after an idea of Heikki's, and with some help from Teodor.
2008-04-15 01:05:34 +08:00
/* this is exact except for RTSameStrategyNumber */
*recheck = (strategy == RTSameStrategyNumber);
Detoast query in g_intbig_consistent and copy query in g_int_consistent
2006-04-03 16:21:05 +08:00
if (strategy == BooleanSearchStrategy)
{
retval = execconsistent((QUERYTYPE *) query,
Add missing intarray files.
2003-06-12 03:31:05 +08:00
(ArrayType *) DatumGetPointer(entry->key),
Detoast query in g_intbig_consistent and copy query in g_int_consistent
2006-04-03 16:21:05 +08:00
GIST_LEAF(entry));
pfree(query);
PG_RETURN_BOOL(retval);
}
Add missing intarray files.
2003-06-12 03:31:05 +08:00
/* sort query for fast search, key is already sorted */
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
CHECKARRVALID(query);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PREPAREARR(query);
switch (strategy)
{
case RTOverlapStrategyNumber:
retval = inner_int_overlap((ArrayType *) DatumGetPointer(entry->key),
query);
break;
case RTSameStrategyNumber:
if (GIST_LEAF(entry))
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
DirectFunctionCall3(g_int_same,
Add missing intarray files.
2003-06-12 03:31:05 +08:00
entry->key,
PointerGetDatum(query),
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
PointerGetDatum(&retval));
Add missing intarray files.
2003-06-12 03:31:05 +08:00
else
retval = inner_int_contains((ArrayType *) DatumGetPointer(entry->key),
query);
break;
case RTContainsStrategyNumber:
Rename contrib contains/contained-by operators to @> and <@, per discussion.
2006-09-11 01:36:52 +08:00
case RTOldContainsStrategyNumber:
Add missing intarray files.
2003-06-12 03:31:05 +08:00
retval = inner_int_contains((ArrayType *) DatumGetPointer(entry->key),
query);
break;
case RTContainedByStrategyNumber:
Rename contrib contains/contained-by operators to @> and <@, per discussion.
2006-09-11 01:36:52 +08:00
case RTOldContainedByStrategyNumber:
Remove <@ from contrib/intarray's GiST operator classes. Since commit efc77cf5f, an indexed query using <@ has required a full-index scan, so that it actually performs worse than a plain seqscan would do. As I noted at the time, we'd be better off to not treat <@ as being indexable by such indexes at all; and that's what this patch does. It would have been difficult to remove these opclass members without dropping the whole opclass before commit 9f9682783 fixed GiST opclass member dependency rules, but now it's quite simple, so let's do it. I left the existing support code in place for the time being, with comments noting it's now unreachable. At some point, perhaps we should remove that code in favor of throwing an error telling people to upgrade the extension version. Discussion: https://postgr.es/m/2176979.1596389859@sss.pgh.pa.us Discussion: https://postgr.es/m/458.1565114141@sss.pgh.pa.us
2020-08-09 05:26:29 +08:00
/*
* This code is unreachable as of intarray 1.4, because the <@
* operator has been removed from the opclass. We keep it for now
* to support older versions of the SQL definitions.
*/
Add missing intarray files.
2003-06-12 03:31:05 +08:00
if (GIST_LEAF(entry))
retval = inner_int_contains(query,
(ArrayType *) DatumGetPointer(entry->key));
else
Fix intarray's GiST opclasses to not fail for empty arrays with <@. contrib/intarray considers "arraycol <@ constant-array" to be indexable, but its GiST opclass code fails to reliably find index entries for empty array values (which of course should trivially match such queries). This is because the test condition to see whether we should descend through a non-leaf node is wrong. Unfortunately, empty array entries could be anywhere in the index, as these index opclasses are currently designed. So there's no way to fix this except by lobotomizing <@ indexscans to scan the whole index ... which is what this patch does. That's pretty unfortunate: the performance is now actually worse than a seqscan, in most cases. We'd be better off to remove <@ from the GiST opclasses entirely, and perhaps a future non-back-patchable patch will do so. In the meantime, applications whose performance is adversely impacted have a couple of options. They could switch to a GIN index, which doesn't have this bug, or they could replace "arraycol <@ constant-array" with "arraycol <@ constant-array AND arraycol && constant-array". That will provide about the same performance as before, and it will find all non-empty subsets of the given constant-array, which is all that could reliably be expected of the query before. While at it, add some more regression test cases to improve code coverage of contrib/intarray. In passing, adjust resize_intArrayType so that when it's returning an empty array, it uses construct_empty_array for that rather than cowboy hacking on the input array. While the hack produces an array that looks valid for most purposes, it isn't bitwise equal to empty arrays produced by other code paths, which could have subtle odd effects. I don't think this code path is performance-critical enough to justify such shortcuts. (Back-patch this part only as far as v11; before commit 01783ac36 we were not careful about this in other intarray code paths either.) Back-patch the <@ fixes to all supported versions, since this was broken from day one. Patch by me; thanks to Alexander Korotkov for review. Discussion: https://postgr.es/m/458.1565114141@sss.pgh.pa.us
2019-08-07 06:04:51 +08:00
{
/*
* Unfortunately, because empty arrays could be anywhere in
* the index, we must search the whole tree.
*/
retval = true;
}
Add missing intarray files.
2003-06-12 03:31:05 +08:00
break;
default:
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
retval = false;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
}
Detoast query in g_intbig_consistent and copy query in g_int_consistent
2006-04-03 16:21:05 +08:00
pfree(query);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PG_RETURN_BOOL(retval);
}
Datum
g_int_union(PG_FUNCTION_ARGS)
{
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
GistEntryVector *entryvec = (GistEntryVector *) PG_GETARG_POINTER(0);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
int *size = (int *) PG_GETARG_POINTER(1);
Replace int2/int4 in C code with int16/int32 The latter was already the dominant use, and it's preferable because in C the convention is that intXX means XX bits. Therefore, allowing mixed use of int2, int4, int8, int16, int32 is obviously confusing. Remove the typedefs for int2 and int4 for now. They don't seem to be widely used outside of the PostgreSQL source tree, and the few uses can probably be cleaned up by the time this ships.
2012-06-25 06:51:46 +08:00
int32 i,
Add missing intarray files.
2003-06-12 03:31:05 +08:00
*ptr;
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
ArrayType *res;
int totlen = 0;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
for (i = 0; i < entryvec->n; i++)
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
{
ArrayType *ent = GETENTRY(entryvec, i);
CHECKARRVALID(ent);
totlen += ARRNELEMS(ent);
}
Add missing intarray files.
2003-06-12 03:31:05 +08:00
res = new_intArrayType(totlen);
ptr = ARRPTR(res);
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
for (i = 0; i < entryvec->n; i++)
Add missing intarray files.
2003-06-12 03:31:05 +08:00
{
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
ArrayType *ent = GETENTRY(entryvec, i);
int nel;
nel = ARRNELEMS(ent);
Replace int2/int4 in C code with int16/int32 The latter was already the dominant use, and it's preferable because in C the convention is that intXX means XX bits. Therefore, allowing mixed use of int2, int4, int8, int16, int32 is obviously confusing. Remove the typedefs for int2 and int4 for now. They don't seem to be widely used outside of the PostgreSQL source tree, and the few uses can probably be cleaned up by the time this ships.
2012-06-25 06:51:46 +08:00
memcpy(ptr, ARRPTR(ent), nel * sizeof(int32));
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
ptr += nel;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
}
QSORT(res, 1);
res = _int_unique(res);
*size = VARSIZE(res);
PG_RETURN_POINTER(res);
}
/*
** GiST Compress and Decompress methods
*/
Datum
g_int_compress(PG_FUNCTION_ARGS)
{
GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
GISTENTRY *retval;
ArrayType *r;
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
int num_ranges = G_INT_GET_NUMRANGES();
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
int len,
lenr;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
int *dr;
int i,
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
j,
Add missing intarray files.
2003-06-12 03:31:05 +08:00
cand;
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
int64 min;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
if (entry->leafkey)
{
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
r = DatumGetArrayTypePCopy(entry->key);
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
CHECKARRVALID(r);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PREPAREARR(r);
Remove usage of ArrayType->flags field, use pgsql's macros BITS_PER_BYTE instead of self-defined macros, add limit of Array to gist__int_ops. BTW, intarray now doesn't support NULLs in arrays.
2005-11-15 00:11:37 +08:00
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
if (ARRNELEMS(r) >= 2 * num_ranges)
intarray: Prevent out-of-bound memory reads with gist__int_ops As gist__int_ops stands in intarray, it is possible to store GiST entries for leaf pages that can cause corruptions when decompressed. Leaf nodes are stored as decompressed all the time by the compression method, and the decompression method should map with that, retrieving the contents of the page without doing any decompression. However, the code authorized the insertion of leaf page data with a higher number of array items than what can be supported, generating a NOTICE message to inform about this matter (199 for a 8k page, for reference). When calling the decompression method, a decompression would be attempted on this leaf node item but the contents should be retrieved as they are. The NOTICE message generated when dealing with the compression of a leaf page and too many elements in the input array for gist__int_ops has been introduced by 08ee64e, removing the marker stored in the array to track if this is actually a leaf node. However, it also missed the fact that the decompression path should do nothing for a leaf page. Hence, as the code stand, a too-large array would be stored as uncompressed but the decompression path would attempt a decompression rather that retrieving the contents as they are. This leads to various problems. First, even if 08ee64e tried to address that, it is possible to do out-of-bound chunk writes with a large input array, with the backend informing about that with WARNINGs. On decompression, retrieving the stored leaf data would lead to incorrect memory reads, leading to crashes or even worse. Perhaps somebody would be interested in expanding the number of array items that can be handled in a leaf page for this operator in the future, which would require revisiting the choice done in 08ee64e, but based on the lack of reports about this problem since 2005 it does not look so. For now, this commit prevents the insertion of data for leaf pages when using more array items that the code can handle on decompression, switching the NOTICE message to an ERROR. If one wishes to use more array items, gist__intbig_ops is an optional choice. While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a limit is reached, because that's what the module is facing in such cases. Author: Ankit Kumar Pandey, Alexander Lakhin Reviewed-by: Richard Guo, Michael Paquier Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org Backpatch-through: 11
2023-06-15 12:45:34 +08:00
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("input array is too big (%d maximum allowed, %d current), use gist__intbig_ops opclass instead",
2 * num_ranges - 1, ARRNELEMS(r))));
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-23 02:17:34 +08:00
Add missing intarray files.
2003-06-12 03:31:05 +08:00
retval = palloc(sizeof(GISTENTRY));
gistentryinit(*retval, PointerGetDatum(r),
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
entry->rel, entry->page, entry->offset, false);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PG_RETURN_POINTER(retval);
}
Remove usage of ArrayType->flags field, use pgsql's macros BITS_PER_BYTE instead of self-defined macros, add limit of Array to gist__int_ops. BTW, intarray now doesn't support NULLs in arrays.
2005-11-15 00:11:37 +08:00
/*
* leaf entries never compress one more time, only when entry->leafkey
* ==true, so now we work only with internal keys
*/
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
r = DatumGetArrayTypeP(entry->key);
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
CHECKARRVALID(r);
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
if (ARRISEMPTY(r))
Add missing intarray files.
2003-06-12 03:31:05 +08:00
{
if (r != (ArrayType *) DatumGetPointer(entry->key))
pfree(r);
PG_RETURN_POINTER(entry);
}
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
if ((len = ARRNELEMS(r)) >= 2 * num_ranges)
Add missing intarray files.
2003-06-12 03:31:05 +08:00
{ /* compress */
if (r == (ArrayType *) DatumGetPointer(entry->key))
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
r = DatumGetArrayTypePCopy(entry->key);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
r = resize_intArrayType(r, 2 * (len));
dr = ARRPTR(r);
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
/*
* "len" at this point is the number of ranges we will construct.
* "lenr" is the number of ranges we must eventually remove by
* merging, we must be careful to remove no more than this number.
*/
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
lenr = len - num_ranges;
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
/*
* Initially assume we can merge consecutive ints into a range. but we
* must count every value removed and stop when lenr runs out
*/
for (j = i = len - 1; i > 0 && lenr > 0; i--, j--)
{
int r_end = dr[i];
int r_start = r_end;
Initial pgindent run for v12. This is still using the 2.0 version of pg_bsd_indent. I thought it would be good to commit this separately, so as to document the differences between 2.0 and 2.1 behavior. Discussion: https://postgr.es/m/16296.1558103386@sss.pgh.pa.us
2019-05-23 00:55:34 +08:00
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
while (i > 0 && lenr > 0 && dr[i - 1] == r_start - 1)
--r_start, --i, --lenr;
dr[2 * j] = r_start;
dr[2 * j + 1] = r_end;
}
/* just copy the rest, if any, as trivial ranges */
for (; i >= 0; i--, j--)
dr[2 * j] = dr[2 * j + 1] = dr[i];
Add missing intarray files.
2003-06-12 03:31:05 +08:00
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
if (++j)
{
/*
* shunt everything down to start at the right place
*/
Remove useless casts to (void *) in arguments of some system functions The affected functions are: bsearch, memcmp, memcpy, memset, memmove, qsort, repalloc Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/fd9adf5d-b1aa-e82f-e4c7-263c30145807%40enterprisedb.com
2023-02-07 13:53:05 +08:00
memmove(&dr[0], &dr[2 * j], 2 * (len - j) * sizeof(int32));
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
}
Initial pgindent run for v12. This is still using the 2.0 version of pg_bsd_indent. I thought it would be good to commit this separately, so as to document the differences between 2.0 and 2.1 behavior. Discussion: https://postgr.es/m/16296.1558103386@sss.pgh.pa.us
2019-05-23 00:55:34 +08:00
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
/*
* make "len" be number of array elements, not ranges
*/
len = 2 * (len - j);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
cand = 1;
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
while (len > num_ranges * 2)
Add missing intarray files.
2003-06-12 03:31:05 +08:00
{
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
min = PG_INT64_MAX;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
for (i = 2; i < len; i += 2)
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
if (min > ((int64) dr[i] - (int64) dr[i - 1]))
Add missing intarray files.
2003-06-12 03:31:05 +08:00
{
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
min = ((int64) dr[i] - (int64) dr[i - 1]);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
cand = i;
}
Remove useless casts to (void *) in arguments of some system functions The affected functions are: bsearch, memcmp, memcpy, memset, memmove, qsort, repalloc Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/fd9adf5d-b1aa-e82f-e4c7-263c30145807%40enterprisedb.com
2023-02-07 13:53:05 +08:00
memmove(&dr[cand - 1], &dr[cand + 1], (len - cand - 1) * sizeof(int32));
Add missing intarray files.
2003-06-12 03:31:05 +08:00
len -= 2;
}
Initial pgindent run for v12. This is still using the 2.0 version of pg_bsd_indent. I thought it would be good to commit this separately, so as to document the differences between 2.0 and 2.1 behavior. Discussion: https://postgr.es/m/16296.1558103386@sss.pgh.pa.us
2019-05-23 00:55:34 +08:00
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
/*
* check sparseness of result
*/
lenr = internal_size(dr, len);
if (lenr < 0 || lenr > MAXNUMELTS)
ereport(ERROR,
intarray: Prevent out-of-bound memory reads with gist__int_ops As gist__int_ops stands in intarray, it is possible to store GiST entries for leaf pages that can cause corruptions when decompressed. Leaf nodes are stored as decompressed all the time by the compression method, and the decompression method should map with that, retrieving the contents of the page without doing any decompression. However, the code authorized the insertion of leaf page data with a higher number of array items than what can be supported, generating a NOTICE message to inform about this matter (199 for a 8k page, for reference). When calling the decompression method, a decompression would be attempted on this leaf node item but the contents should be retrieved as they are. The NOTICE message generated when dealing with the compression of a leaf page and too many elements in the input array for gist__int_ops has been introduced by 08ee64e, removing the marker stored in the array to track if this is actually a leaf node. However, it also missed the fact that the decompression path should do nothing for a leaf page. Hence, as the code stand, a too-large array would be stored as uncompressed but the decompression path would attempt a decompression rather that retrieving the contents as they are. This leads to various problems. First, even if 08ee64e tried to address that, it is possible to do out-of-bound chunk writes with a large input array, with the backend informing about that with WARNINGs. On decompression, retrieving the stored leaf data would lead to incorrect memory reads, leading to crashes or even worse. Perhaps somebody would be interested in expanding the number of array items that can be handled in a leaf page for this operator in the future, which would require revisiting the choice done in 08ee64e, but based on the lack of reports about this problem since 2005 it does not look so. For now, this commit prevents the insertion of data for leaf pages when using more array items that the code can handle on decompression, switching the NOTICE message to an ERROR. If one wishes to use more array items, gist__intbig_ops is an optional choice. While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a limit is reached, because that's what the module is facing in such cases. Author: Ankit Kumar Pandey, Alexander Lakhin Reviewed-by: Richard Guo, Michael Paquier Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org Backpatch-through: 11
2023-06-15 12:45:34 +08:00
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("data is too sparse, recreate index using gist__intbig_ops opclass instead")));
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
Add missing intarray files.
2003-06-12 03:31:05 +08:00
r = resize_intArrayType(r, len);
retval = palloc(sizeof(GISTENTRY));
gistentryinit(*retval, PointerGetDatum(r),
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
entry->rel, entry->page, entry->offset, false);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PG_RETURN_POINTER(retval);
}
else
PG_RETURN_POINTER(entry);
}
Datum
g_int_decompress(PG_FUNCTION_ARGS)
{
GISTENTRY *entry = (GISTENTRY *) PG_GETARG_POINTER(0);
GISTENTRY *retval;
ArrayType *r;
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
int num_ranges = G_INT_GET_NUMRANGES();
Add missing intarray files.
2003-06-12 03:31:05 +08:00
int *dr,
lenr;
ArrayType *in;
int lenin;
int *din;
Fix integer-overflow problem in intarray's g_int_decompress(). An array element equal to INT_MAX gave this code indigestion, causing an infinite loop that surely ended in SIGSEGV. We fixed some nearby problems awhile ago (cf 757c5182f) but missed this. Report and diagnosis by Alexander Lakhin (bug #18273); patch by me Discussion: https://postgr.es/m/18273-9a832d1da122600c@postgresql.org
2024-01-08 04:19:50 +08:00
int i;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
in = DatumGetArrayTypeP(entry->key);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
CHECKARRVALID(in);
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
if (ARRISEMPTY(in))
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 12:21:44 +08:00
{
if (in != (ArrayType *) DatumGetPointer(entry->key))
{
retval = palloc(sizeof(GISTENTRY));
gistentryinit(*retval, PointerGetDatum(in),
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
entry->rel, entry->page, entry->offset, false);
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 12:21:44 +08:00
PG_RETURN_POINTER(retval);
}
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PG_RETURN_POINTER(entry);
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 12:21:44 +08:00
}
Add missing intarray files.
2003-06-12 03:31:05 +08:00
lenin = ARRNELEMS(in);
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
if (lenin < 2 * num_ranges)
Add missing intarray files.
2003-06-12 03:31:05 +08:00
{ /* not compressed value */
if (in != (ArrayType *) DatumGetPointer(entry->key))
{
retval = palloc(sizeof(GISTENTRY));
gistentryinit(*retval, PointerGetDatum(in),
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
entry->rel, entry->page, entry->offset, false);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PG_RETURN_POINTER(retval);
}
PG_RETURN_POINTER(entry);
}
din = ARRPTR(in);
lenr = internal_size(din, lenin);
Avoid crashes in contrib/intarray gist__int_ops (bug #15518) 1. Integer overflow in internal_size could result in memory corruption in decompression since a zero-length array would be allocated and then written to. This leads to crashes or corruption when traversing an index which has been populated with sufficiently sparse values. Fix by using int64 for computations and checking for overflow. 2. Integer overflow in g_int_compress could cause pessimal merge choices, resulting in unnecessarily large ranges (which would in turn trigger issue 1 above). Fix by using int64 again. 3. Even without overflow, array sizes could become large enough to cause unexplained memory allocation errors. Fix by capping the sizes to a safe limit and report actual errors pointing at gist__intbig_ops as needed. 4. Large inputs to the compression function always consist of large runs of consecutive integers, and the compression loop was processing these one at a time in an O(N^2) manner with a lot of overhead. The expected runtime of this function could easily exceed 6 months for a single call as a result. Fix by performing a linear-time first pass, which reduces the worst case to something on the order of seconds. Backpatch all the way, since this has been wrong forever. Per bug #15518 from report from irc user "dymk", analysis and patch by me. Discussion: https://postgr.es/m/15518-799e426c3b4f8358@postgresql.org
2018-11-24 07:56:39 +08:00
if (lenr < 0 || lenr > MAXNUMELTS)
ereport(ERROR,
intarray: Prevent out-of-bound memory reads with gist__int_ops As gist__int_ops stands in intarray, it is possible to store GiST entries for leaf pages that can cause corruptions when decompressed. Leaf nodes are stored as decompressed all the time by the compression method, and the decompression method should map with that, retrieving the contents of the page without doing any decompression. However, the code authorized the insertion of leaf page data with a higher number of array items than what can be supported, generating a NOTICE message to inform about this matter (199 for a 8k page, for reference). When calling the decompression method, a decompression would be attempted on this leaf node item but the contents should be retrieved as they are. The NOTICE message generated when dealing with the compression of a leaf page and too many elements in the input array for gist__int_ops has been introduced by 08ee64e, removing the marker stored in the array to track if this is actually a leaf node. However, it also missed the fact that the decompression path should do nothing for a leaf page. Hence, as the code stand, a too-large array would be stored as uncompressed but the decompression path would attempt a decompression rather that retrieving the contents as they are. This leads to various problems. First, even if 08ee64e tried to address that, it is possible to do out-of-bound chunk writes with a large input array, with the backend informing about that with WARNINGs. On decompression, retrieving the stored leaf data would lead to incorrect memory reads, leading to crashes or even worse. Perhaps somebody would be interested in expanding the number of array items that can be handled in a leaf page for this operator in the future, which would require revisiting the choice done in 08ee64e, but based on the lack of reports about this problem since 2005 it does not look so. For now, this commit prevents the insertion of data for leaf pages when using more array items that the code can handle on decompression, switching the NOTICE message to an ERROR. If one wishes to use more array items, gist__intbig_ops is an optional choice. While on it, use ERRCODE_PROGRAM_LIMIT_EXCEEDED as error code when a limit is reached, because that's what the module is facing in such cases. Author: Ankit Kumar Pandey, Alexander Lakhin Reviewed-by: Richard Guo, Michael Paquier Discussion: https://postgr.es/m/796b65c3-57b7-bddf-b0d5-a8afafb8b627@gmail.com Discussion: https://postgr.es/m/17888-f72930e6b5ce8c14@postgresql.org Backpatch-through: 11
2023-06-15 12:45:34 +08:00
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("compressed array is too big, recreate index using gist__intbig_ops opclass instead")));
Add missing intarray files.
2003-06-12 03:31:05 +08:00
r = new_intArrayType(lenr);
dr = ARRPTR(r);
for (i = 0; i < lenin; i += 2)
Fix integer-overflow problem in intarray's g_int_decompress(). An array element equal to INT_MAX gave this code indigestion, causing an infinite loop that surely ended in SIGSEGV. We fixed some nearby problems awhile ago (cf 757c5182f) but missed this. Report and diagnosis by Alexander Lakhin (bug #18273); patch by me Discussion: https://postgr.es/m/18273-9a832d1da122600c@postgresql.org
2024-01-08 04:19:50 +08:00
{
/* use int64 for j in case din[i + 1] is INT_MAX */
for (int64 j = din[i]; j <= din[i + 1]; j++)
Add missing intarray files.
2003-06-12 03:31:05 +08:00
if ((!i) || *(dr - 1) != j)
Fix integer-overflow problem in intarray's g_int_decompress(). An array element equal to INT_MAX gave this code indigestion, causing an infinite loop that surely ended in SIGSEGV. We fixed some nearby problems awhile ago (cf 757c5182f) but missed this. Report and diagnosis by Alexander Lakhin (bug #18273); patch by me Discussion: https://postgr.es/m/18273-9a832d1da122600c@postgresql.org
2024-01-08 04:19:50 +08:00
*dr++ = (int) j;
}
Add missing intarray files.
2003-06-12 03:31:05 +08:00
if (in != (ArrayType *) DatumGetPointer(entry->key))
pfree(in);
retval = palloc(sizeof(GISTENTRY));
gistentryinit(*retval, PointerGetDatum(r),
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
entry->rel, entry->page, entry->offset, false);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PG_RETURN_POINTER(retval);
}
/*
** The GiST Penalty method for _intments
*/
Datum
g_int_penalty(PG_FUNCTION_ARGS)
{
GISTENTRY *origentry = (GISTENTRY *) PG_GETARG_POINTER(0);
GISTENTRY *newentry = (GISTENTRY *) PG_GETARG_POINTER(1);
float *result = (float *) PG_GETARG_POINTER(2);
ArrayType *ud;
float tmp1,
tmp2;
ud = inner_int_union((ArrayType *) DatumGetPointer(origentry->key),
(ArrayType *) DatumGetPointer(newentry->key));
rt__int_size(ud, &tmp1);
rt__int_size((ArrayType *) DatumGetPointer(origentry->key), &tmp2);
*result = tmp1 - tmp2;
pfree(ud);
PG_RETURN_POINTER(result);
}
Datum
g_int_same(PG_FUNCTION_ARGS)
{
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
ArrayType *a = PG_GETARG_ARRAYTYPE_P(0);
ArrayType *b = PG_GETARG_ARRAYTYPE_P(1);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
bool *result = (bool *) PG_GETARG_POINTER(2);
Replace int2/int4 in C code with int16/int32 The latter was already the dominant use, and it's preferable because in C the convention is that intXX means XX bits. Therefore, allowing mixed use of int2, int4, int8, int16, int32 is obviously confusing. Remove the typedefs for int2 and int4 for now. They don't seem to be widely used outside of the PostgreSQL source tree, and the few uses can probably be cleaned up by the time this ships.
2012-06-25 06:51:46 +08:00
int32 n = ARRNELEMS(a);
int32 *da,
Add missing intarray files.
2003-06-12 03:31:05 +08:00
*db;
Defend against nulls-in-arrays in contrib/intarray. I may have put in more tests than strictly necessary, but did not feel like tracing call paths in detail ...
2005-11-19 11:00:09 +08:00
CHECKARRVALID(a);
CHECKARRVALID(b);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
if (n != ARRNELEMS(b))
{
*result = false;
PG_RETURN_POINTER(result);
}
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
*result = true;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
da = ARRPTR(a);
db = ARRPTR(b);
while (n--)
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
{
Add missing intarray files.
2003-06-12 03:31:05 +08:00
if (*da++ != *db++)
{
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 12:22:32 +08:00
*result = false;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
break;
}
Fix assorted corner-case bugs in contrib/intarray. The array containment operators now behave per mathematical expectation for empty arrays (ie, an empty array is contained in anything). Both these operators and the query_int operators now work as expected in GiST and GIN index searches, rather than having corner cases where the index searches gave different answers. Also, fix unexpected failures where the operators would claim that an array contained nulls, when in fact there was no longer any null present (similar to bug #5784). The restriction to not have nulls is still there, as removing it would take a lot of added code complexity and probably slow things down significantly. Also, remove the arbitrary restriction to 1-D arrays; unlike the other restriction, this was buying us nothing performance-wise. Assorted cosmetic improvements and marginal performance improvements, too.
2011-01-09 13:39:21 +08:00
}
Add missing intarray files.
2003-06-12 03:31:05 +08:00
PG_RETURN_POINTER(result);
}
/*****************************************************************
** Common GiST Method
*****************************************************************/
typedef struct
{
OffsetNumber pos;
float cost;
} SPLITCOST;
static int
comparecost(const void *a, const void *b)
{
Remove many -Wcast-qual warnings This addresses only those cases that are easy to fix by adding or moving a const qualifier or removing an unnecessary cast. There are many more complicated cases remaining.
2011-09-12 02:54:32 +08:00
if (((const SPLITCOST *) a)->cost == ((const SPLITCOST *) b)->cost)
Add missing intarray files.
2003-06-12 03:31:05 +08:00
return 0;
else
Remove many -Wcast-qual warnings This addresses only those cases that are easy to fix by adding or moving a const qualifier or removing an unnecessary cast. There are many more complicated cases remaining.
2011-09-12 02:54:32 +08:00
return (((const SPLITCOST *) a)->cost > ((const SPLITCOST *) b)->cost) ? 1 : -1;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
}
/*
** The GiST PickSplit method for _intments
** We use Guttman's poly time split algorithm
*/
Datum
g_int_picksplit(PG_FUNCTION_ARGS)
{
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
GistEntryVector *entryvec = (GistEntryVector *) PG_GETARG_POINTER(0);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
GIST_SPLITVEC *v = (GIST_SPLITVEC *) PG_GETARG_POINTER(1);
OffsetNumber i,
j;
ArrayType *datum_alpha,
*datum_beta;
ArrayType *datum_l,
*datum_r;
ArrayType *union_d,
*union_dl,
*union_dr;
ArrayType *inter_d;
bool firsttime;
float size_alpha,
size_beta,
size_union,
size_inter;
float size_waste,
waste;
float size_l,
size_r;
int nbytes;
OffsetNumber seed_1 = 0,
seed_2 = 0;
OffsetNumber *left,
*right;
OffsetNumber maxoff;
SPLITCOST *costvector;
#ifdef GIST_DEBUG
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
elog(DEBUG3, "--------picksplit %d", entryvec->n);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
#endif
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
maxoff = entryvec->n - 2;
Add missing intarray files.
2003-06-12 03:31:05 +08:00
nbytes = (maxoff + 2) * sizeof(OffsetNumber);
v->spl_left = (OffsetNumber *) palloc(nbytes);
v->spl_right = (OffsetNumber *) palloc(nbytes);
firsttime = true;
waste = 0.0;
for (i = FirstOffsetNumber; i < maxoff; i = OffsetNumberNext(i))
{
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
datum_alpha = GETENTRY(entryvec, i);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
for (j = OffsetNumberNext(i); j <= maxoff; j = OffsetNumberNext(j))
{
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
datum_beta = GETENTRY(entryvec, j);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
/* compute the wasted space by unioning these guys */
/* size_waste = size_union - size_inter; */
union_d = inner_int_union(datum_alpha, datum_beta);
rt__int_size(union_d, &size_union);
inter_d = inner_int_inter(datum_alpha, datum_beta);
rt__int_size(inter_d, &size_inter);
size_waste = size_union - size_inter;
pfree(union_d);
Eliminate unnecessary NULL checks in picksplit method of intarray. Where these checks were being done there was no code path which could leave them NULL. Michael Paquier per Coverity
2015-02-17 05:26:23 +08:00
pfree(inter_d);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
/*
* are these a more promising split that what we've already seen?
*/
if (size_waste > waste || firsttime)
{
waste = size_waste;
seed_1 = i;
seed_2 = j;
firsttime = false;
}
}
}
left = v->spl_left;
v->spl_nleft = 0;
right = v->spl_right;
v->spl_nright = 0;
if (seed_1 == 0 || seed_2 == 0)
{
seed_1 = 1;
seed_2 = 2;
}
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
datum_alpha = GETENTRY(entryvec, seed_1);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
datum_l = copy_intArrayType(datum_alpha);
rt__int_size(datum_l, &size_l);
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
datum_beta = GETENTRY(entryvec, seed_2);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
datum_r = copy_intArrayType(datum_beta);
rt__int_size(datum_r, &size_r);
maxoff = OffsetNumberNext(maxoff);
/*
* sort entries
*/
costvector = (SPLITCOST *) palloc(sizeof(SPLITCOST) * maxoff);
for (i = FirstOffsetNumber; i <= maxoff; i = OffsetNumberNext(i))
{
costvector[i - 1].pos = i;
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
datum_alpha = GETENTRY(entryvec, i);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
union_d = inner_int_union(datum_l, datum_alpha);
rt__int_size(union_d, &size_alpha);
pfree(union_d);
union_d = inner_int_union(datum_r, datum_alpha);
rt__int_size(union_d, &size_beta);
pfree(union_d);
Use fabsf() instead of Abs() or fabs() where appropriate This function is new in C99. Reviewed-by: Zhang Mingli <zmlpostgres@gmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/4beb42b5-216b-bce8-d452-d924d5794c63%40enterprisedb.com
2022-10-08 19:41:18 +08:00
costvector[i - 1].cost = fabsf((size_alpha - size_l) - (size_beta - size_r));
Add missing intarray files.
2003-06-12 03:31:05 +08:00
}
Remove useless casts to (void *) in arguments of some system functions The affected functions are: bsearch, memcmp, memcpy, memset, memmove, qsort, repalloc Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/fd9adf5d-b1aa-e82f-e4c7-263c30145807%40enterprisedb.com
2023-02-07 13:53:05 +08:00
qsort(costvector, maxoff, sizeof(SPLITCOST), comparecost);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
/*
* Now split up the regions between the two seeds. An important property
* of this split algorithm is that the split vector v has the indices of
* items to be split in order in its left and right vectors. We exploit
* this property by doing a merge in the code that actually splits the
* page.
*
* For efficiency, we also place the new index tuple in this loop. This is
* handled at the very end, when we have placed all the existing tuples
* and i == maxoff + 1.
*/
for (j = 0; j < maxoff; j++)
{
i = costvector[j].pos;
/*
* If we've already decided where to place this item, just put it on
* the right list. Otherwise, we need to figure out which page needs
* the least enlargement in order to store the item.
*/
if (i == seed_1)
{
*left++ = i;
v->spl_nleft++;
continue;
}
else if (i == seed_2)
{
*right++ = i;
v->spl_nright++;
continue;
}
/* okay, which page needs least enlargement? */
Cleanup vectors of GISTENTRY and eliminate problem with 64-bit strict-aligned boxes. Change interface to user-defined GiST support methods union and picksplit. Now instead of bytea struct it used special GistEntryVector structure.
2004-03-30 23:45:33 +08:00
datum_alpha = GETENTRY(entryvec, i);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
union_dl = inner_int_union(datum_l, datum_alpha);
union_dr = inner_int_union(datum_r, datum_alpha);
rt__int_size(union_dl, &size_alpha);
rt__int_size(union_dr, &size_beta);
/* pick which page to add it to */
if (size_alpha - size_l < size_beta - size_r + WISH_F(v->spl_nleft, v->spl_nright, 0.01))
{
Eliminate unnecessary NULL checks in picksplit method of intarray. Where these checks were being done there was no code path which could leave them NULL. Michael Paquier per Coverity
2015-02-17 05:26:23 +08:00
pfree(datum_l);
pfree(union_dr);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
datum_l = union_dl;
size_l = size_alpha;
*left++ = i;
v->spl_nleft++;
}
else
{
Eliminate unnecessary NULL checks in picksplit method of intarray. Where these checks were being done there was no code path which could leave them NULL. Michael Paquier per Coverity
2015-02-17 05:26:23 +08:00
pfree(datum_r);
pfree(union_dl);
Add missing intarray files.
2003-06-12 03:31:05 +08:00
datum_r = union_dr;
size_r = size_beta;
*right++ = i;
v->spl_nright++;
}
}
pfree(costvector);
*right = *left = FirstOffsetNumber;
v->spl_ldatum = PointerGetDatum(datum_l);
v->spl_rdatum = PointerGetDatum(datum_r);
PG_RETURN_POINTER(v);
}
Implement operator class parameters PostgreSQL provides set of template index access methods, where opclasses have much freedom in the semantics of indexing. These index AMs are GiST, GIN, SP-GiST and BRIN. There opclasses define representation of keys, operations on them and supported search strategies. So, it's natural that opclasses may be faced some tradeoffs, which require user-side decision. This commit implements opclass parameters allowing users to set some values, which tell opclass how to index the particular dataset. This commit doesn't introduce new storage in system catalog. Instead it uses pg_attribute.attoptions, which is used for table column storage options but unused for index attributes. In order to evade changing signature of each opclass support function, we implement unified way to pass options to opclass support functions. Options are set to fn_expr as the constant bytea expression. It's possible due to the fact that opclass support functions are executed outside of expressions, so fn_expr is unused for them. This commit comes with some examples of opclass options usage. We parametrize signature length in GiST. That applies to multiple opclasses: tsvector_ops, gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and gist_hstore_ops. Also we parametrize maximum number of integer ranges for gist__int_ops. However, the main future usage of this feature is expected to be json, where users would be able to specify which way to index particular json parts. Catversion is bumped. Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru Author: Nikita Glukhov, revised by me Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-31 00:17:11 +08:00
Datum
g_int_options(PG_FUNCTION_ARGS)
{
local_relopts *relopts = (local_relopts *) PG_GETARG_POINTER(0);
init_local_reloptions(relopts, sizeof(GISTIntArrayOptions));
add_local_int_reloption(relopts, "numranges",
"number of ranges for compression",
G_INT_NUMRANGES_DEFAULT, 1, G_INT_NUMRANGES_MAX,
offsetof(GISTIntArrayOptions, num_ranges));
PG_RETURN_VOID();
}
Reference in New Issue Copy Permalink