postgresql/contrib/intarray
Tom Lane f85f43dfb5 Backend support for autocommit removed, per recent discussions. The
only remnant of this failed experiment is that the server will take
SET AUTOCOMMIT TO ON.  Still TODO: provide some client-side autocommit
logic in libpq.
2003-05-14 03:26:03 +00:00
..
bench
data
expected Backend support for autocommit removed, per recent discussions. The 2003-05-14 03:26:03 +00:00
sql
_int.c This patch fixes a bunch of spelling mistakes in comments throughout the 2003-03-10 22:28:22 +00:00
_int.sql.in Backend support for autocommit removed, per recent discussions. The 2003-05-14 03:26:03 +00:00
Makefile
README.intarray Small fix in documentation and some examples of usage. Please, apply to 2002-12-27 14:10:29 +00:00

This is an implementation of RD-tree data structure using GiST interface
of PostgreSQL. It has built-in lossy compression.

Current implementation provides index support for one-dimensional array of
int4's - gist__int_ops, suitable for small and medium size of arrays (used on
default), and gist__intbig_ops for indexing large arrays (we use superimposed
signature with length of 4096 bits to represent sets).

All work was done by Teodor Sigaev (teodor@stack.net) and Oleg Bartunov
(oleg@sai.msu.su). See http://www.sai.msu.su/~megera/postgres/gist
for additional information. Andrey Oktyabrski did a great work on 
adding new functions and operations.


FUNCTIONS:

  int   icount(int[]) - the number of elements in intarray

test=# select icount('{1,2,3}'::int[]);   
 icount 
--------
      3
(1 row)

  int[] sort(int[], 'asc' | 'desc') - sort intarray

test=# select sort('{1,2,3}'::int[],'desc');
  sort   
---------
 {3,2,1}
(1 row)

  int[] sort(int[]) - sort in ascending order
  int[] sort_asc(int[]),sort_desc(int[]) - shortcuts for sort 

  int[] uniq(int[]) - returns unique elements

test=# select uniq(sort('{1,2,3,2,1}'::int[]));
  uniq   
---------
 {1,2,3}
(1 row)

  int   idx(int[], int item) - returns index of first intarray matching element to item, or
                              '0' if matching failed.

test=# select idx('{1,2,3,2,1}'::int[],2);
 idx 
-----
   2
(1 row)


  int[] subarray(int[],int START [, int LEN]) - returns part of intarray starting from
                                                element number START (from 1) and length LEN. 

test=# select subarray('{1,2,3,2,1}'::int[],2,3);
 subarray 
----------
 {2,3,2}
(1 row)


OPERATIONS:

  int[] && int[]  - overlap - returns TRUE if arrays has at least one common elements.
  int[] @  int[]  - contains - returns TRUE if left array contains right array
  int[] ~ int[]   - contained - returns TRUE if left array is contained in right array
  # int[]         - return the number of elements in array
  int[] + int     - push element to array ( add to end of array)
  int[] + int[]   - merge of arrays (right array added to the end of left one)
  int[] - int     - remove entries matched by right argument from array
  int[] - int[]   - remove right array from left
  int[] | int     - returns intarray - union of arguments
  int[] | int[]   - returns intarray as a union of two arrays
  int[] & int[]   - returns intersection of arrays
  int[] @@ query_int  - returns TRUE if array satisfies query (like '1&(2|3)') 
  query_int ~~ int[]  - -/-

CHANGES:

August 6, 2002
   1. Reworked patch from Andrey Oktyabrski (ano@spider.ru) with
      functions: icount, sort, sort_asc, uniq, idx, subarray
      operations: #, +, -, |, &
October 1, 2001
   1. Change search method in array to binary
September 28, 2001
   1. gist__int_ops now is without lossy
   2. add sort entry in picksplit
September 21, 2001
   1. Added support for boolean query (indexable operator @@, looks like
      a @@ '1|(2&3)', perfomance is better in any case )
   2. Done some small optimizations
March 19, 2001
   1. Added support for toastable keys
   2. Improved split algorithm for intbig (selection speedup is about 30%)

INSTALLATION:

  gmake
  gmake install
  -- load functions
  psql <database> < _int.sql 

REGRESSION TEST:

   gmake installcheck

EXAMPLE USAGE:

  create table message (mid int not null,sections int[]);
  create table message_section_map (mid int not null,sid int not null);

  -- create indices
CREATE unique index message_key on message ( mid );
CREATE unique index message_section_map_key2 on message_section_map (sid, mid );
CREATE INDEX message_rdtree_idx on message using gist ( sections gist__int_ops);

  -- select some messages with section in 1 OR 2 - OVERLAP operator
  select message.mid from message where message.sections && '{1,2}';  

  -- select messages contains in sections 1 AND 2 - CONTAINS operator
  select message.mid from message where message.sections @ '{1,2}';
  -- the same, CONTAINED operator
  select message.mid from message where '{1,2}' ~ message.sections;

BENCHMARK:

  subdirectory bench contains benchmark suite.
  cd ./bench
  1. createdb TEST
  2. psql TEST < ../_int.sql
  3. ./create_test.pl | psql TEST
  4. ./bench.pl - perl script to benchmark queries, supports OR, AND queries
                  with/without RD-Tree. Run script without arguments to 
                  see availbale options.

     a)test without RD-Tree (OR)
       ./bench.pl -d TEST -s 1,2 -v
     b)test with RD-Tree 
       ./bench.pl -d TEST -s 1,2 -v -r

BENCHMARKS:

Size of table <message>: 200000
Size of table <message_section_map>: 268538 

Distribution of messages by sections:

section 0: 73899 messages
section 1: 16298 messages
section 50: 1241 messages
section 99: 705 messages

old - without RD-Tree support,
new - with RD-Tree

+----------+---------------+----------------+
|Search set|OR, time in sec|AND, time in sec|
|          +-------+-------+--------+-------+
|          |  old  |  new  |   old  |  new  |
+----------+-------+-------+--------+-------+
|         1|  1.427|  0.215|       -|      -|
+----------+-------+-------+--------+-------+
|        99|  1.029|  0.018|       -|      -|
+----------+-------+-------+--------+-------+
|       1,2|  1.829|  0.334|   5.654|  0.042|
+----------+-------+-------+--------+-------+
| 1,2,50,60|  2.057|  0.359|   5.044|  0.007|
+----------+-------+-------+--------+-------+