TODO list for PostgreSQL ======================== Last updated: Sat Jan 26 23:20:24 EST 2002 Current maintainer: Bruce Momjian (pgman@candle.pha.pa.us) The most recent version of this document can be viewed at the PostgreSQL web site, http://www.PostgreSQL.org. A dash (-) marks changes that will appear in the upcoming 7.2 release. Bracketed items "[]" have more detailed. RELIABILITY ----------- * -SELECT pg_class FROM pg_class generates strange error (Bruce) ENHANCEMENTS ------------ URGENT * Add replication of distributed databases [replication] o automatic failover o load balancing o master/slave replication o multi-master replication o partition data across servers o sample implementation in contrib/rserv o queries across databases or servers (two-phase commit) o http://gborg.postgresql.org/project/pgreplication/projdisplay.php * Point-in-time data recovery using backup and write-ahead log * Allow row re-use without vacuum (Tom) REPORTING * Allow elog() to return error codes, module name, file name, line number, not just messages (Peter E) * Add error codes (Peter E) * -Allow international error message support (Peter E) * Change DEBUG startup tag to NOTICE; change NOTICE to output to client only if client exists (Bruce) * Show location of syntax error in query [yacc] * -Populate backend status area and write program to dump status data (Jan) PERMISSIONS * Improve control over user privileges, including table creation and lock use [privileges] (Karel, others) * Allow usernames to be specified directly in pg_hba.conf (Bruce) * Add PGPASSWORDFILE environment variable or ~/.pgpass to store user/host/password combinations * Remove PGPASSWORD because is insecure on some OS's * Make single-user local access permissions the default (Peter E) * -Allow single-user access without passwords using Unix socket permissions * -Better document pg_hba.conf host-based authentication (Bruce) * -Add MD5 to ODBC (Bruce) * Use thread-safe crypt() in libpq, if available ADMIN * Incremental backups * Make it easier to create a database owned by someone who can't createdb, perhaps CREATE DATABASE dbname WITH OWNER = "user" (Gavin) * Make equals sign optional in CREATE DATABASE WITH param = 'val' * -Permission to DELETE table also allows UPDATE (Peter E) * -Remove unused sort files on postmaster startup (Bruce) * Remove unreferenced table files and temp tables during database vacuum or postmaster startup (Bruce) * -Remove unreferenced sort files during postmaster startup (Bruce) * Add table name mapping for numeric file names (Bruce) * -Encrypt passwords in pg_shadow table using MD5 (Bruce, Vince) * Remove behavior of postmaster -o after making postmaster/postgres flags unique * Allow logging of query durations * -Put sort files in their own directory (Bruce) DATA TYPES * Add domain capability [domain] * Add IPv6 capability to INET/CIDR types * Remove Money type, add money formatting for decimal type * SELECT cash_out(2) crashes because of opaque * Declare typein/out functions in pg_proc with a special "C string" data type * Functions returning sets do not totally work * -Add SQL standard function bit_length() (Peter E) * -Make oid use unsigned int more reliably (Tom) * Change factorial to return a numeric * Add function to return compressed length of TOAST data values (Tom) * CONVERSION o -Add conversion function from text to inet o Store binary-compatible type information in the system o Allow better handling of numeric constants, type conversion [typeconv] o SELECT col FROM tab WHERE numeric_col = 10.1 fails, requires quotes * ARRAYS o Allow nulls in arrays o Allow arrays to be ORDER'ed o Ensure we have array-eq operators for every built-in array type o Support construction of array result values in expressions * BINARY DATA o -Add non-large-object binary field (already exists -- bytea) o -Make binary interface for TOAST columns (base64) o Improve vacuum of large objects, like /contrib/vacuumlo o Add security checking for large objects o Make file in/out interface for TOAST columns, similar to large object interface (force out-of-line storage and no compression) o Auto-delete large objects when referencing row is deleted MULTI-LANGUAGE SUPPORT * Add NCHAR (as distinguished from ordinary varchar), * Allow LOCALE on a per-column basis, default to ASCII * Support multiple simultaneous character sets, per SQL92 * -Reject character sequences those are not valid in their charset (Tatsuo) * -Make functions more multi-byte aware, e.g. trim() (Tatsuo) * -Make n of CHAR(n)/VARCHAR(n) the number of letters, not bytes (Tatsuo) * Allow setting database character set without multibyte enabled * Improve Unicode combined character handling * Optimize locale to have minimal performance impact when not used (Peter E) * -Optimize textlength(), etc. for single-byte encodings (Tatsuo) * Add octet_length_server() and octet_length_client() (Thomas, Tatsuo) * Make octet_length_client the same as octet_length() (?) VIEWS / RULES * Automatically create rules on views so they are updateable, per SQL92 [view] * Add the functionality for WITH CHECK OPTION clause of CREATE VIEW * Allow NOTIFY in rules involving conditionals * -Evaluate INSERT rules at end of query, rather than beginning (Jan) * Allow temporary views * Move psql backslash information into views * Allow RULE recompilation * Remove brackets as multi-statement rule grouping, must use parens INDEXES * Allow CREATE INDEX zman_index ON test (date_trunc( 'day', zman ) datetime_ops) fails index can't store constant parameters * Order duplicate index entries by tid for faster heap lookups * -Re-enable partial indexes * -Prevent pg_attribute from having duplicate oids for indexes (Tom) * Allow inherited tables to inherit index, UNIQUE constraint, and primary key, foreign key [inheritance] * UNIQUE INDEX on base column not honored on inserts from inherited table INSERT INTO inherit_table (unique_index_col) VALUES (dup) should fail [inheritance] * Allow UPDATE/DELETE on inherited table * Have UPDATE/DELETE clean out indexes * Add UNIQUE capability to non-btree indexes * Add btree index support for reltime, tinterval, regproc * Add rtree index support for line, lseg, path, point * Certain indexes will not shrink, e.g. oid indexes with many inserts * Use indexes for min() and max() or convert to SELECT col FROM tab ORDER BY col DESC LIMIT 1 if appropriate index exists and WHERE clause acceptible * Allow LIKE indexing optimization for non-ASCII locales * Use index to restrict rows returned by multi-key index when used with non-consecutive keys or OR clauses, so fewer heap accesses * Be smarter about insertion of already-ordered data into btree index * -Gather more accurate dispersion statistics using indexes (Tom) * Add deleted bit to index tuples to reduce heap access * Prevent index uniqueness checks when UPDATE does not modifying column * Add bitmap indexes [performance] * Improve handling of index scans for NULL * Allow SELECT * FROM tab WHERE int2col = 4 to use int2col index, int8, float4, numeric/decimal too [optimizer] * -Use indexes with CIDR '<<' (contains) operator * Improve concurrency in GIST * Add FILLFACTOR to index creation SYSTEM TABLES * -Add unique indexes to pg_shadow.usename and pg_shadow.usesysid or switch to pg_shadow.oid as user id (Tom) * -Add unique indexes on pg_database (Tom) * -Check all system tables and add unique indexes as needed (Tom) * -Remove pg_listener index (Tom) * -Remove unused pg_variable, pg_inheritproc, pg_ipl tables (Bruce) COMMANDS * Add SIMILAR TO to allow character classes, 'pg_[a-c]%' * Add BETWEEN ASYMMETRIC/SYMMETRIC * -Allow LOCK TABLE tab1, tab2, tab3 so all tables locked in unison * Remove LIMIT #,# and force use LIMIT and OFFSET clauses in 7.3 (Bruce) * Allow LIMIT/OFFSET to use expressions * -Allow GRANT/REVOKE to handle multiple user/group names (Vince) * -Allow CREATEUSER/CREATEDB ordering in CREATE/ALTER USER (Vince) * Disallow TRUNCATE on tables that are involved in referential constraints * Add OR REPLACE clauses to non-FUNCTION object creation * CREATE TABLE AS can not determine column lengths from expressions [atttypmod] * Allow UPDATE to handle complex aggregates [update] * Prevent create/drop scripts from allowing extra args (Bruce) * Allow command blocks to ignore certain types of errors * ALTER o ALTER TABLE ADD COLUMN does not honor DEFAULT and non-CHECK CONSTRAINT o ALTER TABLE ADD COLUMN to inherited table put column in wrong place [inheritance] o Add ALTER TABLE DROP COLUMN feature [drop] (Bruce) o Add ALTER FUNCTION o Add ALTER TABLE DROP non-CHECK CONSTRAINT o -Add ALTER TABLE DROP CHECK CONSTRAINT (Christopher Kings-Lynne) o ALTER TABLE ADD PRIMARY KEY (Christopher Kings-Lynne) o ALTER TABLE ADD UNIQUE (Christopher Kings-Lynne) o ALTER TABLE ADD COLUMN column SERIAL doesn't create sequence o ALTER TABLE ADD COLUMN column SET DEFAULT should fill existing rows with DEFAULT value o Have ALTER TABLE OWNER change all dependant objects like indexes * CLUSTER o cluster all tables at once o prevent lose of indexes, permissions, inheritance (Bruce) o Automatically maintain clustering on a table o -Keep statistics about clustering (Tom) [optimizer] * COPY o Allow specification of column names o Allow dump/load of CSV format o Change syntax to WITH DELIMITER, (keep old syntax around?) o Allow COPY to report error lines and continue; optionally allow error codes to be specified; requires savepoints or can not be run in a multi-statement transaction * CURSOR o Allow BINARY option to SELECT, just like DECLARE o MOVE 0 should not move to end of cursor o Allow cursors to be DECLAREd/OPENed/CLOSEed outside transactions o Allow UPDATE/DELETE WHERE CURRENT OF cursor using per-cursor tid stored in the backend o Prevent DROP of table being referenced by our own open cursor * INSERT o Allow INSERT/UPDATE of system-generated oid value for a row o Allow INSERT INTO tab (col1, ..) VALUES (val1, ..), (val2, ..) o Allow INSERT INTO my_table VALUES (a, b, c, DEFAULT, x, y, z, ...) o Disallow missing columns in INSERT ... VALUES, per ANSI o Allow INSERT/UPDATE ... RETURNING new.col or old.col; handle RULE cases (Philip) * SHOW/SET o Add SHOW command to display locks o -Add SHOW command to show all settings o -Add a global RESET command for use with connection pooling o Add SET or BEGIN timeout parameter to cancel query o Add SET REAL_FORMAT and SET DOUBLE_PRECISION_FORMAT using printf args o Remove SET KSQO option now that OR processing is improved (Tom) o Add SET PERFORMANCE_TIPS option to suggest INDEX, VACUUM, VACUUM ANALYZE, and CLUSTER o Add SHOW command to see locale * SERVER-SIDE LANGUAGES o Allow PL/PgSQL's RAISE function to take expressions o Fix PL/PgSQL to handle quoted mixed-case identifiers o Change PL/PgSQL to use palloc() instead of malloc() o Add untrusted version of plpython o Add plsh server-side shell language (Peter E) o Allow Java server-side programming (?) [java] CLIENTS * -Make NULL's come out at the beginning or end depending on the ORDER BY direction (Tom) * Have pg_dump use LEFT OUTER JOIN in multi-table SELECTs or multiple SELECTS to avoid bad system catalog entries * Have pg_dump -C dump database location and encoding information * Allow psql \d to show foreign keys * Allow psql \d to show temporary table structure * Add XML interface: psql, pg_dump, COPY, separate server (?) * -Fix libpq to properly handle socket failures under native MS Win32 * Add config file check for $ODBCINI, $HOME/.odbc.ini, installpath/etc/odbc.ini * JDBC o Comprehensive test suite. This may be available already. o Updateable resultSet (must be done in backend code) o -Improved DatabaseMetaData [java] o JDBC-standard BLOB support o Error Codes (pending backend implementation) o -Support for binary data/bytea o Support both 'make' and 'ant' o -Add MD5 capability o Fix LargeObject API to handle OIDs as unsigned ints o Implement cancel() method on Statement o Use cursors implicitly to avoid large results (see setCursorName()) o Add support for CallableStatements * ECPG o Implement set descriptor, using descriptor o Make casts work in variable initializations o -Allow variable to specify the connection name o -Fix variable handling in EXEC SQL AT statement o Implement SQLDA o -Allow SELECT of array of strings into a auto-sized variable o Solve cardinality > 1 for input descriptors / variables o Understand structure definitions outside a declare section o sqlwarn[6] should be 'W' if the PRECISION or SCALE value specified o Improve error handling o Allow :var[:index] or :var[] as cvariable for an array var o Add a semantic check level, e.g. check if a table really exists o -Remove space_or_nl and line_end from pgc.l o Fix nested C comments o Add SQLSTATE o fix handling of DB attributes that are arrays REFERENTIAL INTEGRITY * Add MATCH PARTIAL referential integrity [foreign] * -Check that primary key exists at foreign key definition time * Add deferred trigger queue file (Jan) * Allow oid to act as a foreign key * Implement dirty reads and use them in RI triggers * Make triggers refer to columns by number, not name * Enforce referential integrity for system tables * -INSERT & UPDATE/DELETE in transaction of primary key fails with deferredTriggerGetPreviousEvent or "change violation" * Allow user to control trigger firing order * Change foreign key constraint for array -> element to mean element in array * Fix foreign key constraints to not error on intermediate db states (Stephan) DEPENDENCY CHECKING * Add pg_depend table for dependency recording; use sysrelid, oid, depend_sysrelid, depend_oid, name * Auto-destroy sequence on DROP of table with SERIAL; perhaps a separate SERIAL type * Have SERIAL generate non-colliding sequence names when we have auto-destruction * Prevent column dropping if column is used by foreign key * Propagate column or table renaming to foreign key constraints * Automatically drop constraints/functions when object is dropped * Make constraints clearer in dump file * Make foreign keys easier to identify TRANSACTIONS * Allow autocommit so always in a transaction block * Overhaul bufmgr/lockmgr/transaction manager * Allow savepoints / nested transactions [transactions] * -Handle transaction rollover (Tom) [transactions] EXOTIC FEATURES * Add sql3 recursive unions * Add the concept of dataspaces/tablespaces [tablespaces] * Allow SQL92 schemas [schema] * Allow queries across multiple databases [crossdb] * Add pre-parsing phase that converts non-ANSI features to supported features * Allow plug-in modules to emulate features from other databases * SQL*Net listener that makes PostgreSQL appear as an Oracle database to clients PERFORMANCE ----------- FSYNC * Delay fsync() when other backends are about to commit too [fsync] o Determine optimal commit_delay value * Determine optimal fdatasync/fsync, O_SYNC/O_DSYNC options o Allow multiple blocks to be written to WAL with one write() CACHE * Cache most recent query plan(s) (Karel) [prepare] * Shared catalog cache, reduce lseek()'s by caching table size in shared area * Add free-behind capability for large sequential scans (Bruce) * Allow binding query args over FE/BE protocol * Consider use of open/fcntl(O_DIRECT) to minimize OS caching * Make blind writes go through the file descriptor cache VACUUM * Improve speed with indexes (perhaps recreate index instead) [vacuum] * Reduce lock time by moving tuples with read lock, then write lock and truncate table [vacuum] * -Make ANALYZE a separate command (Tom) * -Allow ANALYZE to ESTIMATE based on certain random percentage of rows (Tom) * Add LAZY VACUUM (Vadim) [performance] LOCKING * Make locking of shared data structures more fine-grained * Add code to detect an SMP machine and handle spinlocks accordingly from distributted.net, http://www1.distributed.net/source, in client/common/cpucheck.cpp * Research use of sched_yield() for spinlock acquisition failure * -Improve spinlock code [performance] (Tom) STARTUP TIME * Experiment with multi-threaded backend [thread] * Add connection pooling [pool] * Allow persistent backends [persistent] * Create a transaction processor to aid in persistent connections and connection pooling * Do listen() in postmaster and accept() in pre-forked backend * Have pre-forked backend pre-connect to last requested database or pass file descriptor to backend pre-forked for matching database WRITE-AHEAD LOG * Have after-change WAL write()'s write only modified data to kernel * Reduce number of after-change WAL writes; they exist only to gaurd against partial page writes * Turn off after-change writes if fsync is disabled (?) * Add WAL index reliability improvement to non-btree indexes * Reorder postgresql.conf WAL items in order of importance * Remove wal_files postgresql.conf option because WAL files are now recycled * Find proper defaults for postgresql.conf WAL entries * Add checkpoint_min_warning postgresql.conf option to warn about checkpoints that are too frequent OPTIMIZER/EXECUTOR * Improve Subplan list handling * Allow Subplans to use efficient joins(hash, merge) with upper variable * Improve dynamic memory allocation by introducing tuple-context memory allocation (Tom) * Add hash for evaluating GROUP BY aggregates * Nested FULL OUTER JOINs don't work (Tom) * Allow merge and hash joins on expressions not just simple variables (Tom) * Add new pg_proc cachable settings to specify whether function can be evaluated only once or once per query * Change FIXED_CHAR_SEL to 0.20 from 0.04 to give better selectivity (Bruce) * Make IN/NOT IN have similar performance to EXISTS/NOT EXISTS [exists] * Missing optimizer selectivities for date, r-tree, etc. [optimizer] * Allow ORDER BY ... LIMIT to select top values without sort or index MISCELLANEOUS * -Allow compression of log and meta data (Tom) * Do async I/O for faster random read-ahead of data * Get faster regex() code from Henry Spencer when it is available * Use mmap() rather than SYSV shared memory or to write WAL files (?) [mmap] * -Allow GUC configuration of maximum number of open files (Tom) * -Improve statistics storage in pg_class [performance] (Tom) * -Read pg_hba.conf only on postmaster startup or SIGHUP (Bruce) SOURCE CODE ----------- * Add use of 'const' for variables in source tree * -Convert remaining fprintf(stderr,...)/perror() to elog() (Peter E) * Fix problems with libpq non-blocking/async code [async] * -Merge global and template BKI files (Tom) * -Fix username/password length limits in all areas, e.g. pg_passwd * -Remove compile-time upper limit on number of backends (MAXBACKENDS) (Tom) * Make sure all block numbers are unsigned to increase maximum table size * Use BlockNumber rather than int where appropriate * Merge LockMethodCtl and LockMethodTable into one shared structure (Bruce) * HOLDER/HOLDERTAB rename to PROCLOCK/PROCLOCKTAG (Bruce) * Remove LockMethodTable.prio field, not used (Bruce) * Add version file format stamp to heap and other table types * Rename some /contrib modules from pg* to pg_* * Move some things from /contrib into main tree * Remove warnings created by -Wcast-align * Move platform-specific ps status display info from ps_status.c to ports * Allow ps status display to work on Solaris/SVr4-based systems * -Decide on spelling of indexes/indices (Peter E) * -Add mention of VACUUM, log rotation to Administrator's Guide (Tom, Bruce) * Make one version of simple_prompt() in code (Bruce, Tom) * Compile in syslog functionaility by default (?) * Modify regression tests to prevent failures do to minor numeric rounding * Remove USING clause from pg_get_indexdef() if index is btree (Bruce) * Use our own getopt() for FreeBSD/OpenBSD to allow --xxx flags (Bruce) * Add OpenBSD's getpeereid() call for local socket authentication (Bruce) * Improve access-permissions check on data directory in Cygwin (Tom) * Report failure to find readline or zlib at end of configure run * Add --port flag to regression tests * Increase identifier length (NAMEDATALEN) if small performance hit, perhaps to standard length of 128; change struct pgNotify to use pid first, breaks notify API; * Increase maximum number of function parameters if little wasted space * Add documentation for perl, including mention of DBI/DBD perl location * Add optional CRC checksum to heap and index pages * Change representation of whole-tuple parameters to functions * Evaluate AIX cs() spinlock macro for performance optimizations (Tatsuo) --------------------------------------------------------------------------- Developers who have claimed items are: -------------------------------------- * Billy is Billy G. Allie * Bruce is Bruce Momjian * D'Arcy is D'Arcy J.M. Cain * Edmund is Edmund Mergl * Hiroshi is Hiroshi Inoue * Karel is Karel Zak * Jan is Jan Wieck * Liam is Liam Stewart * Marc is Marc Fournier * Mark is Mark Hollomon * Marko is Marko Kreen * Michael is Michael Meskes * Oleg is Oleg Bartunov * Peter M is Peter T Mount * Peter E is Peter Eisentraut * Philip is Philip Warner * Ross is Ross J. Reedstrom * Ryan is Ryan Bradetich * Stephan is Stephan Szabo * Tatsuo is Tatsuo Ishii * Thomas is Thomas Lockhart * Tom is Tom Lane * TomH is Tom I Helbekkmo * Vadim is Vadim B. Mikheev