mirror/postgresql
2
0
Fork 0
mirror of https://git.postgresql.org/git/postgresql.git synced 2024-12-27 08:39:28 +08:00
Code Issues Packages Projects Releases Wiki Activity
46,029 Commits 37 Branches 638 Tags 396 MiB
C 85.6%
PLpgSQL 5.9%
Perl 4.2%
Yacc 1.2%
Makefile 0.7%
Other 2.3%
f69c959df0
Go to file
Tomas Vondra f69c959df0 Do not decode TOAST data for table rewrites 
During table rewrites (VACUUM FULL and CLUSTER), the main heap is logged
using XLOG / FPI records, and thus (correctly) ignored in decoding.
But the associated TOAST table is WAL-logged as plain INSERT records,
and so was logically decoded and passed to reorder buffer.

That has severe consequences with TOAST tables of non-trivial size.
Firstly, reorder buffer has to keep all those changes, possibly spilling
them to a file, incurring I/O costs and disk space.

Secondly, ReoderBufferCommit() was stashing all those TOAST chunks into
a hash table, which got discarded only after processing the row from the
main heap.  But as the main heap is not decoded for rewrites, this never
happened, so all the TOAST data accumulated in memory, resulting either
in excessive memory consumption or OOM.

The fix is simple, as commit e9edc1ba already introduced infrastructure
(namely HEAP_INSERT_NO_LOGICAL flag) to skip logical decoding of TOAST
tables, but it only applied it to system tables.  So simply use it for
all TOAST data in raw_heap_insert().

That would however solve only the memory consumption issue - the TOAST
changes would still be decoded and added to the reorder buffer, and
spilled to disk (although without TOAST tuple data, so much smaller).
But we can solve that by tweaking DecodeInsert() to just ignore such
INSERT records altogether, using XLH_INSERT_CONTAINS_NEW_TUPLE flag,
instead of skipping them later in ReorderBufferCommit().

Review: Masahiko Sawada
Discussion: https://www.postgresql.org/message-id/flat/1a17c643-e9af-3dba-486b-fbe31bc1823a%402ndquadrant.com
Backpatch: 9.4-, where logical decoding was introduced
2018-11-28 01:43:08 +01:00
config Update config/ax_pthread.m4 to latest upstream version. 2018-11-19 15:05:33 -05:00
contrib Revert all new recent changes to add PGXS options for TAP and isolation 2018-11-26 11:12:11 +09:00
doc doc: fix wording for plpgsql, add "and" 2018-11-26 19:41:29 -05:00
src Do not decode TOAST data for table rewrites 2018-11-28 01:43:08 +01:00
.dir-locals.el Update documentation editor setup instructions 2018-07-13 21:23:41 +02:00
.gitattributes
.gitignore Support for optimizing and emitting code in LLVM JIT provider. 2018-03-22 11:05:22 -07:00
aclocal.m4 Fix configure's AC_CHECK_DECLS tests to work correctly with clang. 2018-11-19 12:01:47 -05:00
configure Update config/ax_pthread.m4 to latest upstream version. 2018-11-19 15:05:33 -05:00
configure.in Postpone LLVM-related uses of AC_CHECK_DECLS. 2018-11-19 12:43:20 -05:00
COPYRIGHT Update copyright for 2018 2018-01-02 23:30:12 -05:00
GNUmakefile.in Remove unwanted "garbage cleanup" logic in Makefiles. 2018-08-08 14:32:29 -04:00
HISTORY
Makefile
README
README.git

README 

PostgreSQL Database Management System
=====================================

This directory contains the source code distribution of the PostgreSQL
database management system.

PostgreSQL is an advanced object-relational database management system
that supports an extended subset of the SQL standard, including
transactions, foreign keys, subqueries, triggers, user-defined types
and functions.  This distribution also contains C language bindings.

PostgreSQL has many language interfaces, many of which are listed here:

	https://www.postgresql.org/download

See the file INSTALL for instructions on how to build and install
PostgreSQL.  That file also lists supported operating systems and
hardware platforms and contains information regarding any other
software packages that are required to build or run the PostgreSQL
system.  Copyright and license information can be found in the
file COPYRIGHT.  A comprehensive documentation set is included in this
distribution; it can be read as described in the installation
instructions.

The latest version of this software may be obtained at
https://www.postgresql.org/download/.  For more information look at our
web site located at https://www.postgresql.org/.