Commit Graph

48 Commits

Author SHA1 Message Date
Bruce Momjian
7e04792a1c Update copyright for 2014
Update all files in head, and files COPYRIGHT and legal.sgml in all back
branches.
2014-01-07 16:05:30 -05:00
Peter Eisentraut
264aa14a2f pg_upgrade: Split off pg_fatal() from pg_log()
This allows decorating pg_fatal() with noreturn compiler hints, leading
to better diagnostics.

Reviewed-by: Marko Tiikkaja <marko@joh.to>
2013-10-09 22:48:41 -04:00
Bruce Momjian
9af4159fce pgindent run for release 9.3
This is the first run of the Perl-based pgindent script.  Also update
pgindent instructions.
2013-05-29 16:58:43 -04:00
Simon Riggs
87d3b35a1c Fix pg_upgrade for 9.3 with data checksums.
Previous changes misconstrued pg_upgrade internals
causing build farm breakages.
2013-04-30 15:49:24 +01:00
Simon Riggs
be475a2473 Revert previous temporary patch 2013-04-30 15:32:10 +01:00
Simon Riggs
28377213bb Temporarily silence pg_upgrade's checksums check 2013-04-30 14:34:47 +01:00
Bruce Momjian
5286963066 pg_upgrade: Add checksum C comment
We might eventually allow checksum to no-checksum upgrades.
2013-04-17 17:39:19 -04:00
Simon Riggs
96ef3b8ff1 Allow I/O reliability checks using 16-bit checksums
Checksums are set immediately prior to flush out of shared buffers
and checked when pages are read in again. Hint bit setting will
require full page write when block is dirtied, which causes various
infrastructure changes. Extensive comments, docs and README.

WARNING message thrown if checksum fails on non-all zeroes page;
ERROR thrown but can be disabled with ignore_checksum_failure = on.

Feature enabled by an initdb option, since transition from option off
to option on is long and complex and has not yet been implemented.
Default is not to use checksums.

Checksum used is WAL CRC-32 truncated to 16-bits.

Simon Riggs, Jeff Davis, Greg Smith
Wide input and assistance from many community members. Thank you.
2013-03-22 13:54:07 +00:00
Alvaro Herrera
cd7d00ad65 Improve pg_upgrade commentary on multixact change
Per gripe from Andres Freund
2013-02-28 16:29:17 -03:00
Alvaro Herrera
8396447cdb Create libpgcommon, and move pg_malloc et al to it
libpgcommon is a new static library to allow sharing code among the
various frontend programs and backend; this lets us eliminate duplicate
implementations of common routines.  We avoid libpgport, because that's
intended as a place for porting issues; per discussion, it seems better
to keep them separate.

The first use case, and the only implemented by this patch, is pg_malloc
and friends, which many frontend programs were already using.

At the same time, we can use this to provide palloc emulation functions
for the frontend; this way, some palloc-using files in the backend can
also be used by the frontend cleanly.  To do this, we change palloc() in
the backend to be a function instead of a macro on top of
MemoryContextAlloc().  This was previously believed to cause loss of
performance, but this implementation has been tweaked by Tom and Andres
so that on modern compilers it provides a slight improvement over the
previous one.

This lets us clean up some places that were already with
localized hacks.

Most of the pg_malloc/palloc changes in this patch were authored by
Andres Freund. Zoltán Böszörményi also independently provided a form of
that.  libpgcommon infrastructure was authored by Álvaro.
2013-02-12 11:21:05 -03:00
Alvaro Herrera
34da700405 Use the catversion to distinguish old/new clusters
This makes 9.3 -> 9.3 upgrades work when they cross the commit that
added persistent multixacts; early 9.3 pg_controldata did not have the
required oldestMultiXact line, and so would fail to upgrade.

per Bruce Momjian
2013-01-24 12:36:18 -03:00
Alvaro Herrera
2494a9af4c Don't require oldestMultixact if server doesn't have it 2013-01-24 11:55:10 -03:00
Bruce Momjian
bd6aca8a77 pg_upgrade: report failed cluster name
When pg_upgrade can't find required pg_controldata information, report
_which_ cluster is failing, with this message:

	The %s cluster lacks some required control information:
2013-01-24 08:35:57 -05:00
Alvaro Herrera
0ac5ad5134 Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE".  These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE".  UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.

Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.

The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid.  Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates.  This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed.  pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.

Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header.  This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.

Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)

With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.

As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.

Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane.  There's probably room for several more tests.

There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it.  Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.

This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
	AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
	1290721684-sup-3951@alvh.no-ip.org
	1294953201-sup-2099@alvh.no-ip.org
	1320343602-sup-2290@alvh.no-ip.org
	1339690386-sup-8927@alvh.no-ip.org
	4FE5FF020200002500048A3D@gw.wicourts.gov
	4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 12:04:59 -03:00
Bruce Momjian
600250d0ed Improve pg_upgrade error report
If the cluster alignments don't match, output this suggestion:

	Likely one cluster is a 32-bit install, the other 64-bit
2013-01-18 09:26:55 -05:00
Bruce Momjian
bd61a623ac Update copyrights for 2013
Fully update git head, and update back branches in ./COPYRIGHT and
legal.sgml files.
2013-01-01 17:15:01 -05:00
Alvaro Herrera
9e26326ad6 pg_upgrade: fix off-by-one mistake in snprintf
snprintf counts trailing NUL towards the char limit.  Failing to account
for that was causing an invalid value to be passed to pg_resetxlog -l,
aborting the upgrade process.
2012-06-28 23:37:27 -04:00
Heikki Linnakangas
038f3a0509 Fix pg_upgrade, broken by the xlogid/segno -> 64-bit int refactoring.
The xlogid + segno representation of a particular WAL segment doesn't make
much sense in pg_resetxlog anymore, now that we don't use that anywhere
else. Use the WAL filename instead, since that's a convenient way to name a
particular WAL segment.

I did this partially for pg_resetxlog in the original xlogid/segno -> uint64
patch, but I neglected pg_upgrade and the docs. This should now be more
complete.
2012-06-26 07:49:02 +03:00
Bruce Momjian
927d61eeff Run pgindent on 9.2 source tree in preparation for first 9.3
commit-fest.
2012-06-10 15:20:04 -04:00
Bruce Momjian
717f6d6085 In pg_upgrade, add various logging improvements:
add ability to control permissions of created files
	have psql echo its queries for easier debugging
	output four separate log files, and delete them on success
	add -r/--retain option to keep log files after success
	make logs file append-only
	remove -g/-G/-l logging options
	sugggest tailing appropriate log file on failure
	enhance -v/--verbose behavior
2012-03-12 19:48:03 -04:00
Bruce Momjian
2127aac6ef In pg_upgrade, only lock the old cluster if link mode is used, and do it
right after we restore the schema (a common failure point), and right
before we do the link operation.

Per suggesgtions from Robert and ^!C^!^@lvaro
2012-03-05 21:20:06 -05:00
Bruce Momjian
e126958c2e Update copyright notices for year 2012. 2012-01-01 18:01:58 -05:00
Bruce Momjian
a3996754cc In pg_upgrade, improve popen() failure detection by checking for fgets()
failures.
2011-10-06 19:37:29 -04:00
Bruce Momjian
f1312b5ed3 Add postgres.h to *.c files for pg_upgrade, ltree, and btree_gist, and
remove from local *.h files.

Per suggestion from Alvaro.
2011-08-26 21:16:24 -04:00
Peter Eisentraut
912bc4f038 Make pg_upgrade output more consistent with project style
Add errno-based output to error messages where appropriate, reformat
blocks to about 72 characters per line, use spaces instead of tabs for
indentation, and other style adjustments.
2011-07-12 07:13:51 +03:00
Peter Eisentraut
76dfcb942f Postgres -> PostgreSQL in error message 2011-07-05 22:41:14 +03:00
Bruce Momjian
6560407c7d Pgindent run before 9.1 beta2. 2011-06-09 14:32:50 -04:00
Peter Eisentraut
0ee391b77a /contrib/foo -> contrib/foo
Since contrib is a relative directory specification, a leading slash
is inappropriate.
2011-05-19 00:30:31 +03:00
Bruce Momjian
6c19bd96bc Allow pg_upgrade to honor libpq environment variables. Add 'local'
checks for PGHOST and PGHOSTADDR.
2011-05-16 10:48:40 -04:00
Bruce Momjian
bf50caf105 pgindent run before PG 9.1 beta 1. 2011-04-10 11:42:00 -04:00
Bruce Momjian
5d950e3b0c Stamp copyrights for year 2011. 2011-01-01 13:18:15 -05:00
Bruce Momjian
16b5e08dec Use "upgrade" in preference over "migrate" in pg_upgrade messages and
documentation.  (Many were left over from the old pg_migrator naming.)
2010-12-15 07:11:31 -05:00
Tom Lane
71baff1786 Fix buffer overrun in pg_upgrade.
Problem reported, and cause identified, by Hernan Gonzalez.
2010-11-02 17:31:41 -04:00
Bruce Momjian
691a67b922 Rename pg_upgrade 'log' to 'log_opts', to avoid platform naming conflict. 2010-10-20 02:31:17 +00:00
Bruce Momjian
07456b45e6 Pgindent run on pg_upgrade source after restructuring. 2010-10-19 22:37:04 +00:00
Bruce Momjian
e13f7e9a71 Restructure the pg_upgrade code to use several global structures rather
than packing everything into 'ctx' and passing that to every function.
2010-10-19 21:38:34 +00:00
Bruce Momjian
9340fb80b1 In pg_upgrade, properly handle oids > 2^31 by using strtoul() internally
rather than atol().

Per report from Brian Hirt
2010-09-28 21:41:03 +00:00
Magnus Hagander
9f2e211386 Remove cvs keywords from all files. 2010-09-20 22:08:53 +02:00
Bruce Momjian
c5d94a34fb Modify pg_upgrade to set/restore all environment variables related to
collation/encoding to match English when reading controldata.  This now
matches the English variable setting used by pg_regress.c.

Backpatch to 9.0.X.
2010-09-07 14:10:30 +00:00
Bruce Momjian
239d769e7e pgindent run for 9.0, second run 2010-07-06 19:19:02 +00:00
Bruce Momjian
da254e3d16 Make pg_upgrade copyrights just 2010, not 2010-2010. 2010-07-03 16:33:15 +00:00
Bruce Momjian
de255a6eb1 Add copyrights to pg_upgrade and pg_upgrade_tools files, per Tom. 2010-07-03 16:25:01 +00:00
Bruce Momjian
ccbe0c14e6 Add CVS tags to pg_upgrade and pg_upgrade_support files, per request
from Tom.
2010-07-03 14:23:14 +00:00
Tom Lane
600fc1dc9d Get rid of some more stuff that duplicates c.h or port.h. 2010-07-01 15:52:52 +00:00
Tom Lane
0546906064 Remove another unsafe use of random data as a format string. 2010-05-14 00:39:51 +00:00
Tom Lane
8eac39780d Get rid of unsafe sprintf and snprintf usages. Per compiler warnings. 2010-05-14 00:32:21 +00:00
Bruce Momjian
36d3afd2d4 Remove all mentions of EnterpriseDB Advanced Server from pg_upgrade;
EDB must maintain their own patch set for this.
2010-05-13 22:51:00 +00:00
Bruce Momjian
c2e9b2f288 Add pg_upgrade to /contrib; will be in 9.0 beta2.
Add documentation.

Supports migration from PG 8.3 and 8.4.
2010-05-12 02:19:11 +00:00