Commit Graph

100 Commits

Author SHA1 Message Date
Bruce Momjian
2a2e40aaaa pg_upgrade: dramatically reduce memory consumption
Report from Jeff Janes
2014-02-12 16:35:24 -05:00
Tom Lane
cd3e0071b8 Allow unrecognized encoding names in locales, as long as they're the same.
The buildfarm says commit 58274728fb doesn't
work so well on Windows.  This is because the encoding part of Windows
locale names can be just a code page number, eg "1252", which we don't
consider to be a valid encoding name.  Add a check to accept encoding
parts that are case-insensitively string equal; this at least ensures
that the new code doesn't reject any cases that the old code allowed.
2014-01-31 02:03:52 -05:00
Tom Lane
58274728fb Be forgiving of variant spellings of locale names in pg_upgrade.
Even though the server tries to canonicalize stored locale names, the
platform often doesn't cooperate, so it's entirely possible that one DB
thinks its locale is, say, "en_US.UTF-8" while the other has "en_US.utf8".
Rather than failing, we should try to allow this where it's clearly OK.

There is already pretty robust encoding lookup in encnames.c, so make
use of that to compare the encoding parts of the names.  The locale
identifier parts are just compared case-insensitively, which we were
already doing.  The major problem known to exist in the field is variant
encoding-name spellings, so hopefully this will be Good Enough.  If not,
we can try being even laxer.

Pavel Raiskup, reviewed by Rushabh Lathia
2014-01-30 19:07:06 -05:00
Tom Lane
41e364ec67 Fix potential coredump on bad locale value in pg_upgrade.
Thinko in error report (and a typo in the message text, too).  We're
failing anyway, but it would be good to print something useful first.
Noted while reviewing a patch to make pg_upgrade's locale code laxer.
2014-01-30 18:10:24 -05:00
Bruce Momjian
7e04792a1c Update copyright for 2014
Update all files in head, and files COPYRIGHT and legal.sgml in all back
branches.
2014-01-07 16:05:30 -05:00
Peter Eisentraut
55c3d86a2a pg_upgrade: Fix some whitespace oddities 2013-11-16 11:35:44 -05:00
Tom Lane
2c66f9924c Replace pg_asprintf() with psprintf().
This eliminates an awkward coding pattern that's also unnecessarily
inconsistent with backend coding.  psprintf() is now the thing to
use everywhere.
2013-10-22 19:40:26 -04:00
Peter Eisentraut
5b6d08cd29 Add use of asprintf()
Add asprintf(), pg_asprintf(), and psprintf() to simplify string
allocation and composition.  Replacement implementations taken from
NetBSD.

Reviewed-by: Álvaro Herrera <alvherre@2ndquadrant.com>
Reviewed-by: Asif Naeem <anaeem.it@gmail.com>
2013-10-13 00:09:18 -04:00
Peter Eisentraut
264aa14a2f pg_upgrade: Split off pg_fatal() from pg_log()
This allows decorating pg_fatal() with noreturn compiler hints, leading
to better diagnostics.

Reviewed-by: Marko Tiikkaja <marko@joh.to>
2013-10-09 22:48:41 -04:00
Bruce Momjian
f85136106d pg_upgrade: pass username to analyze script
If -U (user) is specified, pass the username into the created analyze
script.
Per request from Ray Stell
2013-06-28 19:11:51 -04:00
Bruce Momjian
9af4159fce pgindent run for release 9.3
This is the first run of the Perl-based pgindent script.  Also update
pgindent instructions.
2013-05-29 16:58:43 -04:00
Bruce Momjian
203d8ae2db pg_upgrade: don't copy/link files for invalid indexes
Now that pg_dump no longer dumps invalid indexes, per commit
683abc73df, have pg_upgrade also skip
them.  Previously pg_upgrade threw an error if invalid indexes existed.

Backpatch to 9.2, 9.1, and 9.0 (where pg_upgrade was added to git)
2013-03-30 22:20:53 -04:00
Bruce Momjian
4765dd7921 pg_upgrade: conditionally create cluster delete script
If users create tablespaces inside the old cluster directory, it is
impossible for the delete script to delete _only_ the old cluster files,
so don't create a script in that case, and issue a message to the user.
2013-02-14 10:53:03 -05:00
Alvaro Herrera
8396447cdb Create libpgcommon, and move pg_malloc et al to it
libpgcommon is a new static library to allow sharing code among the
various frontend programs and backend; this lets us eliminate duplicate
implementations of common routines.  We avoid libpgport, because that's
intended as a place for porting issues; per discussion, it seems better
to keep them separate.

The first use case, and the only implemented by this patch, is pg_malloc
and friends, which many frontend programs were already using.

At the same time, we can use this to provide palloc emulation functions
for the frontend; this way, some palloc-using files in the backend can
also be used by the frontend cleanly.  To do this, we change palloc() in
the backend to be a function instead of a macro on top of
MemoryContextAlloc().  This was previously believed to cause loss of
performance, but this implementation has been tweaked by Tom and Andres
so that on modern compilers it provides a slight improvement over the
previous one.

This lets us clean up some places that were already with
localized hacks.

Most of the pg_malloc/palloc changes in this patch were authored by
Andres Freund. Zoltán Böszörményi also independently provided a form of
that.  libpgcommon infrastructure was authored by Álvaro.
2013-02-12 11:21:05 -03:00
Bruce Momjian
a9ceaa53be pg_upgrade: detect stale postmaster.pid lock files
If the postmaster.pid lock file exists, try starting/stopping the
cluster to check if the lock file is valid.

Per request from Tom.
2013-01-24 15:20:40 -05:00
Bruce Momjian
a89c46f9bc Allow parallel copy/link in pg_upgrade
This patch implements parallel copying/linking of files by tablespace
using the --jobs option in pg_upgrade.
2013-01-09 08:57:47 -05:00
Bruce Momjian
bd61a623ac Update copyrights for 2013
Fully update git head, and update back branches in ./COPYRIGHT and
legal.sgml files.
2013-01-01 17:15:01 -05:00
Bruce Momjian
e95c4bd113 Fix pg_upgrade for invalid indexes
All versions of pg_upgrade upgraded invalid indexes caused by CREATE
INDEX CONCURRENTLY failures and marked them as valid.  The patch adds a
check to all pg_upgrade versions and throws an error during upgrade or
--check.

Backpatch to 9.2, 9.1, 9.0.  Patch slightly adjusted.
2012-12-11 15:09:22 -05:00
Bruce Momjian
12ee6ec71f In pg_upgrade, dump each database separately and use
--single-transaction to restore each database schema.  This yields
performance improvements for databases with many tables.  Also, remove
split_old_dump() as it is no longer needed.
2012-11-30 16:30:13 -05:00
Bruce Momjian
16e1ae77f9 In pg_upgrade, fix a few place that used maloc/free rather than
pg_malloc/pg_free.
2012-11-24 22:12:39 -05:00
Bruce Momjian
ce75457949 In pg_upgrade, use full path name for analyze_new_cluster.sh script.
Backpatch to 9.2.
2012-10-02 21:18:43 -04:00
Bruce Momjian
b61837a49f In pg_upgrade, try to convert the locale names to canonical form before
comparison;  also report the old/new values if they don't match.

Backpatch to 9.2.
2012-10-02 11:42:34 -04:00
Andrew Dunstan
2042185baf Fix transcription error. 2012-09-04 09:39:49 -04:00
Andrew Dunstan
0829c7087e Fix command echoing in pg_upgade's analyze script for Windows. 2012-09-04 05:49:22 -04:00
Andrew Dunstan
2f0c7d5854 Indent fix_path_separator() header properly. 2012-09-03 22:59:19 -04:00
Bruce Momjian
a80b8037cf In pg_upgrade, pull the port number from postmaster.pid, like we do for
socket location.  Also, prevent putting the socket in the current
directory for pre-9.1 servers in live check and non-live check mode,
because pre-9.1 pg_ctl -w can't handle it.

Backpatch to 9.2.
2012-09-03 22:15:09 -04:00
Andrew Dunstan
504aeea692 Use correct path separator for Windows builtin commands.
pg_upgrade produces a platform-specific script to remove the old
directory, but on Windows it has not been making sure that the
paths it writes as arguments for rmdir and del use the backslash
path separator, which will cause these scripts to fail.

The fix is backpatched to Release 9.0.
2012-09-03 18:06:47 -04:00
Tom Lane
f763b77193 Fix pg_upgrade to cope with non-default unix_socket_directory scenarios.
When starting either an old or new postmaster, force it to place its Unix
socket in the current directory.  This makes it even harder for accidental
connections to occur during pg_upgrade, and also works around some
scenarios where the default socket location isn't usable.  (For example,
if the default location is something other than "/tmp", it might not exist
during "make check".)

When checking an already-running old postmaster, find out its actual socket
directory location from postmaster.pid, if possible.  This dodges problems
with an old postmaster having a configured location different from the
default built into pg_upgrade's libpq.  We can't find that out if the old
postmaster is pre-9.1, so also document how to cope with such scenarios
manually.

In support of this, centralize handling of the connection-related command
line options passed to pg_upgrade's subsidiary programs, such as pg_dump.
This should make future changes easier.

Bruce Momjian and Tom Lane
2012-09-03 13:52:40 -04:00
Alvaro Herrera
088c065ce8 pg_upgrade: Fix exec_prog API to be less flaky
The previous signature made it very easy to pass something other than
the printf-format specifier in the corresponding position, without any
warning from the compiler.

While at it, move some of the escaping, redirecting and quoting
responsibilities from the callers into exec_prog() itself.  This makes
the callsites cleaner.
2012-08-27 14:29:14 -04:00
Bruce Momjian
fbcfa90bb8 Fix pg_upgrade file share violation on Windows created by the commit
4741e9afb9.  This was done by adding an
optional second log file parameter to exec_prog(), and closing and
reopening the log file between system() calls.

Backpatch to 9.2.
2012-08-07 13:10:44 -04:00
Bruce Momjian
b9212e379c In pg_upgrade, verify that the install user has the same oid on both
clusters, and make sure the new cluster has no additional users.

Backpatch to 9.1.
2012-06-13 12:19:18 -04:00
Bruce Momjian
927d61eeff Run pgindent on 9.2 source tree in preparation for first 9.3
commit-fest.
2012-06-10 15:20:04 -04:00
Bruce Momjian
02b183acb4 In pg_upgrade, remove dependency on pg_config, as that might not be in
the non-development install.  Instead, use the LOAD mechanism to check
for the pg_upgrade_support shared object, like we do for other shared
object checks.

Backpatch to 9.1.

Report from Àlvaro
2012-03-19 09:31:50 -04:00
Bruce Momjian
ced53c38d7 In pg_upgrade, move new echo quote define into include file. 2012-03-17 09:56:56 -04:00
Bruce Momjian
faec2815c8 In pg_upgrade, create a script to incrementally generate more accurate
optimizer statistics so the cluster can be made available sooner.
2012-03-16 18:54:23 -04:00
Bruce Momjian
717f6d6085 In pg_upgrade, add various logging improvements:
add ability to control permissions of created files
	have psql echo its queries for easier debugging
	output four separate log files, and delete them on success
	add -r/--retain option to keep log files after success
	make logs file append-only
	remove -g/-G/-l logging options
	sugggest tailing appropriate log file on failure
	enhance -v/--verbose behavior
2012-03-12 19:48:03 -04:00
Bruce Momjian
2127aac6ef In pg_upgrade, only lock the old cluster if link mode is used, and do it
right after we restore the schema (a common failure point), and right
before we do the link operation.

Per suggesgtions from Robert and ^!C^!^@lvaro
2012-03-05 21:20:06 -05:00
Bruce Momjian
ecf7a2ea38 Add pg_upgrade C comment about why we check all relkinds for regtypes. 2012-01-19 16:04:34 -05:00
Bruce Momjian
e126958c2e Update copyright notices for year 2012. 2012-01-01 18:01:58 -05:00
Bruce Momjian
0e8f6bf0e7 In pg_upgrade, allow tables using regclass to be upgraded because we
preserve pg_class oids since PG 9.0.
2011-12-05 16:45:19 -05:00
Bruce Momjian
32fb4546e3 Remove ENABLE_SAME_CATVERSION_UPGRADES as unnecessary. 2011-11-25 19:08:41 -05:00
Bruce Momjian
5df1403b0f Add pg_upgrade ENABLE_SAME_CATVERSION_UPGRADES macro for testing to
allow upgrades of the same catalog version.  (Doesn't work for
tablespaces, as indicated by C comment.)
2011-11-23 22:51:45 -05:00
Bruce Momjian
a50d860ae1 Allow pg_upgrade to upgrade an old cluster that doesn't have a
'postgres' database.
2011-11-01 13:49:48 -04:00
Bruce Momjian
9846dcfb9e Clarify pg_upgrade error message that the 'postgres' database must exist
in the old cluster.
2011-10-28 08:09:15 -04:00
Bruce Momjian
a3996754cc In pg_upgrade, improve popen() failure detection by checking for fgets()
failures.
2011-10-06 19:37:29 -04:00
Bruce Momjian
3ced32d24e In pg_upgrade, disallow migration of 8.3 clusters using contrib/ltree
because its internal format was changed in 8.4.

Backpatch to 9.0 and 9.1.

Report by depesz, diagnosis by Tom.
2011-09-07 14:43:12 -04:00
Tom Lane
731ebb64b7 Fix not-backwards-compatible pg_upgrade test for prepared transactions.
There's no reason for this test to use the undocumented pg_prepared_xact()
function, when it can use the stable API pg_prepared_xacts instead.
Fixes breakage against 8.3, as reported by Justin Arnold.
2011-08-30 17:15:00 -04:00
Bruce Momjian
f1312b5ed3 Add postgres.h to *.c files for pg_upgrade, ltree, and btree_gist, and
remove from local *.h files.

Per suggestion from Alvaro.
2011-08-26 21:16:24 -04:00
Bruce Momjian
17a16eeb7c In pg_upgrade, fix the -l/log option to work on Windows.
Also, double-quote the log file name in all places, to allow (on all
platforms) log file names with spaces.

Back patch to 9.0 and 9.1.
2011-07-20 18:31:03 -04:00
Peter Eisentraut
912bc4f038 Make pg_upgrade output more consistent with project style
Add errno-based output to error messages where appropriate, reformat
blocks to about 72 characters per line, use spaces instead of tabs for
indentation, and other style adjustments.
2011-07-12 07:13:51 +03:00