Commit Graph

382 Commits

Author SHA1 Message Date
Bruce Momjian
613c6d26bd Fix incorrect error message reported for non-existent users
Previously, lookups of non-existent user names could return "Success";
it will now return "User does not exist" by resetting errno.  This also
centralizes the user name lookup code in libpgport.

Report and analysis by Nicolas Marchildon;  patch by me
2013-12-18 12:16:21 -05:00
Bruce Momjian
86ef4796f5 build: pass EXTRA_REGRESS_OPTS to secondary regression tests
Christoph Berg
2013-12-04 10:14:45 -05:00
Peter Eisentraut
dfd5151c58 Fix whitespace 2013-12-03 22:57:08 -05:00
Bruce Momjian
9e0a97f1c8 libpq: change PQconndefaults() to ignore invalid service files
Previously missing or invalid service files returned NULL.  Also fix
pg_upgrade to report "out of memory" for a null return from
PQconndefaults().

Patch by Steve Singer, rewritten by me
2013-12-03 11:12:25 -05:00
Bruce Momjian
e7d56aee2d pg_upgrade: Handle default_transaction_read_only settings
Setting default_transaction_read_only=true could prevent pg_upgrade from
completing, so prepend default_transaction_read_only=false to
PGOPTIONS.
2013-11-30 16:50:33 -05:00
Bruce Momjian
dbd786bc4f pg_upgrade: avoid ALTER COLUMN TYPE on inherited columns
This only affects upgrades from 8.3 currently, and is harmless as the
child just generates an error in the script, but we should get it right
in case we ever need this for more complex uses.

Per report from Peter Eisentraut
2013-11-19 15:00:49 -05:00
Peter Eisentraut
bd5a9a50e4 pg_upgrade: Report full disk better
Previously, pg_upgrade would abort copy_file() on a short write without
setting errno, which the caller would report as an error with the
message "Success".  We assume ENOSPC in that case, as we do elsewhere in
the code.  Also set errno in some other error cases in copy_file() to
avoid bogus "Success" error messages.

This was broken in 6b711cf37c, so 9.2 and
before are OK.
2013-11-18 22:01:59 -05:00
Peter Eisentraut
55c3d86a2a pg_upgrade: Fix some whitespace oddities 2013-11-16 11:35:44 -05:00
Andrew Dunstan
c737a2e564 Work around NetBSD shell issue in pg_upgrade test script.
The NetBSD shell apparently returns non-zero from an unset command if
the variable is already unset. This matters when, as in pg_upgrade's
test.sh, we are working under 'set -e'. To protect against this, we
first set the PG variables to an empty string before unsetting them
completely.

Error found on buildfarm member coypu, solution from Rémi Zara.
2013-10-28 11:45:50 -04:00
Tom Lane
2c66f9924c Replace pg_asprintf() with psprintf().
This eliminates an awkward coding pattern that's also unnecessarily
inconsistent with backend coding.  psprintf() is now the thing to
use everywhere.
2013-10-22 19:40:26 -04:00
Peter Eisentraut
5b6d08cd29 Add use of asprintf()
Add asprintf(), pg_asprintf(), and psprintf() to simplify string
allocation and composition.  Replacement implementations taken from
NetBSD.

Reviewed-by: Álvaro Herrera <alvherre@2ndquadrant.com>
Reviewed-by: Asif Naeem <anaeem.it@gmail.com>
2013-10-13 00:09:18 -04:00
Peter Eisentraut
264aa14a2f pg_upgrade: Split off pg_fatal() from pg_log()
This allows decorating pg_fatal() with noreturn compiler hints, leading
to better diagnostics.

Reviewed-by: Marko Tiikkaja <marko@joh.to>
2013-10-09 22:48:41 -04:00
Bruce Momjian
ff2a1f5e84 pg_upgrade: more C comment fixes 2013-09-23 11:12:09 -04:00
Bruce Momjian
f7cf5fa262 pg_upgrade: fix C comment typo 2013-09-23 11:06:19 -04:00
Bruce Momjian
ac76ec27b9 pg_upgrade: shut down server after auth failure
Register atexit() server shutdown if pg_ctl successfully started the
server, but we can't connect to it.

Backpatch to 9.3.

Pavel Raiskup
2013-08-16 11:09:09 -04:00
Bruce Momjian
000b65fd2b pg_upgrade: clarify C comment about Windows thread struct pointers
Backpatch to 9.3 to keep source trees consistent.
2013-07-30 09:23:31 -04:00
Bruce Momjian
815fcd050f pg_upgrade: fix -j race condition on Windows
Pg_Upgrade cannot write the command string to the log file and then call
system() to write to the same file without causing occasional file-share
errors on Windows.  So instead, write the command string to the log file
after system(), in those cases.
Backpatch to 9.3.
2013-07-27 15:00:58 -04:00
Bruce Momjian
76a7650c40 pg_upgrade: adjust umask() calls
Since pg_upgrade -j on Windows uses threads, calling umask()
before/after opening a file via fopen_priv() is no longer possible, so
set umask() as we enter the thread-creating loop, and reset it on exit.
Also adjust internal fopen_priv() calls to just use fopen().
Backpatch to 9.3beta.
2013-07-25 11:33:15 -04:00
Bruce Momjian
e4c6cccd8c pg_upgrade: fix initialization of thread argument
Reorder initialization of thread argument marker to it happens before
reap_child() is called.
Backpatch to 9.3.
2013-07-24 22:01:31 -04:00
Bruce Momjian
910d3a458c pg_upgrade: more Windows parallel/-j fixes
More fixes to handle Windows thread parameter passing.
Backpatch to 9.3 beta.
Patch originally from Andrew Dunstan
2013-07-24 13:15:47 -04:00
Bruce Momjian
d7de6a4790 pg_upgrade: fix parallel/-j crash on Windows
This fixes the problem of passing the wrong function pointer when doing
parallel copy/link operations on Windows.
Backpatched to 9.3beta.
Found and patch supplied by Andrew Dunstan
2013-07-24 10:00:46 -04:00
Fujii Masao
2ef085d0e6 Get rid of pg_class.reltoastidxid.
Treat TOAST index just the same as normal one and get the OID
of TOAST index from pg_index but not pg_class.reltoastidxid.
This change allows us to handle multiple TOAST indexes, and
which is required infrastructure for upcoming
REINDEX CONCURRENTLY feature.

Patch by Michael Paquier, reviewed by Andres Freund and me.
2013-07-04 03:24:09 +09:00
Bruce Momjian
cce5d851ed pg_upgrade: revert changing '' to ""
On the command line, GUC option strings are handled by the guc parser,
not by the shell parser, so '' is the proper way to represent a
zero-length string.  This reverts commit
3132a9b7ab.
2013-07-02 10:29:49 -04:00
Bruce Momjian
3132a9b7ab pg_upgrade: use "" rather than '', for Windows
If we ever support unix sockets on Windows, we should use "" rather than
'' for zero-length strings on the command-line, so use that.
2013-07-01 14:45:45 -04:00
Bruce Momjian
f85136106d pg_upgrade: pass username to analyze script
If -U (user) is specified, pass the username into the created analyze
script.
Per request from Ray Stell
2013-06-28 19:11:51 -04:00
Bruce Momjian
e029275cde pg_upgrade: remove -h option
-h (help) is not needed;  pg_upgrade already supports --help and -?,
which is consistent with other tools.
2013-06-28 18:10:13 -04:00
Bruce Momjian
fb771f9086 pg_upgrade: trim down --help and doc option descriptions
Previous code had old/new prefixes on option values, e.g.
--old-datadir=OLDDATADIR.  Remove them, for simplicity;  now:
--old-datadir=DATADIR.  Also update docs to do the same.
2013-06-28 18:01:49 -04:00
Bruce Momjian
457d6cf049 pg_upgrade: change -u to -U, for consistency
Change -u (user) option to -U, for consistency with other tools like
pg_dump and psql.  Also expand --user to --username, again for
consistency.
BACKWARD INCOMPATIBILITY
2013-06-28 17:27:43 -04:00
Stephen Frost
c9fc28a7f1 Minor spelling fixes
Fix a few spelling mistakes.

Per bug report #8193 from Lajos Veres.
2013-06-01 10:18:59 -04:00
Stephen Frost
551938ae22 Post-pgindent cleanup
Make slightly better decisions about indentation than what pgindent
is capable of.  Mostly breaking out long function calls into one
line per argument, with a few other minor adjustments.

No functional changes- all whitespace.
pgindent ran cleanly (didn't change anything) after.
Passes all regressions.
2013-06-01 09:38:15 -04:00
Bruce Momjian
9af4159fce pgindent run for release 9.3
This is the first run of the Perl-based pgindent script.  Also update
pgindent instructions.
2013-05-29 16:58:43 -04:00
Peter Eisentraut
d2b920cded pg_upgrade: Replace tabs in output string constants by spaces 2013-05-13 21:42:19 -04:00
Tom Lane
8cade04c10 Fix buildfarm incompatibility in updated pg_upgrade test script.
Looks like some versions of the buildfarm script try to set the port via
--port in $EXTRA_REGRESS_OPTS.  Override that ...
2013-05-11 16:14:19 -04:00
Tom Lane
7e2b1c03ce Make pg_upgrade's test script attempt to select a non-conflicting port.
Previously, the port number used in this test script was hard-wired at
pg_upgrade's default of 50432; which is not so great because parallel build
runs might conflict.  Commit 3d53173e20
removed this setting for the postmasters started by the script proper
(not by pg_upgrade), which didn't do anything to fix that problem and also
guaranteed a failure if there was a live postmaster at the build's default
port number.  Instead, select a non-conflicting temporary port number in
the same way that pg_regress.c does.  (Its method isn't entirely
bulletproof, but given the lack of complaints I'm not going to worry
about that today.)

In passing, unset MAKEFLAGS and MAKELEVEL to avoid problems with the
script's internal invocations of make, for the same reason pg_regress.c
does: it could cause problems in a parallel make.
2013-05-11 14:22:18 -04:00
Tom Lane
1c36700e9e Use pg_dump's --quote-all-identifiers option in pg_upgrade.
This helps guard against changes in the set of reserved keywords from
one version to another.  In theory it should only be an issue if we
de-reserve a keyword in a newer release, since that can create the type
of problem shown in bug #8128.

Back-patch to 9.1 where the --quote-all-identifiers option was added.
2013-05-09 17:34:34 -04:00
Simon Riggs
87d3b35a1c Fix pg_upgrade for 9.3 with data checksums.
Previous changes misconstrued pg_upgrade internals
causing build farm breakages.
2013-04-30 15:49:24 +01:00
Simon Riggs
be475a2473 Revert previous temporary patch 2013-04-30 15:32:10 +01:00
Simon Riggs
28377213bb Temporarily silence pg_upgrade's checksums check 2013-04-30 14:34:47 +01:00
Peter Eisentraut
3d53173e20 pg_upgrade: Remove PGPORT handling from test suite
This code was left over from when pg_upgrade paid attention to PGPORT.
Now it would only affects the regression test run before the test run of
pg_upgrade.  You can still set PGPORT for that, but there is no reason
to have the test driver default it to 50432.
2013-04-29 22:17:29 -04:00
Bruce Momjian
5286963066 pg_upgrade: Add checksum C comment
We might eventually allow checksum to no-checksum upgrades.
2013-04-17 17:39:19 -04:00
Bruce Momjian
203d8ae2db pg_upgrade: don't copy/link files for invalid indexes
Now that pg_dump no longer dumps invalid indexes, per commit
683abc73df, have pg_upgrade also skip
them.  Previously pg_upgrade threw an error if invalid indexes existed.

Backpatch to 9.2, 9.1, and 9.0 (where pg_upgrade was added to git)
2013-03-30 22:20:53 -04:00
Andrew Dunstan
67eb3e5075 Avoid moving data directory in upgrade testing.
Windows sometimes gets upset if we rename a large directory and then try
to use the old name quickly, as seen in occasional buildfarm failures.
So we avoid that by building the old version in the intended
destination in the first place instead of renaming it, similar to the
change made for the same reason in commit b7f8465c.
2013-03-30 12:44:29 -04:00
Simon Riggs
96ef3b8ff1 Allow I/O reliability checks using 16-bit checksums
Checksums are set immediately prior to flush out of shared buffers
and checked when pages are read in again. Hint bit setting will
require full page write when block is dirtied, which causes various
infrastructure changes. Extensive comments, docs and README.

WARNING message thrown if checksum fails on non-all zeroes page;
ERROR thrown but can be disabled with ignore_checksum_failure = on.

Feature enabled by an initdb option, since transition from option off
to option on is long and complex and has not yet been implemented.
Default is not to use checksums.

Checksum used is WAL CRC-32 truncated to 16-bits.

Simon Riggs, Jeff Davis, Greg Smith
Wide input and assistance from many community members. Thank you.
2013-03-22 13:54:07 +00:00
Kevin Grittner
3bf3ab8c56 Add a materialized view relations.
A materialized view has a rule just like a view and a heap and
other physical properties like a table.  The rule is only used to
populate the table, references in queries refer to the
materialized data.

This is a minimal implementation, but should still be useful in
many cases.  Currently data is only populated "on demand" by the
CREATE MATERIALIZED VIEW and REFRESH MATERIALIZED VIEW statements.
It is expected that future releases will add incremental updates
with various timings, and that a more refined concept of defining
what is "fresh" data will be developed.  At some point it may even
be possible to have queries use a materialized in place of
references to underlying tables, but that requires the other
above-mentioned features to be working first.

Much of the documentation work by Robert Haas.
Review by Noah Misch, Thom Brown, Robert Haas, Marko Tiikkaja
Security review by KaiGai Kohei, with a decision on how best to
implement sepgsql still pending.
2013-03-03 18:23:31 -06:00
Alvaro Herrera
cd7d00ad65 Improve pg_upgrade commentary on multixact change
Per gripe from Andres Freund
2013-02-28 16:29:17 -03:00
Bruce Momjian
4765dd7921 pg_upgrade: conditionally create cluster delete script
If users create tablespaces inside the old cluster directory, it is
impossible for the delete script to delete _only_ the old cluster files,
so don't create a script in that case, and issue a message to the user.
2013-02-14 10:53:03 -05:00
Bruce Momjian
74205266d4 Fix pg_upgrade log file cleanup code
Recent pg_upgrade parallel improvements introduced a bug that prevented
cleanup of per-database log files.
2013-02-14 00:04:15 -05:00
Alvaro Herrera
8396447cdb Create libpgcommon, and move pg_malloc et al to it
libpgcommon is a new static library to allow sharing code among the
various frontend programs and backend; this lets us eliminate duplicate
implementations of common routines.  We avoid libpgport, because that's
intended as a place for porting issues; per discussion, it seems better
to keep them separate.

The first use case, and the only implemented by this patch, is pg_malloc
and friends, which many frontend programs were already using.

At the same time, we can use this to provide palloc emulation functions
for the frontend; this way, some palloc-using files in the backend can
also be used by the frontend cleanly.  To do this, we change palloc() in
the backend to be a function instead of a macro on top of
MemoryContextAlloc().  This was previously believed to cause loss of
performance, but this implementation has been tweaked by Tom and Andres
so that on modern compilers it provides a slight improvement over the
previous one.

This lets us clean up some places that were already with
localized hacks.

Most of the pg_malloc/palloc changes in this patch were authored by
Andres Freund. Zoltán Böszörményi also independently provided a form of
that.  libpgcommon infrastructure was authored by Álvaro.
2013-02-12 11:21:05 -03:00
Bruce Momjian
a9ceaa53be pg_upgrade: detect stale postmaster.pid lock files
If the postmaster.pid lock file exists, try starting/stopping the
cluster to check if the lock file is valid.

Per request from Tom.
2013-01-24 15:20:40 -05:00
Alvaro Herrera
34da700405 Use the catversion to distinguish old/new clusters
This makes 9.3 -> 9.3 upgrades work when they cross the commit that
added persistent multixacts; early 9.3 pg_controldata did not have the
required oldestMultiXact line, and so would fail to upgrade.

per Bruce Momjian
2013-01-24 12:36:18 -03:00