Commit Graph

34983 Commits

Author SHA1 Message Date
Simon Riggs
e60d20a35e Minor rewording of README comments 2013-04-08 17:20:26 +01:00
Heikki Linnakangas
594041311c Fix calculation of how many segments to retain for wal_keep_segments.
KeepLogSeg function was broken when we switched to use a 64-bit int for the
segment number.

Per report from Jeff Janes.
2013-04-08 16:29:56 +03:00
Simon Riggs
5787c6730e Skip extraneous locking in XLogCheckBuffer().
Heikki reported comment was wrong, so fixed
code to match the comment: we only need to
take additional locking precautions when we
have a shared lock on the buffer.
2013-04-08 09:11:49 +01:00
Simon Riggs
47c4333189 Avoid tricky race condition recording XLOG_HINT
We copy the buffer before inserting an XLOG_HINT to avoid WAL CRC errors
caused by concurrent hint writes to buffer while share locked. To make this work
we refactor RestoreBackupBlock() to allow an XLOG_HINT to avoid the normal
path for backup blocks, which assumes the underlying buffer is exclusive locked.
Resulting code completely changes layout of XLOG_HINT WAL records, but
this isn't even beta code, so this is a low impact change.
In passing, avoid taking WALInsertLock for full page writes on checksummed
hints, remove related cruft from XLogInsert() and improve xlog_desc record for
XLOG_HINT.

Andres Freund

Bug report by Fujii Masao, testing by Jeff Janes and Jaime Casanova,
review by Jeff Davis and Simon Riggs. Applied with changes from review
and some comment editing.
2013-04-08 08:52:39 +01:00
Simon Riggs
a4b94b8515 README comments on checksums on page holes. 2013-04-08 08:42:52 +01:00
Simon Riggs
1be203519a Tune BufferGetLSNAtomic() when checksums !enabled
From performance analysis by Heikki Linnakangas
2013-04-07 22:37:39 +01:00
Simon Riggs
cf8dc9e10c Fix checksums for CLUSTER, VACUUM FULL etc.
In CLUSTER, VACUUM FULL and ALTER TABLE SET TABLESPACE
I erroneously set checksum before log_newpage, which
sets the LSN and invalidates the checksum. So set
checksum immediately *after* log_newpage.

Bug report Fujii Masao, Fix and patch by Jeff Davis
2013-04-07 22:16:51 +01:00
Tom Lane
7844608e54 Get rid of USE_WIDE_UPPER_LOWER dependency in trigram construction.
contrib/pg_trgm's make_trigrams() was coded to ignore multibyte character
boundaries and just make trigrams from bytes if USE_WIDE_UPPER_LOWER wasn't
defined.  This is a bit odd, since there's no obvious reason why trigram
compaction rules should depend on the presence of towlower() and friends.
What's more, there was an Assert() that would fail if that code path was
fed any multibyte characters.

We need to do something about this since the pending regex-indexing patch
has an assumption that you get just one "trgm" from any three characters.
The best solution seems to be to remove the USE_WIDE_UPPER_LOWER
dependency, which shouldn't really have been there in the first place.
The second loop in make_trigrams() is now just a fast path and not a
potentially incompatible algorithm.

If there is anybody still using Postgres on machines without wcstombs() or
towlower(), and they have non-ASCII data indexed by pg_trgm, they'll need
to REINDEX those indexes after pg_upgrade to 9.3, else searches may fail
incorrectly. It seems likely that there are no such installations, though.

In passing, rename cnt_trigram to compact_trigram, which seems to better
describe its functionality, and improve make_trigrams' test for whether it
has to use the slow path or not (per a suggestion from Alexander Korotkov).
2013-04-07 14:46:17 -04:00
Tom Lane
faf4726c9f In isolationtester, retry after EINTR return from select(2).
Per report from Jaime Casanova.  Very curious that no one else has seen
this failure ... but the code is clearly wrong as-is.
2013-04-06 22:28:49 -04:00
Robert Haas
e965e6344c sepgsql: Enforce db_schema:search permission.
KaiGai Kohei, with comment and doc wordsmithing by me
2013-04-05 08:51:31 -04:00
Tom Lane
52f436b807 Improve documentation about the relationship of extensions and schemas.
There's been some confusion expressed about this point, so clarify.
Extended version of a patch by David Wheeler.
2013-04-04 22:37:25 -04:00
Peter Eisentraut
e3f719c367 doc: Fix number of columns in table 2013-04-04 21:07:05 -04:00
Tom Lane
927e1dc96c Fix line count in slashUsage().
Counting newlines shows that quite a few recent patches have neglected
to update the output-lines count given to PageOutput().  Fortunately
it's not terribly critical that this be exact, since we long since
exceeded the height of most people's terminal windows.  Still, maybe
we ought to think of a way to not have to maintain this manually anymore.
2013-04-04 20:29:46 -04:00
Tom Lane
c6a3fce7dd Add \watch [SEC] command to psql.
This allows convenient re-execution of commands.

Will Leinweber, reviewed by Peter Eisentraut, Daniel Farina, and Tom Lane
2013-04-04 19:56:59 -04:00
Andrew Dunstan
e75feb2834 Fix off by one error in JSON extract path code.
Bug report by David Wheeler, diagnosis assistance from Tom Lane.
2013-04-04 18:26:52 -04:00
Bruce Momjian
48a2cd370e psql: fix startup crash caused by PSQLRC containing a tilde
'strdup' the PSQLRC environment variable value before calling a routine
that might free() it.

Backpatch to 9.2, where the bug first appeared.
2013-04-04 12:56:24 -04:00
Heikki Linnakangas
bf2b0a1478 Fix crash on compiling a regular expression with more than 32k colors.
Throw an error instead.

Backpatch to all supported branches.
2013-04-04 19:48:11 +03:00
Andrew Dunstan
d7d5832012 Fix a few thinkos in the JSON functions docs.
Dickson S. Guedes
2013-04-04 10:41:29 -04:00
Heikki Linnakangas
b8ed4cc962 Calculate # of semaphores correctly with --disable-spinlocks.
The old formula didn't take into account that each WAL sender process needs
a spinlock. We had also already exceeded the fixed number of spinlocks
reserved for misc purposes (10). Bump that to 30.

Backpatch to 9.0, where WAL senders were introduced. If I counted correctly,
9.0 had exactly 10 predefined spinlocks, and 9.1 exceeded that, but bump the
limit in 9.0 too because 10 is uncomfortably close to the edge.
2013-04-04 16:40:37 +03:00
Tom Lane
f7b0006f42 Avoid updating our PgBackendStatus entry when track_activities is off.
The point of turning off track_activities is to avoid this reporting
overhead, but a thinko in commit 4f42b546fd
caused pgstat_report_activity() to perform half of its updates anyway.
Fix that, and also make sure that we clear all the now-disabled fields
when transitioning to the non-reporting state.
2013-04-03 14:13:28 -04:00
Tom Lane
0f1345d38b Fix typo in FDW docs.
Laurenz Albe
2013-04-03 10:26:18 -04:00
Tom Lane
845d335a90 Minor robustness improvements for isolationtester.
Notice and complain about PQcancel() failures.  Also, don't dump core if
an error PGresult doesn't contain severity and message subfields, as it
might not if it was generated by libpq itself.  (We have a longstanding
TODO item to improve that, but in the meantime isolationtester had better
cope.)

I tripped across the latter item while investigating a trouble report on
buildfarm member spoonbill.  As for the former, there's no evidence that
PQcancel failure is actually involved in spoonbill's problem, but it still
seems like a bad idea to ignore an error return code.
2013-04-02 21:15:37 -04:00
Tom Lane
89b661bab9 Update release notes for 9.2.4, 9.1.9, 9.0.13, 8.4.17.
Security: CVE-2013-1899, CVE-2013-1901
2013-04-01 14:11:11 -04:00
Tom Lane
17fe2793ea Fix insecure parsing of server command-line switches.
An oversight in commit e710b65c1c allowed
database names beginning with "-" to be treated as though they were secure
command-line switches; and this switch processing occurs before client
authentication, so that even an unprivileged remote attacker could exploit
the bug, needing only connectivity to the postmaster's port.  Assorted
exploits for this are possible, some requiring a valid database login,
some not.  The worst known problem is that the "-r" switch can be invoked
to redirect the process's stderr output, so that subsequent error messages
will be appended to any file the server can write.  This can for example be
used to corrupt the server's configuration files, so that it will fail when
next restarted.  Complete destruction of database tables is also possible.

Fix by keeping the database name extracted from a startup packet fully
separate from command-line switches, as had already been done with the
user name field.

The Postgres project thanks Mitsumasa Kondo for discovering this bug,
Kyotaro Horiguchi for drafting the fix, and Noah Misch for recognizing
the full extent of the danger.

Security: CVE-2013-1899
2013-04-01 14:00:51 -04:00
Tom Lane
ce9ab88981 Make REPLICATION privilege checks test current user not authenticated user.
The pg_start_backup() and pg_stop_backup() functions checked the privileges
of the initially-authenticated user rather than the current user, which is
wrong.  For example, a user-defined index function could successfully call
these functions when executed by ANALYZE within autovacuum.  This could
allow an attacker with valid but low-privilege database access to interfere
with creation of routine backups.  Reported and fixed by Noah Misch.

Security: CVE-2013-1901
2013-04-01 13:09:24 -04:00
Peter Eisentraut
85079078ac Revert "ecpg: Don't link compatlib with libpq"
This reverts commit 3780fc679c.

HP-UX didn't like it.  There would probably be a way to fix that, but
since the net effect of all of this is zero because ecpg ends up using
libpq anyway, it's not worth bothering further.
2013-03-31 23:50:51 -04:00
Tom Lane
e48a7bd527 Update release notes for changes through today. 2013-03-31 18:59:35 -04:00
Tom Lane
d931ac0ec4 Ignore extra subquery outputs in set_subquery_size_estimates().
In commit 0f61d4dd1b, I added code to copy up
column width estimates for each column of a subquery.  That code supposed
that the subquery couldn't have any output columns that didn't correspond
to known columns of the current query level --- which is true when a query
is parsed from scratch, but the assumption fails when planning a view that
depends on another view that's been redefined (adding output columns) since
the upper view was made.  This results in an assertion failure or even a
crash, as per bug #8025 from lindebg.  Remove the Assert and instead skip
the column if its resno is out of the expected range.
2013-03-31 18:34:15 -04:00
Peter Eisentraut
64f890905f Add pkg-config files for libpq and ecpg libraries
This will hopefully be easier to use than pg_config for users who are
already used to the pkg-config interface.  It also works better for
multi-arch installations.

reviewed by Tom Lane
2013-03-31 16:58:40 -04:00
Peter Eisentraut
3780fc679c ecpg: Don't link compatlib with libpq
It doesn't actually use libpq.  But we need to keep libpq in the
CPPFLAGS for building, because compatlib uses ecpglib.h which uses
libpq-fe.h, but we don't need to refer to libpq for linking.

reviewed by Tom Lane
2013-03-31 16:51:00 -04:00
Bruce Momjian
203d8ae2db pg_upgrade: don't copy/link files for invalid indexes
Now that pg_dump no longer dumps invalid indexes, per commit
683abc73df, have pg_upgrade also skip
them.  Previously pg_upgrade threw an error if invalid indexes existed.

Backpatch to 9.2, 9.1, and 9.0 (where pg_upgrade was added to git)
2013-03-30 22:20:53 -04:00
Tom Lane
22f7b9613e Improve code documentation about "magnetic disk" storage manager.
The modern incarnation of md.c is by no means specific to magnetic disk
technology, but every so often we hear from someone who's misled by the
label.  Try to clarify that it will work for anything that supports
standard filesystem operations.  Per suggestion from Andrew Dunstan.
2013-03-30 14:23:45 -04:00
Andrew Dunstan
67eb3e5075 Avoid moving data directory in upgrade testing.
Windows sometimes gets upset if we rename a large directory and then try
to use the old name quickly, as seen in occasional buildfarm failures.
So we avoid that by building the old version in the intended
destination in the first place instead of renaming it, similar to the
change made for the same reason in commit b7f8465c.
2013-03-30 12:44:29 -04:00
Bruce Momjian
b015558091 Remove tab from SGML file. 2013-03-29 23:19:03 -04:00
Peter Eisentraut
602070f9cc ecpg: Parallel make fix
In some parallel make situations, the install-headers target could be
called before the installation directories are created by installdirs,
causing the installation to fail.  Fix that by making install-headers
depend on installdirs.
2013-03-29 21:39:55 -04:00
Andrew Dunstan
6caf759f3f Fix page title for JSON Functions and Operators. 2013-03-29 16:56:22 -04:00
Andrew Dunstan
a570c98d7f Add new JSON processing functions and parser API.
The JSON parser is converted into a recursive descent parser, and
exposed for use by other modules such as extensions. The API provides
hooks for all the significant parser event such as the beginning and end
of objects and arrays, and providing functions to handle these hooks
allows for fairly simple construction of a wide variety of JSON
processing functions. A set of new basic processing functions and
operators is also added, which use this API, including operations to
extract array elements, object fields, get the length of arrays and the
set of keys of a field, deconstruct an object into a set of key/value
pairs, and create records from JSON objects and arrays of objects.

Catalog version bumped.

Andrew Dunstan, with some documentation assistance from Merlin Moncure.
2013-03-29 14:12:13 -04:00
Tom Lane
9ad27c2153 Document encode(bytea, 'escape')'s behavior correctly.
I changed this in commit fd15dba543, but
missed the fact that the SGML documentation of the function specified
exactly what it did.  Well, one of the two places where it's specified
documented that --- probably I looked at the other place and thought
nothing needed to be done.  Sync the two places where encode() and
decode() are described.
2013-03-28 23:14:54 -04:00
Tom Lane
aa02864f64 Must check indisready not just indisvalid when dumping from 9.2 server.
9.2 uses a kluge representation of "indislive"; we have to account for
that when examining pg_index.  Simplest solution is to check indisready
for 9.0 and 9.1 as well; that's harmless though unnecessary, so it's
not worth making a version distinction for.

Fixes oversight in commit 683abc73df,
as noted by Andres Freund.
2013-03-28 22:09:12 -04:00
Tom Lane
29505a894e Draft release notes for 9.2.4, 9.1.9, 9.0.13, 8.4.17.
Covers commits through today.  Not back-patching into back branches
yet, since this is just for people to review in advance.
2013-03-28 21:40:19 -04:00
Robert Haas
2a3db8cec2 sepgsql: Documentation improvements.
Fixes by me, per griping by Thom Brown.
2013-03-28 15:56:35 -04:00
Robert Haas
0f05840bf4 Allow sepgsql labels to depend on object name.
The main change here is to call security_compute_create_name_raw()
rather than security_compute_create_raw().  This ups the minimum
requirement for libselinux from 2.0.99 to 2.1.10, but it looks
like most distributions will have picked that up before 9.3 is out.

KaiGai Kohei
2013-03-28 15:41:38 -04:00
Tom Lane
ae7f1c3ef2 Update time zone data files to tzdata release 2013b.
DST law changes in Chile, Haiti, Morocco, Paraguay, some Russian areas.
Historical corrections for numerous places.
2013-03-28 15:25:48 -04:00
Tom Lane
58bc48179b Avoid "variable might be clobbered by longjmp" warning.
On older-model gcc, the original coding of UTILITY_BEGIN_QUERY() can
draw this error because of multiple assignments to _needCleanup.
Rather than mark that variable volatile, we can suppress the warning
by arranging to have just one unconditional assignment before PG_TRY.
2013-03-28 13:19:49 -04:00
Alvaro Herrera
473ab40c8b Add sql_drop event for event triggers
This event takes place just before ddl_command_end, and is fired if and
only if at least one object has been dropped by the command.  (For
instance, DROP TABLE IF EXISTS of a table that does not in fact exist
will not lead to such a trigger firing).  Commands that drop multiple
objects (such as DROP SCHEMA or DROP OWNED BY) will cause a single event
to fire.  Some firings might be surprising, such as
ALTER TABLE DROP COLUMN.

The trigger is fired after the drop has taken place, because that has
been deemed the safest design, to avoid exposing possibly-inconsistent
internal state (system catalogs as well as current transaction) to the
user function code.  This means that careful tracking of object
identification is required during the object removal phase.

Like other currently existing events, there is support for tag
filtering.

To support the new event, add a new pg_event_trigger_dropped_objects()
set-returning function, which returns a set of rows comprising the
objects affected by the command.  This is to be used within the user
function code, and is mostly modelled after the recently introduced
pg_identify_object() function.

Catalog version bumped due to the new function.

Dimitri Fontaine and Álvaro Herrera
Review by Robert Haas, Tom Lane
2013-03-28 13:05:48 -03:00
Simon Riggs
593c39d156 Revoke bc5334d867 2013-03-28 09:18:02 +00:00
Simon Riggs
d139a5e26b Revoke 7a5a59d378 2013-03-28 09:12:55 +00:00
Tom Lane
0d1ecd6300 Reset OpenSSL randomness state in each postmaster child process.
Previously, if the postmaster initialized OpenSSL's PRNG (which it will do
when ssl=on in postgresql.conf), the same pseudo-random state would be
inherited by each forked child process.  The problem is masked to a
considerable extent if the incoming connection uses SSL encryption, but
when it does not, identical pseudo-random state is made available to
functions like contrib/pgcrypto.  The process's PID does get mixed into any
requested random output, but on most systems that still only results in 32K
or so distinct random sequences available across all Postgres sessions.
This might allow an attacker who has database access to guess the results
of "secure" operations happening in another session.

To fix, forcibly reset the PRNG after fork().  Each child process that has
need for random numbers from OpenSSL's generator will thereby be forced to
go through OpenSSL's normal initialization sequence, which should provide
much greater variability of the sequences.  There are other ways we might
do this that would be slightly cheaper, but this approach seems the most
future-proof against SSL-related code changes.

This has been assigned CVE-2013-1900, but since the issue and the patch
have already been publicized on pgsql-hackers, there's no point in trying
to hide this commit.

Back-patch to all supported branches.

Marko Kreen
2013-03-27 18:50:21 -04:00
Kevin Grittner
40e873d854 Fix pasto which broke docs build.
Commit bc5334d867 accidentally
included a second <variablelist> tag for a new list item.
2013-03-27 16:11:46 -05:00
Heikki Linnakangas
3cfb572dde Fix buffer pin leak in heap update redo routine.
In a heap update, if the old and new tuple were on different pages, and the
new page no longer existed (because it was subsequently truncated away by
vacuum), heap_xlog_update forgot to release the pin on the old buffer. This
bug was introduced by the "Fix multiple problems in WAL replay" patch,
commit 3bbf668de9 (on master branch).

With full_page_writes=off, this triggered an "incorrect local pin count"
error later in replay, if the old page was vacuumed.

This fixes bug #7969, reported by Yunong Xiao. Backpatch to 9.0, like the
commit that introduced this bug.
2013-03-27 22:00:01 +02:00