Commit Graph

35808 Commits

Author SHA1 Message Date
Tom Lane
8b151558c8 Be sure to release proc->backendLock after SetupLockInTable() failure.
The various places that transferred fast-path locks to the main lock table
neglected to release the PGPROC's backendLock if SetupLockInTable failed
due to being out of shared memory.  In most cases this is no big deal since
ensuing error cleanup would release all held LWLocks anyway.  But there are
some hot-standby functions that don't consider failure of
FastPathTransferRelationLocks to be a hard error, and in those cases this
oversight could lead to system lockup.  For consistency, make all of these
places look the same as FastPathTransferRelationLocks.

Noted while looking for the cause of Dan Wood's bugs --- this wasn't it,
but it's a bug anyway.
2013-11-29 17:35:09 -05:00
Tom Lane
16e1b7a1b7 Fix assorted race conditions in the new timeout infrastructure.
Prevent handle_sig_alarm from losing control partway through due to a query
cancel (either an asynchronous SIGINT, or a cancel triggered by one of the
timeout handler functions).  That would at least result in failure to
schedule any required future interrupt, and might result in actual
corruption of timeout.c's data structures, if the interrupt happened while
we were updating those.

We could still lose control if an asynchronous SIGINT arrives just as the
function is entered.  This wouldn't break any data structures, but it would
have the same effect as if the SIGALRM interrupt had been silently lost:
we'd not fire any currently-due handlers, nor schedule any new interrupt.
To forestall that scenario, forcibly reschedule any pending timer interrupt
during AbortTransaction and AbortSubTransaction.  We can avoid any extra
kernel call in most cases by not doing that until we've allowed
LockErrorCleanup to kill the DEADLOCK_TIMEOUT and LOCK_TIMEOUT events.

Another hazard is that some platforms (at least Linux and *BSD) block a
signal before calling its handler and then unblock it on return.  When we
longjmp out of the handler, the unblock doesn't happen, and the signal is
left blocked indefinitely.  Again, we can fix that by forcibly unblocking
signals during AbortTransaction and AbortSubTransaction.

These latter two problems do not manifest when the longjmp reaches
postgres.c, because the error recovery code there kills all pending timeout
events anyway, and it uses sigsetjmp(..., 1) so that the appropriate signal
mask is restored.  So errors thrown outside any transaction should be OK
already, and cleaning up in AbortTransaction and AbortSubTransaction should
be enough to fix these issues.  (We're assuming that any code that catches
a query cancel error and doesn't re-throw it will do at least a
subtransaction abort to clean up; but that was pretty much required already
by other subsystems.)

Lastly, ProcSleep should not clear the LOCK_TIMEOUT indicator flag when
disabling that event: if a lock timeout interrupt happened after the lock
was granted, the ensuing query cancel is still going to happen at the next
CHECK_FOR_INTERRUPTS, and we want to report it as a lock timeout not a user
cancel.

Per reports from Dan Wood.

Back-patch to 9.3 where the new timeout handling infrastructure was
introduced.  We may at some point decide to back-patch the signal
unblocking changes further, but I'll desist from that until we hear
actual field complaints about it.
2013-11-29 16:41:00 -05:00
Peter Eisentraut
50107ee7a5 doc: Enhance documentation of ssl_ciphers setting a bit 2013-11-29 09:06:28 -05:00
Peter Eisentraut
384eb1d40d doc: Allow selecting web site CSS style sheet in XSLT HTML build 2013-11-28 22:45:08 -05:00
Robert Haas
8e18d04d4d Refine our definition of what constitutes a system relation.
Although user-defined relations can't be directly created in
pg_catalog, it's possible for them to end up there, because you can
create them in some other schema and then use ALTER TABLE .. SET SCHEMA
to move them there.  Previously, such relations couldn't afterwards
be manipulated, because IsSystemRelation()/IsSystemClass() rejected
all attempts to modify objects in the pg_catalog schema, regardless
of their origin.  With this patch, they now reject only those
objects in pg_catalog which were created at initdb-time, allowing
most operations on user-created tables in pg_catalog to proceed
normally.

This patch also adds new functions IsCatalogRelation() and
IsCatalogClass(), which is similar to IsSystemRelation() and
IsSystemClass() but with a slightly narrower definition: only TOAST
tables of system catalogs are included, rather than *all* TOAST tables.
This is currently used only for making decisions about when
invalidation messages need to be sent, but upcoming logical decoding
patches will find other uses for this information.

Andres Freund, with some modifications by me.
2013-11-28 20:57:20 -05:00
Heikki Linnakangas
2fe69cacff Another gin_desc fix.
The number of items inserted was incorrectly printed as if it was a boolean.
2013-11-28 23:35:50 +02:00
Heikki Linnakangas
97c19e6c38 Fix gin_desc routine to match the WAL format.
In the GIN incomplete-splits patch, I used BlockIdDatas to store the block
number of left and right children, when inserting a downlink after a split
to an internal page posting list page. But gin_desc thought they were stored
as BlockNumbers.
2013-11-28 21:57:42 +02:00
Tom Lane
da8a716089 Fix latent(?) race condition in LockReleaseAll.
We have for a long time checked the head pointer of each of the backend's
proclock lists and skipped acquiring the corresponding locktable partition
lock if the head pointer was NULL.  This was safe enough in the days when
proclock lists were changed only by the owning backend, but it is pretty
questionable now that the fast-path patch added cases where backends add
entries to other backends' proclock lists.  However, we don't really wish
to revert to locking each partition lock every time, because in simple
transactions that would add a lot of useless lock/unlock cycles on
already-heavily-contended LWLocks.  Fortunately, the only way that another
backend could be modifying our proclock list at this point would be if it
was promoting a formerly fast-path lock of ours; and any such lock must be
one that we'd decided not to delete in the previous loop over the locallock
table.  So it's okay if we miss seeing it in this loop; we'd just decide
not to delete it again.  However, once we've detected a non-empty list,
we'd better re-fetch the list head pointer after acquiring the partition
lock.  This guards against possibly fetching a corrupt-but-non-null pointer
if pointer fetch/store isn't atomic.  It's not clear if any practical
architectures are like that, but we've never assumed that before and don't
wish to start here.  In any case, the situation certainly deserves a code
comment.

While at it, refactor the partition traversal loop to use a for() construct
instead of a while() loop with goto's.

Back-patch, just in case the risk is real and not hypothetical.
2013-11-28 12:17:46 -05:00
Alvaro Herrera
d51a8c52ba Unbreak buildfarm
I removed an intermediate commit before pushing and forgot to test the
resulting tree :-(
2013-11-28 12:59:45 -03:00
Alvaro Herrera
247c76a989 Use a more granular approach to follow update chains
Instead of simply checking the KEYS_UPDATED bit, we need to check
whether each lock held on the future version of the tuple conflicts with
the lock we're trying to acquire.

Per bug report #8434 by Tomonari Katsumata
2013-11-28 12:00:12 -03:00
Alvaro Herrera
e4828e9ccb Compare Xmin to previous Xmax when locking an update chain
Not doing so causes us to traverse an update chain that has been broken
by concurrent page pruning.  All other code that traverses update chains
uses this check as one of the cases in which to stop iterating, so
replicate it here too.  Failure to do so leads to erroneous CLOG,
subtrans or multixact lookups.

Per discussion following the bug report by J Smith in
CADFUPgc5bmtv-yg9znxV-vcfkb+JPRqs7m2OesQXaM_4Z1JpdQ@mail.gmail.com
as diagnosed by Andres Freund.
2013-11-28 12:00:12 -03:00
Alvaro Herrera
c235a6a589 Don't try to set InvalidXid as page pruning hint
If a transaction updates/deletes a tuple just before aborting, and a
concurrent transaction tries to prune the page concurrently, the pruner
may see HeapTupleSatisfiesVacuum return HEAPTUPLE_DELETE_IN_PROGRESS,
but a later call to HeapTupleGetUpdateXid() return InvalidXid.  This
would cause an assertion failure in development builds, but would be
otherwise Mostly Harmless.

Fix by checking whether the updater Xid is valid before trying to apply
it as page prune point.

Reported by Andres in 20131124000203.GA4403@alap2.anarazel.de
2013-11-28 12:00:12 -03:00
Alvaro Herrera
e518fa7adf Cope with heap_fetch failure while locking an update chain
The reason for the fetch failure is that the tuple was removed because
it was dead; so the failure is innocuous and can be ignored.  Moreover,
there's no need for further work and we can return success to the caller
immediately.  EvalPlanQualFetch is doing something very similar to this
already.

Report and test case from Andres Freund in
20131124000203.GA4403@alap2.anarazel.de
2013-11-28 12:00:12 -03:00
Peter Eisentraut
b6ecde8ed4 doc: Set chunk.first.sections in XSLT, for consistency with DSSSL output 2013-11-27 22:42:04 -05:00
Bruce Momjian
9ef780d4d7 pg_buffercache docs: adjust order of fields
Adjust order of fields to match view order.

Jaime Casanova
2013-11-27 22:33:59 -05:00
Peter Eisentraut
a607b6903e doc: Put data types in alphabetical order
From: Andreas Karlsson <andreas@proxel.se>
2013-11-27 21:50:27 -05:00
Tom Lane
7db285afc9 Fix stale-pointer problem in fast-path locking logic.
When acquiring a lock in fast-path mode, we must reset the locallock
object's lock and proclock fields to NULL.  They are not necessarily that
way to start with, because the locallock could be left over from a failed
lock acquisition attempt earlier in the transaction.  Failure to do this
led to all sorts of interesting misbehaviors when LockRelease tried to
clean up no-longer-related lock and proclock objects in shared memory.
Per report from Dan Wood.

In passing, modify LockRelease to elog not just Assert if it doesn't find
lock and proclock objects for a formerly fast-path lock, matching the code
in FastPathGetRelationLockEntry and LockRefindAndRelease.  This isn't a
bug but it will help in diagnosing any future bugs in this area.

Also, modify FastPathTransferRelationLocks and FastPathGetRelationLockEntry
to break out of their loops over the fastpath array once they've found the
sole matching entry.  This was inconsistently done in some search loops
and not others.

Improve assorted related comments, too.

Back-patch to 9.2 where the fast-path mechanism was introduced.
2013-11-27 18:10:00 -05:00
Kevin Grittner
89ba815092 Minor correction of READ COMMITTED isolation level docs.
Per report from AK
2013-11-27 14:34:12 -06:00
Tom Lane
8c84803e14 Minor corrections in lmgr/README.
Correct an obsolete statement that no backend touches another backend's
PROCLOCK lists.  This was probably wrong even when written (the deadlock
checker looks at everybody's lists), and it's certainly quite wrong now
that fast-path locking can require creation of lock and proclock objects
on behalf of another backend.  Also improve some statements in the hot
standby explanation, and do one or two other trivial bits of wordsmithing/
reformatting.
2013-11-27 15:07:13 -05:00
Heikki Linnakangas
631118fe1e Get rid of the post-recovery cleanup step of GIN page splits.
Replace it with an approach similar to what GiST uses: when a page is split,
the left sibling is marked with a flag indicating that the parent hasn't been
updated yet. When the parent is updated, the flag is cleared. If an insertion
steps on a page with the flag set, it will finish split before proceeding
with the insertion.

The post-recovery cleanup mechanism was never totally reliable, as insertion
to the parent could fail e.g because of running out of memory or disk space,
leaving the tree in an inconsistent state.

This also divides the responsibility of WAL-logging more clearly between
the generic ginbtree.c code, and the parts specific to entry and posting
trees. There is now a common WAL record format for insertions and deletions,
which is written by ginbtree.c, followed by tree-specific payload, which is
returned by the placetopage- and split- callbacks.
2013-11-27 19:21:23 +02:00
Heikki Linnakangas
ce5326eed3 More GIN refactoring.
Separate the insertion payload from the more static portions of GinBtree.
GinBtree now only contains information related to searching the tree, and
the information of what to insert is passed separately.

Add root block number to GinBtree, instead of passing it around all the
functions as argument.

Split off ginFinishSplit() from ginInsertValue(). ginFinishSplit is
responsible for finding the parent and inserting the downlink to it.
2013-11-27 15:43:05 +02:00
Heikki Linnakangas
4118f7e8ed Fix plpython3 expected output.
I neglected this in the previous commit that updated the plpython2 output,
which I forgot to "git add" earlier.

As pointed out by Rodolfo Campero and Marko Kreen.
2013-11-27 14:25:13 +02:00
Heikki Linnakangas
82b43f7df2 Don't update relfrozenxid if any pages were skipped.
Vacuum recognizes that it can update relfrozenxid by checking whether it has
processed all pages of a relation. Unfortunately it performed that check
after truncating the dead pages at the end of the relation, and used the new
number of pages to decide whether all pages have been scanned. If the new
number of pages happened to be smaller or equal to the number of pages
scanned, it incorrectly decided that all pages were scanned.

This can lead to relfrozenxid being updated, even though some pages were
skipped that still contain old XIDs. That can lead to data loss due to xid
wraparounds with some rows suddenly missing. This likely has escaped notice
so far because it takes a large number (~2^31) of xids being used to see the
effect, while a full-table vacuum before that would fix the issue.

The incorrect logic was introduced by commit
b4b6923e03. Backpatch this fix down to 8.4,
like that commit.

Andres Freund, with some modifications by me.
2013-11-27 13:43:27 +02:00
Michael Meskes
2390f2b2d3 Documentation fix for ecpg.
The latest fixes removed a limitation that was still in the docs, so Zoltan updated the docs, too.
2013-11-27 11:03:59 +01:00
Michael Meskes
51867a0f9b ECPG: Fix searching for quoted cursor names case-sensitively.
Patch by Böszörményi Zoltán <zb@cybertec.at>
2013-11-27 11:02:13 +01:00
Fujii Masao
d1b88f6b36 Add --xlogdir option to pg_basebackup, for specifying the pg_xlog directory.
Haribabu kommi, slightly modified by me.
2013-11-27 14:00:16 +09:00
Fujii Masao
551c78281b Fix typo in release note.
Backpatch to 9.1.

Josh Kupershmidt
2013-11-27 13:48:01 +09:00
Peter Eisentraut
85ed91ee7d Implement information_schema.parameters.parameter_default column
Reviewed-by: Ali Dar <ali.munir.dar@gmail.com>
Reviewed-by: Amit Khandekar <amit.khandekar@enterprisedb.com>
Reviewed-by: Rodolfo Campero <rodolfo.campero@anachronics.com>
2013-11-26 23:21:35 -05:00
Peter Eisentraut
3803ff985c doc: Add id to index in XSLT build
That way, the HTML file name of the index will be the same as currently
for the DSSSL build.
2013-11-26 23:02:40 -05:00
Heikki Linnakangas
4c83e0353f Oops, forgot to "git add" last minute changes to regression test. 2013-11-26 23:05:48 +02:00
Michael Meskes
d2542f9270 ECPG: Fix offset to NULL/size indicator array.
Patch by Boszormenyi Zoltan <zb@cybertec.at>
2013-11-26 17:42:33 +01:00
Michael Meskes
f641fc86fb ECPG: Simplify free_variable()
Patch by Boszormenyi Zoltan <zb@cybertec.at>
2013-11-26 17:42:32 +01:00
Michael Meskes
1ec4c56e76 ECPG: Add EXEC SQL CLOSE C to the tests.
Patch by Boszormenyi Zoltan <zb@cybertec.at>
2013-11-26 17:42:32 +01:00
Michael Meskes
db58e8ff7c ECPG: Free the malloc()'ed variables in the test so it comes out clean on
Valgrind runs.

Patch by Boszormenyi Zoltan <zb@cybertec.at>
2013-11-26 17:42:32 +01:00
Michael Meskes
b46fa32100 ECPG: Make the preprocessor emit ';' if the variable type for a list of
variables is varchar. This fixes this test case:

int main(void)
{
    exec sql begin declare section;
    varchar a[50], b[50];
    exec sql end declare section;

    return 0;
}

Since varchars are internally turned into custom structs and
the type name is emitted for these variable declarations,
the preprocessed code previously had:

struct varchar_1  { ... }  a _,_  struct varchar_2  { ... }  b ;

The comma in the generated C file was a syntax error.

There are no regression test changes since it's not exercised.

Patch by Boszormenyi Zoltan <zb@cybertec.at>
2013-11-26 17:42:32 +01:00
Heikki Linnakangas
37364c6311 Handle domains over arrays like plain arrays in PL/python.
Domains over arrays are now converted to/from python lists when passed as
arguments or return values. Like regular arrays.

This has some potential to break applications that rely on the old behavior
that they are passed as strings, but in practice there probably aren't many
such applications out there.

Rodolfo Campero
2013-11-26 14:33:31 +02:00
Jeff Davis
7cc0ba9f17 Add missing entry for session_preload_libraries in sample config.
The omission was apparently an oversight in the original patch.
2013-11-25 21:03:07 -08:00
Bruce Momjian
a6542a4b68 Change SET LOCAL/CONSTRAINTS/TRANSACTION and ABORT behavior
Change SET LOCAL/CONSTRAINTS/TRANSACTION behavior outside of a
transaction block from error (post-9.3) to warning.  (Was nothing in <=
9.3.)  Also change ABORT outside of a transaction block from notice to
warning.
2013-11-25 19:19:40 -05:00
Michael Meskes
05b476c298 More improvement to comment parsing in ecpg.
ECPG is not supposed to allow and output nested comments in C. These comments
are only allowed in the SQL parts and must not be written into the C file.
Also the different handling of different comments is documented.
2013-11-25 15:38:09 +01:00
Michael Meskes
ef8b3b00b5 Fix ecpg parsing of sizeof().
The last fix used the wrong non-terminal to define valid types.
2013-11-25 15:11:39 +01:00
Jeff Davis
559d535819 Lessen library-loading log level.
Previously, messages were emitted at the LOG level every time a
backend preloaded a library. That was acceptable (though unnecessary)
for shared_preload_libraries; but it was excessive for
local_preload_libraries and session_preload_libraries. Reduce to
DEBUG1.

Also, there was logic in the EXEC_BACKEND case to avoid repeated
messages for shared_preload_libraries by demoting them to
DEBUG2. DEBUG1 seems more appropriate there, as well, so eliminate
that special case.

Peter Geoghegan.
2013-11-24 10:50:54 -08:00
Tom Lane
36a3be6540 Fix new and latent bugs with errno handling in secure_read/secure_write.
These functions must be careful that they return the intended value of
errno to their callers.  There were several scenarios where this might
not happen:

1. The recent SSL renegotiation patch added a hunk of code that would
execute after setting errno.  In the first place, it's doubtful that we
should consider renegotiation to be successfully completed after a failure,
and in the second, there's no real guarantee that the called OpenSSL
routines wouldn't clobber errno.  Fix by not executing that hunk except
during success exit.

2. errno was left in an unknown state in case of an unrecognized return
code from SSL_get_error().  While this is a "can't happen" case, it seems
like a good idea to be sure we know what would happen, so reset errno to
ECONNRESET in such cases.  (The corresponding code in libpq's fe-secure.c
already did this.)

3. There was an (undocumented) assumption that client_read_ended() wouldn't
change errno.  While true in the current state of the code, this seems less
than future-proof.  Add explicit saving/restoring of errno to make sure
that changes in the called functions won't break things.

I see no need to back-patch, since #1 is new code and the other two issues
are mostly hypothetical.

Per discussion with Amit Kapila.
2013-11-24 13:09:38 -05:00
Michael Meskes
08d1b22b3b Allow C array definitions to use sizeof().
When parsing C variable definitions ecpg should allow sizeof() operators as array dimensions.
2013-11-24 12:51:21 +01:00
Michael Meskes
8ac5e88f9f Distinguish between C and SQL mode for C-style comments.
SQL standard asks for allowing nested comments, while C does not. Therefore the
two comments, while mostly similar, have to be parsed seperately.
2013-11-24 12:26:00 +01:00
Tom Lane
64d15e4240 Defend against bad trigger definitions in contrib/lo's lo_manage() trigger.
This function formerly crashed if called as a statement-level trigger,
or if a column-name argument wasn't given.

In passing, add the trigger name to all error messages from the function.
(None of them are expected cases, so this shouldn't pose any compatibility
risk.)

Marc Cousin, reviewed by Sawada Masahiko
2013-11-23 22:46:43 -05:00
Peter Eisentraut
a5036ca998 PL/Tcl: Add event trigger support
From: Dimitri Fontaine <dimitri@2ndQuadrant.fr>
2013-11-23 21:32:00 -05:00
Tom Lane
45e02e3232 Fix array slicing of int2vector and oidvector values.
The previous coding labeled expressions such as pg_index.indkey[1:3] as
being of int2vector type; which is not right because the subscript bounds
of such a result don't, in general, satisfy the restrictions of int2vector.
To fix, implicitly promote the result of slicing int2vector to int2[],
or oidvector to oid[].  This is similar to what we've done with domains
over arrays, which is a good analogy because these types are very much
like restricted domains of the corresponding regular-array types.

A side-effect is that we now also forbid array-element updates on such
columns, eg while "update pg_index set indkey[4] = 42" would have worked
before if you were superuser (and corrupted your catalogs irretrievably,
no doubt) it's now disallowed.  This seems like a good thing since, again,
some choices of subscripting would've led to results not satisfying the
restrictions of int2vector.  The case of an array-slice update was
rejected before, though with a different error message than you get now.
We could make these cases work in future if we added a cast from int2[]
to int2vector (with a cast function checking the subscript restrictions)
but it seems unlikely that there's any value in that.

Per report from Ronan Dunklau.  Back-patch to all supported branches
because of the crash risks involved.
2013-11-23 20:03:56 -05:00
Tom Lane
f145454d57 Ensure _dosmaperr() actually sets errno correctly.
If logging is enabled, either ereport() or fprintf() might stomp on errno
internally, causing this function to return the wrong result.  That might
only end in a misleading error report, but in any code that's examining
errno to decide what to do next, the consequences could be far graver.

This has been broken since the very first version of this file in 2006
... it's a bit astonishing that we didn't identify this long ago.

Reported by Amit Kapila, though this isn't his proposed fix.
2013-11-23 18:24:26 -05:00
Peter Eisentraut
b7212c9726 Fix thinko in SPI_execute_plan() calls
Two call sites were apparently thinking that the last argument of
SPI_execute_plan() is the number of query parameters, but it is actually
the row limit.  Change the calls to 0, since we don't care about the
limit there.  The previous code didn't break anything, but it was still
wrong.
2013-11-23 09:34:57 -05:00
Peter Eisentraut
4053189d59 Avoid potential buffer overflow crash
A pointer to a C string was treated as a pointer to a "name" datum and
passed to SPI_execute_plan().  This pointer would then end up being
passed through datumCopy(), which would try to copy the entire 64 bytes
of name data, thus running past the end of the C string.  Fix by
converting the string to a proper name structure.

Found by LLVM AddressSanitizer.
2013-11-23 07:25:37 -05:00