Commit Graph

38252 Commits

Author SHA1 Message Date
Peter Eisentraut
30982be4e5 Integrate pg_upgrade_support module into backend
Previously, these functions were created in a schema "binary_upgrade",
which was deleted after pg_upgrade was finished.  Because we don't want
to keep that schema around permanently, move them to pg_catalog but
rename them with a binary_upgrade_... prefix.

The provided functions are only small wrappers around global variables
that were added specifically for pg_upgrade use, so keeping the module
separate does not create any modularity.

The functions still check that they are only called in binary upgrade
mode, so it is not possible to call these during normal operation.

Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2015-04-14 19:26:37 -04:00
Heikki Linnakangas
936546dcbc Optimize pg_comp_crc32c_sse42 routine slightly, and also use it on x86.
Eliminate the separate 'len' variable from the loops, and also use the 4
byte instruction. This shaves off a few more cycles. Even though this
routine that uses the special SSE 4.2 instructions is much faster than a
generic routine, it's still a hot spot, so let's make it as fast as
possible.

Change the configure test to not test _mm_crc32_u64. That variant is only
available in the 64-bit x86-64 architecture, not in 32-bit x86. Modify
pg_comp_crc32c_sse42 so that it only uses _mm_crc32_u64 on x86-64. With
these changes, the SSE accelerated CRC-32C implementation can also be used
on 32-bit x86 systems.

This also fixes the 32-bit MSVC build.
2015-04-14 23:58:16 +03:00
Heikki Linnakangas
b73e7a0716 Oops, fix misspelled #endif
I hope this fixes the Windows builfarm failures.
2015-04-14 22:00:52 +03:00
Heikki Linnakangas
b4eb2d168d Try to fix the CRC-32C autoconf magic for icc compiler.
On gcc and clang, the _mm_crc32_u8 and _mm_crc32_u64 intrinsics are not
defined at all, when not building with -msse4.2. But on icc, they are.
So we cannot assume that if those intrinsics are defined, we can always use
them safely, we might still need the runtime check.

To fix, check if the __SSE_4_2__ preprocessor symbol is defined. That's
supposed to be defined only when the compiler is targeting a processor that
has SSE 4.2 support.

Per buildfarm members fulmar and okapi.
2015-04-14 19:57:19 +03:00
Alvaro Herrera
0a52fafce4 Fix typo in comment
SLRU_SEGMENTS_PER_PAGE -> SLRU_PAGES_PER_SEGMENT

I introduced this ancient typo in subtrans.c and later propagated it to
multixact.c.  I fixed the latter in f741300c, but only back to 9.3;
backpatch to all supported branches for consistency.
2015-04-14 12:12:18 -03:00
Heikki Linnakangas
3dc2d62d04 Use Intel SSE 4.2 CRC instructions where available.
Modern x86 and x86-64 processors with SSE 4.2 support have special
instructions, crc32b and crc32q, for calculating CRC-32C. They greatly
speed up CRC calculation.

Whether the instructions can be used or not depends on the compiler and the
target architecture. If generation of SSE 4.2 instructions is allowed for
the target (-msse4.2 flag on gcc and clang), use them. If they are not
allowed by default, but the compiler supports the -msse4.2 flag to enable
them, compile just the CRC-32C function with -msse4.2 flag, and check at
runtime whether the processor we're running on supports it. If it doesn't,
fall back to the slicing-by-8 algorithm. (With the common defaults on
current operating systems, the runtime-check variant is what you get in
practice.)

Abhijit Menon-Sen, heavily modified by me, reviewed by Andres Freund.
2015-04-14 17:05:03 +03:00
Heikki Linnakangas
4f700bcd20 Reorganize our CRC source files again.
Now that we use CRC-32C in WAL and the control file, the "traditional" and
"legacy" CRC-32 variants are not used in any frontend programs anymore.
Move the code for those back from src/common to src/backend/utils/hash.

Also move the slicing-by-8 implementation (back) to src/port. This is in
preparation for next patch that will add another implementation that uses
Intel SSE 4.2 instructions to calculate CRC-32C, where available.
2015-04-14 17:03:42 +03:00
Peter Eisentraut
d577bb868d pgbench: Attempt fix build on Windows 2015-04-13 15:32:57 -04:00
Alvaro Herrera
b5213e14a4 Remove duplicated word in README 2015-04-13 14:28:21 -03:00
Peter Eisentraut
81134af3ec Move pgbench from contrib/ to src/bin/
Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2015-04-13 13:07:16 -04:00
Heikki Linnakangas
b22a36a62c Fix pg_rewind regression tests in VPATH builds
Should call just "pg_rewind", instead of "./pg_rewind". The tests are called
so that PATH contains the temporariy installation bin dir.

Per report from Alvaro Herrera
2015-04-13 18:30:38 +03:00
Heikki Linnakangas
53ba10770a Refactor and fix TAP tests of pg_rewind
* Don't pass arguments to prove, since that's not supported on perl 5.8
which is the minimum version supported by the TAP tests. Refactor the
test files themselves to run the tests twice, in both local and remote mode.

* Use eq rather than == for string comparison. This thinko caused the remote
versions of the tests to never run.

* Add "use strict" and "use warnings", and fix warnings that that produced.

* Increase the delay after standby promotion, to make the tests more robust.

* In remote mode, the connection string to the promoted standby was
incorrect, leading to connection errors.

Patch by Michael Paquier, to address Peter Eisentraut's report.
2015-04-13 18:09:20 +03:00
Heikki Linnakangas
b2a5545bd6 Don't archive bogus recycled or preallocated files after timeline switch.
After a timeline switch, we would leave behind recycled WAL segments that
are in the future, but on the old timeline. After promotion, and after they
become old enough to be recycled again, we would notice that they don't have
a .ready or .done file, create a .ready file for them, and archive them.
That's bogus, because the files contain garbage, recycled from an older
timeline (or prealloced as zeros). We shouldn't archive such files.

This could happen when we're following a timeline switch during replay, or
when we switch to new timeline at end-of-recovery.

To fix, whenever we switch to a new timeline, scan the data directory for
WAL segments on the old timeline, but with a higher segment number, and
remove them. Those don't belong to our timeline history, and are most
likely bogus recycled or preallocated files. They could also be valid files
that we streamed from the primary ahead of time, but in any case, they're
not needed to recover to the new timeline.
2015-04-13 16:53:49 +03:00
Fujii Masao
1f94bec7a9 Silence gettext warning about '\r' escape sequence in translatable string.
gettext was unhappy about the commit b216ad7 because it revealed
the problem that internationalized messages may contain '\r' escape
sequence in pg_rewind. This commit moves '\r' to a separate printf() call.

Michael Paquier, bug reported by Peter Eisentraut
2015-04-13 13:30:59 +09:00
Peter Eisentraut
442663f133 emacs: Set indent-tabs-mode in perl-mode
This matches existing practice, but makes the setup complete and
consistent with the C code setup.
2015-04-12 23:53:23 -04:00
Heikki Linnakangas
74a68e37d0 Free leaked result set in pg_rewind
It was not significant in practice, it was just one instance of a small
result set, but let's pacify Coverity.

Michael Paquier
2015-04-12 22:42:01 +03:00
Magnus Hagander
9029f4b374 Add system view pg_stat_ssl
This view shows information about all connections, such as if the
connection is using SSL, which cipher is used, and which client
certificate (if any) is used.

Reviews by Alex Shulgin, Heikki Linnakangas, Andres Freund & Michael Paquier
2015-04-12 19:07:46 +02:00
Heikki Linnakangas
a10589a512 Remove duplicated words in comments.
David Rowley
2015-04-12 10:46:17 +03:00
Peter Eisentraut
83aca89f7c Move pg_archivecleanup from contrib/ to src/bin/
Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2015-04-11 23:29:18 -04:00
Alvaro Herrera
27846f02c1 Optimize locking a tuple already locked by another subxact
Locking and updating the same tuple repeatedly led to some strange
multixacts being created which had several subtransactions of the same
parent transaction holding locks of the same strength.  However,
once a subxact of the current transaction holds a lock of a given
strength, it's not necessary to acquire the same lock again.  This made
some coding patterns much slower than required.

The fix is twofold.  First we change HeapTupleSatisfiesUpdate to return
HeapTupleBeingUpdated for the case where the current transaction is
already a single-xid locker for the given tuple; it used to return
HeapTupleMayBeUpdated for that case.  The new logic is simpler, and the
change to pgrowlocks is a testament to that: previously we needed to
check for the single-xid locker separately in a very ugly way.  That
test is simpler now.

As fallout from the HTSU change, some of its callers need to be amended
so that tuple-locked-by-own-transaction is taken into account in the
BeingUpdated case rather than the MayBeUpdated case.  For many of them
there is no difference; but heap_delete() and heap_update now check
explicitely and do not grab tuple lock in that case.

The HTSU change also means that routine MultiXactHasRunningRemoteMembers
introduced in commit 11ac4c73cb is no longer necessary and can be
removed; the case that used to require it is now handled naturally as
result of the changes to heap_delete and heap_update.

The second part of the fix to the performance issue is to adjust
heap_lock_tuple to avoid the slowness:

1. Previously we checked for the case that our own transaction already
held a strong enough lock and returned MayBeUpdated, but only in the
multixact case.  Now we do it for the plain Xid case as well, which
saves having to LockTuple.

2. If the current transaction is the only locker of the tuple (but with
a lock not as strong as what we need; otherwise it would have been
caught in the check mentioned above), we can skip sleeping on the
multixact, and instead go straight to create an updated multixact with
the additional lock strength.

3. Most importantly, make sure that both the single-xid-locker case and
the multixact-locker case optimization are applied always.  We do this
by checking both in a single place, rather than them appearing in two
separate portions of the routine -- something that is made possible by
the HeapTupleSatisfiesUpdate API change.  Previously we would only check
for the single-xid case when HTSU returned MayBeUpdated, and only
checked for the multixact case when HTSU returned BeingUpdated.  This
was at odds with what HTSU actually returned in one case: if our own
transaction was locker in a multixact, it returned MayBeUpdated, so the
optimization never applied.  This is what led to the large multixacts in
the first place.

Per bug report #8470 by Oskari Saarenmaa.
2015-04-10 13:47:15 -03:00
Peter Eisentraut
8a0d34e4e4 libpq: Don't overwrite existing OpenSSL thread callbacks
If someone else already set the callbacks, don't overwrite them with
ours.  When unsetting the callbacks, only unset them if they point to
ours.

Author: Jan Urbański <wulczer@wulczer.org>
2015-04-09 20:45:34 -04:00
Magnus Hagander
a6f3c1f1e2 Show owner of types in psql \dT+ 2015-04-09 21:39:35 +02:00
Heikki Linnakangas
5d79b67bdd Make SSL regression test suite more portable by avoiding cp.
Use perl 'glob' and File::Copy instead of "cp". This takes us one step
closer to running the suite on Windows.

Michael Paquier
2015-04-09 22:07:18 +03:00
Heikki Linnakangas
0fb256dc82 Gitignore temp files generated by SSL regression suite
Michael Paquier
2015-04-09 22:02:21 +03:00
Magnus Hagander
c9970ab937 Fix typo
Michael Paquier
2015-04-09 14:15:39 +02:00
Magnus Hagander
8ae4600cd9 Fix incorrect punctuation
Amit Langote
2015-04-09 13:35:30 +02:00
Andres Freund
06d36fa40c Fix typo in eb68379c3.
I'd accidentally missed to rename PG_FORCE_NULL to BKI_FORCE_NULL in one
place.

Author: Jeevan Chalke
Discussion: CAM2+6=VPoow5PqgqiTjPX4QNeokb7op8aD_8Zg3QnHZMvvU0GQ@mail.gmail.com
2015-04-09 13:29:22 +02:00
Fujii Masao
17d436d2e8 Remove obsolete FORCE option from REINDEX.
FORCE option has been marked "obsolete" since very old version 7.4
but existed for backwards compatibility. Per discussion on pgsql-hackers,
we concluded that it's no longer worth keeping supporting the option.
2015-04-09 11:31:42 +09:00
Alvaro Herrera
73206812cd Change SQLSTATE for event triggers "wrong context" message
When certain event-trigger-only functions are called when not in the
wrong context, they were reporting the "feature not supported" SQLSTATE,
which is somewhat misleading.  Create a new custom error code for such
uses instead.

Not backpatched since it may be seen as an undesirable behavioral
change.

Author: Michael Paquier
Discussion: https://www.postgresql.org/message-id/CAB7nPqQ-5NAkHQHh_NOm7FPep37NCiLKwPoJ2Yxb8TDoGgbYYA@mail.gmail.com
2015-04-08 15:26:50 -03:00
Alvaro Herrera
5df64f298d Fix autovacuum launcher shutdown sequence
It was previously possible to have the launcher re-execute its main loop
before shutting down if some other signal was received or an error
occurred after getting SIGTERM, as reported by Qingqing Zhou.

While investigating, Tom Lane further noticed that if autovacuum had
been disabled in the config file, it would misbehave by trying to start
a new worker instead of bailing out immediately -- it would consider
itself as invoked in emergency mode.

Fix both problems by checking the shutdown flag in a few more places.
These problems have existed since autovacuum was introduced, so
backpatch all the way back.
2015-04-08 13:19:49 -03:00
Bruce Momjian
e4f1e0d842 libpq: add newlines to SSPI error messages
Report by Tom Lane
2015-04-08 10:28:47 -04:00
Bruce Momjian
90a8b1f82b libpq: issue clear error message for nested service files
Previously an odd error message was generated.  Nested service files are
not supported.

Report by David Johnston
2015-04-08 10:26:58 -04:00
Fujii Masao
026fafde91 Fix typo in comment. 2015-04-08 20:55:43 +09:00
Fujii Masao
29407f9774 Add file_ops.c to GETTEXT_FILES in nls.mk.
Since file_ops.c contains translatable strings, it should have been listed
in GETTEXT_FILES.
2015-04-08 13:46:58 +09:00
Robert Haas
aea652abd3 Make trace_sort control abbreviation debug output for the text opclass.
This is consistent with what the new numeric suppor for abbreviated keys
now does, and seems much more convenient than having a separate compiler
define to control this debug output.

Peter Geoghegan
2015-04-07 22:45:17 -04:00
Fujii Masao
b216ad7bf1 Mark the second argument of pg_log as the translatable string in nls.mk. 2015-04-08 11:06:25 +09:00
Tom Lane
393de3a098 Fix assorted inconsistent function declarations.
While gcc doesn't complain if you declare a function "static" and then
define it not-static, other compilers do; and in any case the code is
highly misleading this way.  Add the missing "static" keywords to a
couple of recent patches.  Per buildfarm member pademelon.
2015-04-07 16:56:21 -04:00
Heikki Linnakangas
ee075fcb13 Fix reporting of missing or invalid command line arguments in pg_rewind.
pg_fatal never returns, so a multi-line message cannot be printed by
calling it twice.

Michael Paquier and Fujii Masao
2015-04-07 23:28:28 +03:00
Alvaro Herrera
4e17e32f53 Remove variable shadowing
Commit a2e35b53 should have removed the variable declaration in the
inner block, but didn't.  As a result, the returned address might end up
not being what was intended.
2015-04-07 17:14:00 -03:00
Heikki Linnakangas
8a06c36aff Fix process startup in pg_rewind.
Don't allow pg_rewind to run as root on Unix platforms, as any new or
replaced files in the data directory would become owned by root. On Windows,
it can run under a user that has Administrator rights, but a restricted
token needs to be used. This is the same we do e.g. in pg_resetxlog.

Also, add missing set_pglocale_pgservice() call, to fix localization.

Michael Paquier and Fujii Masao
2015-04-07 23:05:25 +03:00
Simon Riggs
1cdf4d0b6a Fix spelling of author's name 2015-04-07 14:04:29 -04:00
Alvaro Herrera
e9a077cad3 pg_event_trigger_dropped_objects: add is_temp column
It now also reports temporary objects dropped that are local to the
backend.  Previously we weren't reporting any temp objects because it
was deemed unnecessary; but as it turns out, it is necessary if we want
to keep close track of DDL command execution inside one session.  Temp
objects are reported as living in schema pg_temp, which works because
such a schema-qualification always refers to the temp objects of the
current session.
2015-04-06 11:40:55 -03:00
Alvaro Herrera
70dc2db7f1 Fix object identities for pg_conversion objects
This was already fixed in 0d906798f, but I failed to update the
array-formatted case.  This is not backpatched, since this only affects
the code path introduced by commit a676201490.
2015-04-06 11:15:13 -03:00
Fujii Masao
5f6a9d056a Fix typo in libpq.sgml.
Back-patch to all supported versions.

Michael Paquier
2015-04-06 12:15:20 +09:00
Tom Lane
73b416b2e4 Suppress clang's unhelpful gripes about -pthread switch being unused.
Considering the number of cases in which "unused" command line arguments
are silently ignored by compilers, it's fairly astonishing that anybody
thought this warning was useful; it's certainly nothing but an annoyance
when building Postgres.  One such case is that neither gcc nor clang
complain about unrecognized -Wno-foo switches, making it more difficult
to figure out whether the switch does anything than one could wish.

Back-patch to 9.3, which is as far back as the patch applies conveniently
(we'd have to back-patch PGAC_PROG_CC_VAR_OPT to go further, and it doesn't
seem worth that).
2015-04-05 13:01:59 -04:00
Simon Riggs
e8fde1f6a0 Remove extraneous > 2015-04-05 12:35:24 -04:00
Simon Riggs
35ecc24407 Add new test files for lock level patch 2015-04-05 12:03:58 -04:00
Andrew Dunstan
cf376a4adc Enable float8-byval as the default for 64 bit MSVC builds
This is a long-standing inconsistency that was probably just missed when
we got 64 bit MSVC builds. This brings the platform into line with all
other systems.
2015-04-05 11:49:49 -04:00
Simon Riggs
0ef0396ae1 Reduce lock levels of some trigger DDL and add FKs
Reduce lock levels to ShareRowExclusive for the following SQL
 CREATE TRIGGER (but not DROP or ALTER)
 ALTER TABLE ENABLE TRIGGER
 ALTER TABLE DISABLE TRIGGER
 ALTER TABLE … ADD CONSTRAINT FOREIGN KEY

Original work by Simon Riggs, extracted and refreshed by Andreas Karlsson
New test cases added by Andreas Karlsson
Reviewed by Noah Misch, Andres Freund, Michael Paquier and Simon Riggs
2015-04-05 11:37:08 -04:00
Tom Lane
ca6805338f Fix incorrect matching of subexpressions in outer-join plan nodes.
Previously we would re-use input subexpressions in all expression trees
attached to a Join plan node.  However, if it's an outer join and the
subexpression appears in the nullable-side input, this is potentially
incorrect for apparently-matching subexpressions that came from above
the outer join (ie, targetlist and qpqual expressions), because the
executor will treat the subexpression value as NULL when maybe it should
not be.

The case is fairly hard to hit because (a) you need a non-strict
subexpression (else NULL is correct), and (b) we don't usually compute
expressions in the outputs of non-toplevel plan nodes.  But we might do
so if the expressions are sort keys for a mergejoin, for example.

Probably in the long run we should make a more explicit distinction between
Vars appearing above and below an outer join, but that will be a major
planner redesign and not at all back-patchable.  For the moment, just hack
set_join_references so that it will not match any non-Var expressions
coming from nullable inputs to expressions that came from above the join.
(This is somewhat overkill, in that a strict expression could still be
matched, but it doesn't seem worth the effort to check that.)

Per report from Qingqing Zhou.  The added regression test case is based
on his example.

This has been broken for a very long time, so back-patch to all active
branches.
2015-04-04 19:55:15 -04:00