Commit Graph

32697 Commits

Author SHA1 Message Date
Simon Riggs
9aceb6ab3c Refactor xlog.c to create src/backend/postmaster/startup.c
Startup process now has its own dedicated file, just like all other
special/background processes. Reduces role and size of xlog.c
2011-11-02 14:25:01 +00:00
Simon Riggs
86e3364899 Derive oldestActiveXid at correct time for Hot Standby.
There was a timing window between when oldestActiveXid was derived
and when it should have been derived that only shows itself under
heavy load. Move code around to ensure correct timing of derivation.
No change to StartupSUBTRANS() code, which is where this failed.

Bug report by Chris Redekop
2011-11-02 08:54:56 +00:00
Simon Riggs
10b7c686e5 Start Hot Standby faster when initial snapshot is incomplete.
If the initial snapshot had overflowed then we can start whenever
the latest snapshot is empty, not overflowed or as we did already,
start when the xmin on primary was higher than xmax of our starting
snapshot, which proves we have full snapshot data.

Bug report by Chris Redekop
2011-11-02 08:47:43 +00:00
Simon Riggs
2296e62a32 Remove spurious entry from missed catch while patch juggling 2011-11-02 08:37:52 +00:00
Simon Riggs
f8409b39d1 Fix timing of Startup CLOG and MultiXact during Hot Standby
Patch by me, bug report by Chris Redekop, analysis by Florian Pflug
2011-11-02 08:07:44 +00:00
Robert Haas
c2891b46a4 Initialize myProcLocks queues just once, at postmaster startup.
In assert-enabled builds, we assert during the shutdown sequence that
the queues have been properly emptied, and during process startup that
we are inheriting empty queues.  In non-assert enabled builds, we just
save a few cycles.
2011-11-01 22:44:54 -04:00
Tom Lane
391af9f784 Preserve Var location information during flatten_join_alias_vars.
This allows us to give correct syntax error pointers when complaining
about ungrouped variables in a join query with aggregates or GROUP BY.
It's pretty much irrelevant for the planner's use of the function, though
perhaps it might aid debugging sometimes.
2011-11-01 22:13:11 -04:00
Tom Lane
08e261cbc9 Fix race condition with toast table access from a stale syscache entry.
If a tuple in a syscache contains an out-of-line toasted field, and we
try to fetch that field shortly after some other transaction has committed
an update or deletion of the tuple, there is a race condition: vacuum
could come along and remove the toast tuples before we can fetch them.
This leads to transient failures like "missing chunk number 0 for toast
value NNNNN in pg_toast_2619", as seen in recent reports from Andrew
Hammond and Tim Uckun.

The design idea of syscache is that access to stale syscache entries
should be prevented by relation-level locks, but that fails for at least
two cases where toasted fields are possible: ANALYZE updates pg_statistic
rows without locking out sessions that might want to plan queries on the
same table, and CREATE OR REPLACE FUNCTION updates pg_proc rows without
any meaningful lock at all.

The least risky fix seems to be an idea that Heikki suggested when we
were dealing with a related problem back in August: forcibly detoast any
out-of-line fields before putting a tuple into syscache in the first place.
This avoids the problem because at the time we fetch the parent tuple from
the catalog, we should be holding an MVCC snapshot that will prevent
removal of the toast tuples, even if the parent tuple is outdated
immediately after we fetch it.  (Note: I'm not convinced that this
statement holds true at every instant where we could be fetching a syscache
entry at all, but it does appear to hold true at the times where we could
fetch an entry that could have a toasted field.  We will need to be a bit
wary of adding toast tables to low-level catalogs that don't have them
already.)  An additional benefit is that subsequent uses of the syscache
entry should be faster, since they won't have to detoast the field.

Back-patch to all supported versions.  The problem is significantly harder
to reproduce in pre-9.0 releases, because of their willingness to flush
every entry in a syscache whenever the underlying catalog is vacuumed
(cf CatalogCacheFlushRelation); but there is still a window for trouble.
2011-11-01 19:49:58 -04:00
Peter Eisentraut
654e1f96b0 Clean up whitespace and indentation in parser and scanner files
These are not touched by pgindent, so clean them up a bit manually.
2011-11-01 21:51:30 +02:00
Simon Riggs
f3ebaad45b Comment changes to show bgwriter no longer performs checkpoints. 2011-11-01 18:48:47 +00:00
Simon Riggs
3ba182056f Have checkpointer send stats once each processing loop.
Noted by Fujii Masao
2011-11-01 18:38:27 +00:00
Bruce Momjian
09d1174e5a Update pg_upgrade comment on missing 'postgres' database. 2011-11-01 14:34:35 -04:00
Simon Riggs
bf405ba8e4 Add new file for checkpointer.c 2011-11-01 18:07:29 +00:00
Bruce Momjian
a50d860ae1 Allow pg_upgrade to upgrade an old cluster that doesn't have a
'postgres' database.
2011-11-01 13:49:48 -04:00
Simon Riggs
806a2aee37 Split work of bgwriter between 2 processes: bgwriter and checkpointer.
bgwriter is now a much less important process, responsible for page
cleaning duties only. checkpointer is now responsible for checkpoints
and so has a key role in shutdown. Later patches will correct doc
references to the now old idea that bgwriter performs checkpoints.
Has beneficial effect on performance at high write rates, but mainly
refactoring to more easily allow changes for power reduction by
simplifying previously tortuous code around required to allow page
cleaning and checkpointing to time slice in the same process.

Patch by me, Review by Dickson Guedes
2011-11-01 17:14:47 +00:00
Magnus Hagander
589adb86ee Document that multiple LDAP servers can be specified 2011-11-01 15:44:26 +01:00
Tom Lane
6980f817e8 Stop btree indexscans upon reaching nulls in either direction.
The existing scan-direction-sensitive tests were overly complex, and
failed to stop the scan in cases where it's perfectly legitimate to do so.
Per bug #6278 from Maksym Boguk.

Back-patch to 8.3, which is as far back as the patch applies easily.
Doesn't seem worth sweating over a relatively minor performance issue in
8.2 at this late date.  (But note that this was a performance regression
from 8.1 and before, so 8.2 is being left as an outlier.)
2011-10-31 16:40:04 -04:00
Tom Lane
6743a878a4 Support more locale-specific formatting options in cash_out().
The POSIX spec defines locale fields for controlling the ordering of the
value, sign, and currency symbol in monetary output, but cash_out only
supported a small subset of these options.  Fully implement p/n_sign_posn,
p/n_cs_precedes, and p/n_sep_by_space per spec.  Fix up cash_in so that
it will accept all these format variants.

Also, make sure that thousands_sep is only inserted to the left of the
decimal point, as required by spec.

Per bug #6144 from Eduard Kracmar and discussion of bug #6277.  This patch
includes some ideas from Alexander Lakhin's proposed patch, though it is
very different in detail.
2011-10-30 15:02:58 -04:00
Tom Lane
eb5834d5af Further improvement of make_greater_string.
Make sure that it considers all the possibilities that the old code did,
instead of trying only one possibility per character position.  To keep the
runtime in bounds, instead tweak the character incrementers to not try
every possible multibyte character code.  Remove unnecessary logic to
restore the old character value on failure.  Additional comment and
formatting cleanup.
2011-10-30 12:22:11 -04:00
Robert Haas
fae54e4a16 Update visibilitymap.c header comments.
Recent work on index-only scans left this somewhat out of date.
2011-10-29 14:46:59 -04:00
Tom Lane
7609239f3e Fix assorted bogosities in cash_in() and cash_out().
cash_out failed to handle multiple-byte thousands separators, as per bug
#6277 from Alexander Law.  In addition, cash_in didn't handle that either,
nor could it handle multiple-byte positive_sign.  Both routines failed to
support multiple-byte mon_decimal_point, which I did not think was worth
changing, but at least now they check for the possibility and fall back to
using '.' rather than emitting invalid output.  Also, make cash_in handle
trailing negative signs, which formerly it would reject.  Since cash_out
generates trailing negative signs whenever the locale tells it to, this
last omission represents a fail-to-reload-dumped-data bug.  IMO that
justifies patching this all the way back.
2011-10-29 14:32:06 -04:00
Robert Haas
78d523b633 Improve make_greater_string() with encoding-specific incrementers.
This infrastructure doesn't in any way guarantee that the character
we produce will sort before the one we incremented; but it does at least
make it much more likely that we'll end up with something that is a valid
character, which improves our chances.

Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 14:22:20 -04:00
Bruce Momjian
51eba98cf4 Remove pg_upgrade dependency on the 'postgres' database existing in the
new cluster.   vacuumdb, used by pg_upgrade, still has this dependency.
2011-10-28 21:19:21 -04:00
Robert Haas
53f1ca59b5 Allow hint bits to be set sooner for temporary and unlogged tables.
We need not wait until the commit record is durably on disk, because
in the event of a crash the page we're updating with hint bits will
be gone anyway.  Per off-list report from Heikki Linnakangas, this
can significantly degrade the performance of unlogged tables; I was
able to show a 2x speedup from this patch on a pgbench run with scale
factor 15.  In practice, this will mostly help small, heavily updated
tables, because on larger tables you're unlikely to run into the same
row again before the commit record makes it out to disk.
2011-10-28 17:08:09 -04:00
Robert Haas
b6335a3f1b Demote some sanity checks in BufferIsValid() to assertions.
Testing reveals that this macro is a hot-spot for index-only-scans.
Per discussion with Tom Lane.
2011-10-28 17:04:22 -04:00
Robert Haas
deb1580324 Remove hard-coded "\connect postgres" from pg_dumpall.
This doesn't appear to accompish anything useful, and does make the
restore fail if the postgres database happens to have been dropped.
2011-10-28 15:50:42 -04:00
Tom Lane
74812624f2 De-parallelize ecpg build some more.
Make sure ecpg/include/ is rebuilt before the other subdirectories,
so that ecpg_config.h is up to date.  This is not likely to matter
during production builds, only development, so no back-patch.
2011-10-28 15:16:40 -04:00
Robert Haas
9cf12dfd4e Clarify that ORDER BY/FOR UPDATE can't malfunction at higher iso levels.
Kevin Grittner
2011-10-28 12:02:04 -04:00
Robert Haas
6c21105fb4 Change "and and" to "and".
Report by Vik Reykja, patch by Kevin Grittner.
2011-10-28 11:59:55 -04:00
Bruce Momjian
9846dcfb9e Clarify pg_upgrade error message that the 'postgres' database must exist
in the old cluster.
2011-10-28 08:09:15 -04:00
Tom Lane
ece12659cf Update docs to point to the timezone library's new home at IANA.
The recent unpleasantness with copyrights has accelerated a move that
was already in planning.
2011-10-27 23:09:05 -04:00
Bruce Momjian
38f3c7c4dd Update pg_upgrade testing instructions. 2011-10-27 21:29:03 -04:00
Heikki Linnakangas
cbf65509bb Fix the number of lwlocks needed by the "fast path" lock patch. It needs
one lock per backend or auxiliary process - the need for a lock for each
aux processes was not accounted for in NumLWLocks(). No-one noticed,
because the three locks needed for the three aux processes fit into the
few extra lwlocks we allocate for 3rd party modules that don't call
RequestAddinLWLocks() (NUM_USER_DEFINED_LWLOCKS, 4 by default).
2011-10-27 22:39:58 +03:00
Tom Lane
051d1ba7a0 Avoid recursion while processing ELSIF lists in plpgsql.
The original implementation of ELSIF in plpgsql converted the construct
into nested simple IF statements.  This was prone to stack overflow with
long ELSIF lists, in two different ways.  First, it's difficult to generate
the parsetree without using right-recursion in the bison grammar, and
that's prone to parser stack overflow since nothing can be reduced until
the whole list has been read.  Second, we'd recurse during execution, thus
creating an unnecessary risk of execution-time stack overflow.  Rewrite
so that the ELSIF list is represented as a flat list, scanned via iteration
not recursion, and generated through left-recursion in the grammar.
Per a gripe from Håvard Kongsgård.
2011-10-27 15:21:51 -04:00
Tom Lane
756a4ed5ad Add simple script to check for right recursion in Bison grammars.
We should generally use left-recursion not right-recursion to parse lists.
Bison hasn't got any built-in way to check for this type of inefficiency,
and I didn't find anything on the net in a quick search, so I wrote a
little Perl script to do it.  Add to src/tools/ so we don't have to
re-invent this wheel next time we wonder if we're doing anything stupid.

Currently, the only place that seems to need fixing is plpgsql's stmt_else
production, so the problem doesn't appear to be common enough to warrant
trying to include such a test in our standard build process.  If we did
want to do that, we'd need a way to ignore some false positives, such as
a_expr := '-' a_expr
2011-10-27 13:50:57 -04:00
Tom Lane
bf82013631 Typo fixes.
expect -> except, noted by Andrew Dunstan.  Also, "cannot" seems more
readable here than "can not", per David Wheeler.
2011-10-26 18:04:13 -04:00
Tom Lane
3e4b3465b6 Improve planner's ability to recognize cases where an IN's RHS is unique.
If the right-hand side of a semijoin is unique, then we can treat it like a
normal join (or another way to say that is: we don't need to explicitly
unique-ify the data before doing it as a normal join).  We were recognizing
such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP
BY decoration, but there's another way: if the RHS is a plain relation with
unique indexes, we can check if any of the indexes prove the output is
unique.  Most of the infrastructure for that was there already in the join
removal code, though I had to rearrange it a bit.  Per reflection about a
recent example in pgsql-performance.
2011-10-26 17:52:29 -04:00
Bruce Momjian
360429e1d1 Fix pg_bsd_indent bug where newlines were not being trimmed from typedef
lines.  Update pg_bsd_indent required version to 1.1 (and update ftp
site).

Problem reported by Magnus.
2011-10-26 17:24:19 -04:00
Magnus Hagander
d9bae53173 Implement streaming xlog for backup tools
Add option for parallel streaming of the transaction log while a
base backup is running, to get the logfiles before the server has
removed them.

Also add a tool called pg_receivexlog, which streams the transaction
log into files, creating a log archive without having to wait for
segments to complete, thus decreasing the window of data loss without
having to waste space using archive_timeout. This works best in
combination with archive_command - suggested usage docs etc coming later.
2011-10-26 20:13:33 +02:00
Magnus Hagander
2b64f3f17a MingW doesn't support wcstombs_s()... 2011-10-26 19:26:13 +02:00
Tom Lane
1e3b21dd5e Change FK trigger naming convention to fix self-referential FKs.
Use names like "RI_ConstraintTrigger_a_NNNN" for FK action triggers and
"RI_ConstraintTrigger_c_NNNN" for FK check triggers.  This ensures the
action trigger fires first in self-referential cases where the very same
row update fires both an action and a check trigger.  This change provides
a non-probabilistic solution for bug #6268, at the risk that it could break
client code that is making assumptions about the exact names assigned to
auto-generated FK triggers.  Hence, change this in HEAD only.  No need for
forced initdb since old triggers continue to work fine.
2011-10-26 13:19:42 -04:00
Tom Lane
58958726ff Change FK trigger creation order to better support self-referential FKs.
When a foreign-key constraint references another column of the same table,
row updates will queue both the PK's ON UPDATE action and the FK's CHECK
action in the same event.  The ON UPDATE action must execute first, else
the CHECK will check a non-final state of the row and possibly throw an
inappropriate error, as seen in bug #6268 from Roman Lytovchenko.

Now, the firing order of multiple triggers for the same event is determined
by the sort order of their pg_trigger.tgnames, and the auto-generated names
we use for FK triggers are "RI_ConstraintTrigger_NNNN" where NNNN is the
trigger OID.  So most of the time the firing order is the same as creation
order, and so rearranging the creation order fixes it.

This patch will fail to fix the problem if the OID counter wraps around or
adds a decimal digit (eg, from 99999 to 100000) while we are creating the
triggers for an FK constraint.  Given the small odds of that, and the low
usage of self-referential FKs, we'll live with that solution in the back
branches.  A better fix is to change the auto-generated names for FK
triggers, but it seems unwise to do that in stable branches because there
may be client code that depends on the naming convention.  We'll fix it
that way in HEAD in a separate patch.

Back-patch to all supported branches, since this bug has existed for a long
time.
2011-10-26 13:02:28 -04:00
Magnus Hagander
b0bec068e2 Fix typo 2011-10-25 22:46:14 +02:00
Magnus Hagander
a87b9ae161 Make event_source visible on all platforms
On non-windows platform, we just ignore any value set there.

Noted by Jaime Casanova
2011-10-25 22:40:58 +02:00
Magnus Hagander
9c4c8c8468 Remove argument decoration that appears unsupported on mingw 2011-10-25 21:30:53 +02:00
Magnus Hagander
d8ea33f2c0 Support configurable eventlog application names on Windows
This allows different instances to use the eventlog with different
identifiers, by setting the event_source GUC, similar to how
syslog_ident works.

Original patch by MauMau, heavily modified by Magnus Hagander
2011-10-25 20:02:55 +02:00
Alvaro Herrera
90d8e8ff7e Add debugging aid in isolationtester 2011-10-24 22:14:22 -03:00
Magnus Hagander
a74a5f5913 Make TABLE tab complation in psql include all relations
Not just tables, since views also work fine with the
TABLE command.
2011-10-24 13:22:59 +02:00
Tom Lane
8140c1bcf3 Make psql support tab completion of EXECUTE <prepared-statement-name>.
Andreas Karlsson, reviewed by Josh Kupershmidt
2011-10-23 19:25:34 -04:00
Tom Lane
7299778a95 Improve git_changelog's handling of inconsistent commit orderings.
Use the CommitDate not the AuthorDate, as the former is representative of
the order in which things went into the main repository, and the latter
isn't very; we now have instances where the AuthorDate is as much as a
month before the patch really went in.  Also, get rid of the "commit order
inversions" heuristic, which turns out not to do anything very desirable.
Instead we just print commits in strict timestamp order, interpreting the
"timestamp" of a merged commit as its timestamp on the newest branch it
appears in.  This fixes some cases where very ancient commits were being
printed relatively early in the report.
2011-10-23 14:34:36 -04:00