Commit Graph

34097 Commits

Author SHA1 Message Date
Peter Eisentraut
a84bf4922e Avoid extra newlines in XML mapping in table forest mode
found by P. Broennimann
2012-07-12 23:52:50 +03:00
Tom Lane
a36088bcfa Skip text->binary conversion of unnecessary columns in contrib/file_fdw.
When reading from a text- or CSV-format file in file_fdw, the datatype
input routines can consume a significant fraction of the runtime.
Often, the query does not need all the columns, so we can get a useful
speed boost by skipping I/O conversion for unnecessary columns.

To support this, add a "convert_selectively" option to the core COPY code.
This is undocumented and not accessible from SQL (for now, anyway).

Etsuro Fujita, reviewed by KaiGai Kohei
2012-07-12 16:26:59 -04:00
Bruce Momjian
76720bdf1a Remove 'x =- 1' check for pgindent, not needed, per report from Andrew
Dunstan.
2012-07-12 14:37:47 -04:00
Magnus Hagander
058a050ec7 Fix memory and file descriptor leaks in pg_receivexlog/pg_basebackup
When the internal loop mode was added, freeing memory and closing
filedescriptors before returning became important, and a few cases
in the code missed that.

Fujii Masao
2012-07-12 13:33:58 +02:00
Tom Lane
84a42560c8 Add array_remove() and array_replace() functions.
These functions support removing or replacing array element value(s)
matching a given search value.  Although intended mainly to support a
future array-foreign-key feature, they seem useful in their own right.

Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 13:59:35 -04:00
Bruce Momjian
f9951252db Document that Log-Shipping Standby Servers cannot be upgraded by
pg_upgrade.

Backpatch to 9.2.
2012-07-10 23:08:19 -04:00
Tom Lane
01215d61a7 Fix bogus macro definition.
Per buildfarm complaints.
2012-07-10 22:36:11 -04:00
Tatsuo Ishii
1c7a7faa5b Add comments about additional mule-internal charsets from emacs's
source code(lisp/international/mule-conf.el).  These charsets have not
been supported up to now anyway, so this is just for adding
commentary.  Also add mention that we follow emacs's implementation,
not xemacs's.
2012-07-11 08:10:50 +09:00
Tom Lane
60e9c224a1 Fix ASCII case in pg_wchar2mule_with_len.
Also some cosmetic improvements for wchar-to-mblen patch.
2012-07-10 15:59:39 -04:00
Alvaro Herrera
379607c9e8 plperl: Skip setting UTF8 flag when in SQL_ASCII encoding
When in SQL_ASCII encoding, strings passed around are not necessarily
UTF8-safe.  We had already fixed this in some places, but it looks like
we missed some.

I had to backpatch Peter Eisentraut's a8b92b60 to 9.1 in order for this
patch to cherry-pick more cleanly.

Patch from Alex Hunsaker, tweaked by Kyotaro HORIGUCHI and myself.

Some desultory cleanup and comment addition by me, during patch review.

Per bug report from Christoph Berg in
20120209102116.GA14429@msgid.df7cb.de
2012-07-10 15:15:16 -04:00
Alvaro Herrera
fc4a8a6d74 perltidy adjustments to new file 2012-07-10 15:15:16 -04:00
Tom Lane
628cbb50ba Re-implement extraction of fixed prefixes from regular expressions.
To generate btree-indexable conditions from regex WHERE conditions (such as
WHERE indexed_col ~ '^foo'), we need to be able to identify any fixed
prefix that a regex might have; that is, find any string that must be a
prefix of all strings satisfying the regex.  We used to do that with
entirely ad-hoc code that looked at the source text of the regex.  It
didn't know very much about regex syntax, which mostly meant that it would
fail to identify some optimizable cases; but Viktor Rosenfeld reported that
it would produce actively wrong answers for quantified parenthesized
subexpressions, such as '^(foo)?bar'.  Rather than trying to extend the
ad-hoc code to cover this, let's get rid of it altogether in favor of
identifying prefixes by examining the compiled form of a regex.

To do this, I've added a new entry point "pg_regprefix" to the regex library;
hopefully it is defined in a sufficiently general fashion that it can remain
in the library when/if that code gets split out as a standalone project.

Since this bug has been there for a very long time, this fix needs to get
back-patched.  However it depends on some other recent commits (particularly
the addition of wchar-to-database-encoding conversion), so I'll commit this
separately and then go to work on back-porting the necessary fixes.
2012-07-10 14:54:37 -04:00
Tom Lane
00dac6000d Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns.
Previously, pattern_fixed_prefix() was defined to return whatever fixed
prefix it could extract from the pattern, plus the "rest" of the pattern.
That definition was sensible for LIKE patterns, but not so much for
regexes, where reconstituting a valid pattern minus the prefix could be
quite tricky (certainly the existing code wasn't doing that correctly).
Since the only thing that callers ever did with the "rest" of the pattern
was to pass it to like_selectivity() or regex_selectivity(), let's cut out
the middle-man and just have pattern_fixed_prefix's subroutines do this
directly.  Then pattern_fixed_prefix can return a simple selectivity
number, and the question of how to cope with partial patterns is removed
from its API specification.

While at it, adjust the API spec so that callers who don't actually care
about the pattern's selectivity (which is a lot of them) can pass NULL for
the selectivity pointer to skip doing the work of computing a selectivity
estimate.

This patch is only an API refactoring that doesn't actually change any
processing, other than allowing a little bit of useless work to be skipped.
However, it's necessary infrastructure for my upcoming fix to regex prefix
extraction, because after that change there won't be any simple way to
identify the "rest" of the regex, not even to the low level of fidelity
needed by regex_selectivity.  We can cope with that if regex_fixed_prefix
and regex_selectivity communicate directly, but not if we have to work
within the old API.  Hence, back-patch to all active branches.
2012-07-09 23:22:55 -04:00
Tom Lane
e7ef6d7e24 Fix planner to pass correct collation to operator selectivity estimators.
We can do this without creating an API break for estimation functions
by passing the collation using the existing fmgr functionality for
passing an input collation as a hidden parameter.

The need for this was foreseen at the outset, but we didn't get around to
making it happen in 9.1 because of the decision to sort all pg_statistic
histograms according to the database's default collation.  That meant that
selectivity estimators generally need to use the default collation too,
even if they're estimating for an operator that will do something
different.  The reason it's suddenly become more interesting is that
regexp interpretation also uses a collation (for its LC_TYPE not LC_COLLATE
property), and we no longer want to use the wrong collation when examining
regexps during planning.  It's not that the selectivity estimate is likely
to change much from this; rather that we are thinking of caching compiled
regexps during planner estimation, and we won't get the intended benefit
if we cache them with a different collation than the executor will use.

Back-patch to 9.1, both because the regexp change is likely to get
back-patched and because we might as well get this right in all
collation-supporting branches, in case any third-party code wants to
rely on getting the collation.  The patch turns out to be minuscule
now that I've done it ...
2012-07-08 23:51:08 -04:00
Tom Lane
c6aae3042b Simplify and document regex library's compact-NFA representation.
The previous coding abused the first element of a cNFA state's arcs list
to hold a per-state flag bit, which was confusing, undocumented, and not
even particularly efficient.  Get rid of that in favor of a separate
"stflags" vector.  Since there's only one bit in use, I chose to allocate a
char per state; we could possibly replace this with a bitmap at some point,
but that would make accesses a little slower.  It's already about 8X
smaller than before, so let's not get overly tense.

Also document the representation better than it was before, which is to say
not at all.

This patch is a byproduct of investigations towards extracting a "fixed
prefix" string from the compact-NFA representation of regex patterns.
Might need to back-patch it if we decide to back-patch that fix, but for
now it's just code cleanup so I'll just put it in HEAD.
2012-07-07 17:39:50 -04:00
Alvaro Herrera
a184e4db83 Convert libpq regress script to Perl
This should ease its use on the Windows build environment.
2012-07-06 16:45:48 -04:00
Alvaro Herrera
adb9b7d53b Update libpq test expected output
Commit 2b443063 changed wording for some of the error messages, but
neglected updating the regress output to match.
2012-07-06 16:45:47 -04:00
Bruce Momjian
3c9b406420 Run updated copyright.pl on HEAD and 9.2 trees, updating the psql
\copyright output to 2012.

Backpatch to 9.2.
2012-07-06 12:28:18 -04:00
Bruce Momjian
d17c0135cd Have copyright.pl skip updating something that is just the current year,
to avoid producing dups, e.g. 2012-2012

Backpatch to 9.2.
2012-07-06 12:21:43 -04:00
Bruce Momjian
95203e0833 Modify copyright.pl so all lines are processed, not just the first
match, so files that contain embedded copyrights are updated, e.g.
pgsql/help.c.

Backpatch to 9.2.
2012-07-06 11:58:55 -04:00
Bruce Momjian
5198ae8992 Fix copyright.pl to properly skip the .git directory by adding a
basename() qualification.
2012-07-06 11:43:59 -04:00
Bruce Momjian
b9eb808bf2 Fix spacing in copyright.pl after being run with missing regex slash
(now added).

Backpatch to 9.2.
2012-07-06 10:57:08 -04:00
Bruce Momjian
c742d1dbe7 Update pg_upgrade comments for recent configpath fix. 2012-07-06 09:39:22 -04:00
Robert Haas
f6a05fd973 Fix failure of new wchar->mb functions to advance from pointer.
Bug spotted by Tom Lane.
2012-07-05 23:47:53 -04:00
Bruce Momjian
2eeb5eb23f Fix PGDATAOLD and PGDATANEW to properly set pgconfig location, per
report from Tom.

Backpatch to 9.2.
2012-07-05 23:36:43 -04:00
Tom Lane
8525419947 Don't try to trim "../" in join_path_components().
join_path_components() tried to remove leading ".." components from its
tail argument, but it was not nearly bright enough to do so correctly
unless the head argument was (a) absolute and (b) canonicalized.
Rather than try to fix that logic, let's just get rid of it: there is no
correctness reason to remove "..", and cosmetic concerns can be taken
care of by a subsequent canonicalize_path() call.  Per bug #6715 from
Greg Davidson.

Back-patch to all supported branches.  It appears that pre-9.2, this
function is only used with absolute paths as head arguments, which is why
we'd not noticed the breakage before.  However, third-party code might be
expecting this function to work in more general cases, so it seems wise
to back-patch.

In HEAD and 9.2, also make some minor cosmetic improvements to callers.
2012-07-05 17:16:11 -04:00
Heikki Linnakangas
de479e2ed2 Revert part of the previous patch that avoided using PLy_elog().
That caused the plpython_unicode regression test to fail on SQL_ASCII
encoding, as evidenced by the buildfarm. The reason is that with the patch,
you don't get the detail in the error message that you got before. That
detail is actually very informative, so rather than just adjust the expected
output, let's revert that part of the patch for now to make the buildfarm
green again, and figure out some other way to avoid the recursion of
PLy_elog() that doesn't lose the detail.
2012-07-05 23:40:25 +03:00
Heikki Linnakangas
b66de4c6d7 Fix mapping of PostgreSQL encodings to Python encodings.
Windows encodings, "win1252" and so forth, are named differently in Python,
like "cp1252". Also, if the PyUnicode_AsEncodedString() function call fails
for some reason, use a plain ereport(), not a PLy_elog(), to report that
error. That avoids recursion and crash, if PLy_elog() tries to call
PLyUnicode_Bytes() again.

This fixes bug reported by Asif Naeem. Backpatch down to 9.0, before that
plpython didn't even try these conversions.

Jan Urbański, with minor comment improvements by me.
2012-07-05 22:31:29 +03:00
Tom Lane
fc548b2296 Remove support for using wait3() in place of waitpid().
All Unix-oid platforms that we currently support should have waitpid(),
since it's in V2 of the Single Unix Spec.  Our git history shows that
the wait3 code was added to support NextStep, which we officially dropped
support for as of 9.2.  So get rid of the configure test, and simplify the
macro spaghetti in reaper().  Per suggestion from Fujii Masao.
2012-07-05 14:00:40 -04:00
Alvaro Herrera
666d494d19 pg_upgrade: abstract out copying of files from old cluster to new
Currently only pg_clog is copied, but some other directories could need
the same treatment as well, so create a subroutine to do it.

Extracted from my (somewhat larger) FOR KEY SHARE patch.
2012-07-05 11:38:42 -04:00
Magnus Hagander
3644a63984 Fix function argument tab completion for schema-qualified or quoted function names
Dean Rasheed, reviewed by Josh Kupershmidt
2012-07-05 14:06:55 +02:00
Bruce Momjian
539d38757a Fix missing regex slash that caused perltidy to get confused on
copyright.pl.

Backpatch to 9.2.
2012-07-04 21:58:48 -04:00
Bruce Momjian
042d9ffc28 Run newly-configured perltidy script on Perl files.
Run on HEAD and 9.2.
2012-07-04 21:47:49 -04:00
Robert Haas
d7c734841b Reduce messages about implicit indexes and sequences to DEBUG1.
Per recent discussion on pgsql-hackers, these messages are too
chatty for most users.
2012-07-04 20:35:29 -04:00
Bruce Momjian
3e00d33261 Have pg_dump in binary-upgrade mode properly drop user-created
extensions that might exist in the new empty cluster databases, like
plpgsql.

Backpatch to 9.2.
2012-07-04 17:37:01 -04:00
Robert Haas
0fc32c00d7 Fix sample INSTR function to return 0 if third arg is 0.
Albe Laurenz, per a report by Greg Smith that our sample function
doesn't quite match Oracle's behavior.
2012-07-04 17:19:23 -04:00
Robert Haas
72dd6291f2 Add wchar -> mb conversion routines.
This is infrastructure for Alexander Korotkov's work on indexing regular
expression searches.

Alexander Korotkov, with a bit of further hackery on the MULE conversion
by me
2012-07-04 17:10:10 -04:00
Robert Haas
248b5fce06 More doc cleanups for recent shared memory changes.
Josh Kupershmidt
2012-07-04 15:57:48 -04:00
Robert Haas
390bfc643d Documentation cleanups for recent shared memory changes. 2012-07-04 15:56:12 -04:00
Robert Haas
f358428280 Increase the maximum initdb-configured value for shared_buffers to 128MB.
The old value of 32MB has been around for a very long time, and in the
meantime typical system memories have become vastly larger.  Also, now
that we no longer depend on being able to fit the entirety of our
shared memory segment into the system's limit on System V shared
memory, there's a much better chance of the higher limit actually
proving productive.

Per recent discussion on pgsql-hackers.
2012-07-04 15:55:21 -04:00
Robert Haas
17676c785a Make oid2name, pgbench, and vacuumlo set fallback_application_name.
Amit Kapila, reviewed by Shigeru Hanada and Peter Eisentraut,
with some modifications by me.
2012-07-04 15:39:33 -04:00
Magnus Hagander
10e0dd8f91 Remove duplicate, unnecessary, variable declaration 2012-07-04 16:17:30 +02:00
Magnus Hagander
dbc6fcf35d Set the write location in the pg_receivexlog status messages
This makes it possible for the master to track how much data has
actually been written my pg_receivexlog - and not just how much
has been sent towards it.
2012-07-04 15:14:49 +02:00
Magnus Hagander
0c4b468692 Always treat a standby returning an an invalid flush location as async
This ensures that a standby such as pg_receivexlog will not be selected
as sync standby - which would cause the master to block waiting for
a location that could never happen.

Fujii Masao
2012-07-04 15:14:42 +02:00
Magnus Hagander
817d870cf9 Remove reference to default wal_buffers being 8
This hasn't been true since 9.1, when the default was changed to -1.
Remove the reference completely, keeping the discussion of the parameter
and it's shared memory effects on the config page.
2012-07-04 09:23:51 +02:00
Magnus Hagander
51fc406819 Remove references to pgfoundry as recommended hosting platform
pgfoundry is deprectaed and no longer accepting new projects,
so we really shouldn't be directing people there.
2012-07-04 08:59:35 +02:00
Magnus Hagander
d80785e6ed Remove references to PostgreSQL bundled on Solaris
Also remove special references to downloads off pgfoundry since they are
not correct - downloads are done through the main website.
2012-07-04 08:58:31 +02:00
Tom Lane
09022de1f5 Improve documentation about MULE encoding.
This commit improves the comments in pg_wchar.h and creates #define symbols
for some formerly hard-coded values.  No substantive code changes.

Tatsuo Ishii and Tom Lane
2012-07-04 00:29:57 -04:00
Alvaro Herrera
47a2adc83c Forgot an #include in the previous patch :-( 2012-07-03 16:40:15 -04:00
Alvaro Herrera
0c7b9dc7d0 Have REASSIGN OWNED work on extensions, too
Per bug #6593, REASSIGN OWNED fails when the affected role has created
an extension.  Even though the user related to the extension is not
nominally the owner, its OID appears on pg_shdepend and thus causes
problems when the user is to be dropped.

This commit adds code to change the "ownership" of the extension itself,
not of the contained objects.  This is fine because it's currently only
called from REASSIGN OWNED, which would also modify the ownership of the
contained objects.  However, this is not sufficient for a working ALTER
OWNER implementation extension.

Back-patch to 9.1, where extensions were introduced.

Bug #6593 reported by Emiliano Leporati.
2012-07-03 15:09:59 -04:00