Commit Graph

29375 Commits

Author SHA1 Message Date
Tom Lane
350cb921ae Restructure hba.c to replace 3 parallel lists with single list of structs.
tokenize_file() now returns a single list of TokenizedLine structs,
carrying the same information as before.  We were otherwise going to grow a
fourth list to deal with error messages, and that was getting a bit silly.

Haribabu Kommi, revised a bit by me

Discussion: https://postgr.es/m/CAJrrPGfbgbKsjYp=bgZXhMcgxoaGSoBb9fyjrDoOW_YymXv1Kw@mail.gmail.com
2017-01-27 13:43:00 -05:00
Tom Lane
fde5c03792 Improve comments about ProcessUtility's queryString parameter.
Per discussion with Craig Ringer.
2017-01-27 10:02:04 -05:00
Tom Lane
fefb86b147 Orthography fixes for new castNode() macro.
Clean up hastily-composed comment.  Normalize whitespace.

Erik Rijkers and myself
2017-01-27 08:33:58 -05:00
Tom Lane
7afd56c3c6 Use castNode() in a bunch of statement-list-related code.
When I wrote commit ab1f0c822, I really missed the castNode() macro that
Peter E. had proposed shortly before.  This back-fills the uses I would
have put it to.  It's probably not all that significant, but there are
more assertions here than there were before, and conceivably they will
help catch any bugs associated with those representation changes.

I left behind a number of usages like "(Query *) copyObject(query_var)".
Those could have been converted as well, but Peter has proposed another
notational improvement that would handle copyObject cases automatically,
so I let that be for now.
2017-01-26 22:09:34 -05:00
Andres Freund
9ba8a9ce45 Use the new castNode() macro in a number of places.
This is far from a pervasive conversion, but it's a good starting
point.

Author: Peter Eisentraut, with some minor changes by me
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/c5d387d9-3440-f5e0-f9d4-71d53b9fbe52@2ndquadrant.com
2017-01-26 16:47:03 -08:00
Andres Freund
5bcab11142 Add castNode(type, ptr) for safe casting between NodeTag based types.
The new function allows to cast from one NodeTag based type to
another, while asserting that the conversion is valid.  This replaces
the common pattern of doing a cast and a Assert(IsA(ptr, type))
close-by.

As this seems likely to be used pervasively, we decided to backpatch
this change the addition of this macro. Otherwise backpatched fixes
are more likely not to work on back-branches.

On branches before 9.6, where we do not yet rely on inline functions
being available, the type assertion is only performed if PG_USE_INLINE
support is detected. The cast obviously is performed regardless.

For the benefit of verifying the macro compiles in the back-branches,
this commit contains a single use of the new macro. On master, a
somewhat larger conversion will be committed separately.

Author: Peter Eisentraut and Andres Freund
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/c5d387d9-3440-f5e0-f9d4-71d53b9fbe52@2ndquadrant.com
Backpatch: 9.2-
2017-01-26 16:47:03 -08:00
Alvaro Herrera
331f8c311b Remove test for COMMENT ON DATABASE
Our current DDL only allows a database name to be specified in COMMENT
ON DATABASE, which Andrew Dunstan reports to make this test fail on the
buildfarm.  Remove the line until we gain a DDL command that allows the
current database to be operated on without having the specify it by
name.

Backpatch to 9.5, where these tests appeared.

Discussion: https://postgr.es/m/e6084b89-07a7-7e57-51ee-d7b8fc9ec864@2ndQuadrant.com
2017-01-26 17:45:22 -03:00
Peter Eisentraut
e630faacd8 Fill in no_priv_msg for publications and subscriptions
Even though these messages are not used yet, we should keep the list
complete.
2017-01-26 15:38:13 -05:00
Peter Eisentraut
9c18104c74 Simplify sequence test
We maintained two separate expected files because log_cnt could be one
of two values.  Rewrite the test so that we only need one file.

Reviewed-by: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-01-26 15:23:25 -05:00
Simon Riggs
e8ee3d6b85 Check interrupts during hot standby waits 2017-01-26 18:59:58 +00:00
Peter Eisentraut
2a2bdcab2c Add object_address tests for publications and subscriptions
Add test cases to object_address.sql to test the new logical replication
related object classes, and fix some small bugs discovered by that.
2017-01-26 13:21:22 -05:00
Simon Riggs
ec4b975016 Reset hot standby xmin on master after restart
Hot_standby_feedback could be reset by reload and worked correctly, but if
the server was restarted rather than reloaded the xmin was not reset.
Force reset always if hot_standby_feedback is enabled at startup.

Ants Aasma, Craig Ringer

Reported-by: Ants Aasma
2017-01-26 18:14:02 +00:00
Tom Lane
9d4ca01314 Ensure that a tsquery like '!foo' matches empty tsvectors.
!foo means "the tsvector does not contain foo", and therefore it should
match an empty tsvector.  ts_match_vq() overenthusiastically supposed
that an empty tsvector could never match any query, so it forcibly
returned FALSE, the wrong answer.  Remove the premature optimization.

Our behavior on this point was inconsistent, because while seqscans and
GIST index searches both failed to match empty tsvectors, GIN index
searches would find them, since GIN scans don't rely on ts_match_vq().
That makes this certainly a bug, not a debatable definition disagreement,
so back-patch to all supported branches.

Report and diagnosis by Tom Dunstan (bug #14515); added test cases by me.

Discussion: https://postgr.es/m/20170126025524.1434.97828@wrigleys.postgresql.org
2017-01-26 12:18:07 -05:00
Fujii Masao
bdadf36eb4 Fix typo in description for pg_replication_origin_advance function. 2017-01-27 00:42:33 +09:00
Peter Eisentraut
5a366b4ff4 Fix typo: pg_statistics -> pg_statistic 2017-01-25 14:38:33 -05:00
Tom Lane
f7c6246240 Introduce convenience macros to hide JsonbContainer header accesses better.
This improves readability a bit and may make future improvements easier.

In passing, make sure that the JB_ROOT_IS_XXX macros deliver boolean (0/1)
results; the previous coding was a bug hazard, though no actual bugs are
known.

Nikita Glukhov, extended a bit by me

Discussion: https://postgr.es/m/9e21a39c-c1d7-b9b5-44a0-c5345a5029f6@postgrespro.ru
2017-01-25 13:28:38 -05:00
Peter Eisentraut
3d9e73ea5f Update copyright years in some recently added files 2017-01-25 12:32:05 -05:00
Peter Eisentraut
65df150a18 Close replication connection when slot creation errors
From: Petr Jelinek <pjmodos@pjmodos.net>
2017-01-25 10:47:53 -05:00
Tom Lane
aebeb4790c Remove vestigial resolveUnknown arguments from transformSortClause etc.
There's really no situation where we don't want these unknown-to-text
conversions to happen.  The alternative is failure anyway, and the one
caller that was passing "false" did so only because it expected the
case could not arise.  Might as well simplify the code.

Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 09:35:00 -05:00
Tom Lane
d8d32d9a56 Make UNKNOWN into an actual pseudo-type.
Previously, type "unknown" was labeled as a base type in pg_type, which
perhaps had some sense to it because you were allowed to create tables with
unknown-type columns.  But now that we don't allow that, it makes more
sense to label it a pseudo-type.  This has the additional effects of
forbidding use of "unknown" as a domain base type, cast source or target
type, PL function argument or result type, or plpgsql local variable type;
all of which seem like good holes to plug.

Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 09:27:09 -05:00
Tom Lane
1e7c4bb004 Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column).  This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on.  For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison".  Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.

To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command.  In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.

Also, change creation of an unknown-type column in a relation from a
warning to a hard error.  The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE.  We want to forbid that because
it's nothing but a foot-gun.

This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have.  Add a checking pass to detect that.  While at it,
we can detect tables or composite types that would fail, essentially
for free.  Those would fail safely anyway later on, but we might as
well fail earlier.

This patch is by me, but it owes something to previous investigations
by Rahila Syed.  Also thanks to Ashutosh Bapat and Michael Paquier for
review.

Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 09:17:24 -05:00
Robert Haas
3838074f86 Be more aggressive in avoiding tuple conversion.
According to the comments in tupconvert.c, it's necessary to perform
tuple conversion when either table has OIDs, and this was previously
checked by ensuring that the tdtypeid value matched between the tables
in question.  However, that's overly stringent: we have access to
tdhasoid and can test directly whether OIDs are present, which lets us
avoid conversion in cases where the type OIDs are different but the
tuple descriptors are entirely the same (and neither has OIDs).  This
is useful to the partitioning code, which can thereby avoid converting
tuples when inserting into a partition whose columns appear in the
same order as the parent columns, the normal case.  It's possible
for the tuple routing code to avoid some additional overhead in this
case as well, so do that, too.

It's not clear whether it would be OK to skip this when both tables
have OIDs: do callers count on this to build a new tuple (losing the
previous OID) in such instances?  Until we figure it out, leave the
behavior in that case alone.

Amit Langote, reviewed by me.
2017-01-24 21:53:38 -05:00
Tom Lane
7fa7bf18e4 Use non-conflicting table names in new regression test case.
Commit 587cda35c added a test to updatable_views.sql that created
tables named the same as tables used by the concurrent inherit.sql
script.  Unsurprisingly, this results in random failures.
Pick different names.

Per buildfarm.
2017-01-24 19:02:22 -05:00
Peter Eisentraut
da4d1c0c15 pg_dump: Fix some schema issues when dumping sequences
In the new code for selecting sequence data from pg_sequence, set the
schema to pg_catalog instead of the sequences own schema, and refer to
the sequence by OID instead of name, which was missing a schema
qualification.

Reported-by: Stephen Frost <sfrost@snowman.net>
2017-01-24 17:19:55 -05:00
Tom Lane
ba005f193d Allow password file name to be specified as a libpq connection parameter.
Formerly an alternate password file could only be selected via the
environment variable PGPASSFILE; now it can also be selected via a
new connection parameter "passfile", corresponding to the conventions
for most other connection parameters.  There was some concern about
this creating a security weakness, but it was agreed that that argument
was pretty thin, and there are clear use-cases for handling password
files this way.

Julian Markwort, reviewed by Fabien Coelho, some adjustments by me

Discussion: https://postgr.es/m/a4b4f4f1-7b58-a0e8-5268-5f7db8e8ccaa@uni-muenster.de
2017-01-24 17:06:34 -05:00
Robert Haas
d1ecd53947 Add a SHOW command to the replication command language.
This is useful infrastructure for an upcoming proposed patch to
allow the WAL segment size to be changed at initdb time; tools like
pg_basebackup need the ability to interrogate the server setting.
But it also doesn't seem like a bad thing to have independently of
that; it may find other uses in the future.

Robert Haas and Beena Emerson.  (The original patch here was by
Beena, but I rewrote it to such a degree that most of the code
being committed here is mine.)

Discussion: http://postgr.es/m/CA+TgmobNo4qz06wHEmy9DszAre3dYx-WNhHSCbU9SAwf+9Ft6g@mail.gmail.com
2017-01-24 17:04:12 -05:00
Robert Haas
a84069d935 Add a new DestReceiver for printing tuples without catalog access.
If you create a DestReciver of type DestRemote and try to use it from
a replication connection that is not bound to a specific daabase, or
any other hypothetical type of backend that is not bound to a specific
database, it will fail because it doesn't have a pg_proc catalog to
look up properties of the types being printed.  In general, that's
an unavoidable problem, but we can hardwire the properties of a few
builtin types in order to support utility commands.  This new
DestReceiver of type DestRemoteSimple does just that.

Patch by me, reviewed by Michael Paquier.

Discussion: http://postgr.es/m/CA+TgmobNo4qz06wHEmy9DszAre3dYx-WNhHSCbU9SAwf+9Ft6g@mail.gmail.com
2017-01-24 16:53:56 -05:00
Robert Haas
7b4ac19982 Extend index AM API for parallel index scans.
This patch doesn't actually make any index AM parallel-aware, but it
provides the necessary functions at the AM layer to do so.

Rahila Syed, Amit Kapila, Robert Haas
2017-01-24 16:42:58 -05:00
Robert Haas
587cda35ca Fix things so that updatable views work with partitioned tables.
Previously, ExecInitModifyTable was missing handling for WITH CHECK
OPTION, and view_query_is_auto_updatable was missing handling for
RELKIND_PARTITIONED_TABLE.

Amit Langote, reviewed by me.
2017-01-24 15:46:50 -05:00
Robert Haas
132488bfee Set ecxt_scantuple correctly for tuple routing.
In 2ac3ef7a01, we changed things so that
it's possible for a different TupleTableSlot to be used for partitioned
tables at successively lower levels.  If we do end up changing the slot
from the original, we must update ecxt_scantuple to point to the new one
for partition key of the tuple to be computed correctly.

Reported by Rajkumar Raghuwanshi.  Patch by Amit Langote.

Discussion: http://postgr.es/m/CAKcux6%3Dm1qyqB2k6cjniuMMrYXb75O-MB4qGQMu8zg-iGGLjDw%40mail.gmail.com
2017-01-24 15:34:39 -05:00
Robert Haas
27cdb3414b Reindent table partitioning code.
We've accumulated quite a bit of stuff with which pgindent is not
quite happy in this code; clean it up to provide a less-annoying base
for future pgindent runs.
2017-01-24 10:20:02 -05:00
Robert Haas
96e0ccc2b5 Fix incorrect comment: pgtime's tm_mon is 1-based, not 0-based.
The comments in formatting.c already said that tm_mon was 1-based not
0-based, but the comments here disagreed.

Dmitry Fedin
2017-01-24 09:36:17 -05:00
Robert Haas
52df3420cd Remove unused variable.
This was intended to be included in the previous commit,
but I goofed.
2017-01-24 09:08:13 -05:00
Robert Haas
289992c462 Don't invoke arbitrary code inside a possibly-aborted transaction.
The code here previously tried to call the partitioning operator, but
really the right thing to do (and the safe thing to do) is use
datumIsEqual().

Amit Langote, but I expanded the comment and fixed a compiler warning.
2017-01-24 08:57:10 -05:00
Robert Haas
b1ecb9b3fc Fix interaction of partitioned tables with BulkInsertState.
When copying into a partitioned table, the target heap may change from
one tuple to next.  We must ask ReadBufferBI() to get a new buffer
every time such change occurs.  To do that, use new function
ReleaseBulkInsertStatePin().  This fixes the bug that tuples ended up
being inserted into the wrong partition, which occurred exactly
because the wrong buffer was used.

Amit Langote, per a suggestion from Robert Haas.  Some cosmetic
adjustments by me.

Reports by 高增琦 (Gao Zengqi), Venkata B Nagothi, and
Ragnar Ouchterlony.

Discussion: http://postgr.es/m/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail.gmail.com
Discussion: http://postgr.es/m/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40mail.gmail.com
Discussion: http://postgr.es/m/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
Discussion: http://postgr.es/m/CA+TgmoaiZpDVUUN8LZ4jv1qFE_QyR+H9ec+79f5vNczYarg5Zg@mail.gmail.com
2017-01-24 08:50:16 -05:00
Fujii Masao
3eaf03b5d3 Mention logical replication tests in src/test/README.
Craig Ringer
2017-01-24 12:57:25 +09:00
Fujii Masao
dc82f5a640 Be sure to release the lock on failure to launch logical replication worker.
Petr Jelinek
2017-01-24 12:41:09 +09:00
Tatsuo Ishii
73293ebae2 Fix comments in StrategyNotifyBgWriter().
The interface for the function was changed in
d72731a704 but the comments of the
function was not updated.

Patch by Yugo Nagata.
2017-01-24 09:39:11 +09:00
Peter Eisentraut
1a2d9a655a Fix parallel make issue with new fmgrprotos.h
The make rules needed further refinement so that we don't run multiple
generations per build.

reported by Tom Lane
2017-01-23 15:36:27 -05:00
Peter Eisentraut
0bc1207aeb Fix default minimum value for descending sequences
For some reason that is lost in history, a descending sequence would
default its minimum value to -2^63+1 (-PG_INT64_MAX) instead of
-2^63 (PG_INT64_MIN), even though explicitly specifying a minimum value
of -2^63 would work.  Fix this inconsistency by using the full range by
default.

Reported-by: Daniel Verite <daniel@manitou-mail.org>
Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-01-23 14:00:58 -05:00
Peter Eisentraut
46d482814c Don't error when no system locales were found
initdb used to warn about that, but it was changed to an error in
pg_import_system_locales, but some build farm members failed because of
that.  Change it back to a warning.
2017-01-23 13:45:32 -05:00
Peter Eisentraut
70c56a014e Fix NULL pointer access in logical replication workers
From: Petr Jelinek <pjmodos@pjmodos.net>
2017-01-23 12:33:27 -05:00
Peter Eisentraut
6cffe54aef Fix pointer confusion
get_object_address_publication_rel() needed to check *relation, not
relation.  Rename the variables to match style used nearby to avoid the
confusion.
2017-01-23 11:55:06 -05:00
Peter Eisentraut
16a61884b5 Fix memory leaks in libpqwalreceiver
The results of the libpq functions PQescapeIdentifier() and
PQescapeLiteral() must be freed explicitly.  Also handle errors in these
functions better.
2017-01-23 11:06:30 -05:00
Alvaro Herrera
7e26e02eec Prefetch blocks during lazy vacuum's truncation scan
Vacuum truncation scan can be sped up on rotating media by prefetching
blocks in forward direction.  That makes the blocks already present in
memory by the time they are needed, while also letting OS read-ahead
kick in.

The truncate scan has been measured to be five times faster than without
this patch (that was on a slow disk, but it shouldn't hurt on fast
disks.)

Author: Álvaro Herrera, loosely based on a submission by Claudio Freire
Discussion: https://postgr.es/m/CAGTBQpa6NFGO_6g_y_7zQx8L9GcHDSQKYdo1tGuh791z6PYgEg@mail.gmail.com
2017-01-23 12:55:18 -03:00
Tom Lane
3c821466ab Fix example plan in optimizer/README.
Joining three tables only takes two join nodes.  I think when I (tgl)
wrote this, I was envisioning possible additional joins; but since the
example doesn't show any fourth table, it's just confusing to write
a third join node.

Etsuro Fujita

Discussion: https://postgr.es/m/e6cfbaa3-af02-1abc-c25e-8fa5c6bc4e21@lab.ntt.co.jp
2017-01-23 09:38:36 -05:00
Tom Lane
c0ef456b97 Volatile-ize some plperl variables that must survive into PG_CATCH blocks.
This appears to be necessary to fix a failure seen on buildfarm member
sittella.  It shouldn't be necessary according to the letter of the C
standard, because we don't change the values of these variables within
the PG_TRY blocks; but somehow gcc 4.7.2 is dropping the ball.

Discussion: https://postgr.es/m/17555.1485179975@sss.pgh.pa.us
2017-01-23 09:15:49 -05:00
Peter Eisentraut
366d2a3d88 pg_dump: Fix minor memory leak
Missing a destroyPQExpBuffer() in the early exit branch.  The early
exits aren't really necessary.  Most similar functions just proceed
running the rest of the code zero times and clean up at the end.
2017-01-23 08:28:39 -05:00
Peter Eisentraut
5654912907 Fix typo 2017-01-23 08:26:31 -05:00
Tom Lane
90992e0e2f Relocate static function declarations to be after typedefs in jsonfuncs.c.
Project style is to put things in this order, for the good and sufficient
reason that you often need the typedefs in the function declarations.
There already was one function declaration that needed a typedef, which
was randomly placed away from all the other static function declarations
in consequence.  And the submitted patch for better json_populate_record
functionality jumped through even more hoops in order to preserve this
bad idea.

This patch only moves lines from point A to point B, no other changes.
2017-01-22 14:08:26 -05:00