Adjust behavior of single-user -j mode for better initdb error reporting.

Previously, -j caused the entire input file to be read in and executed as
a single command string.  That's undesirable, not least because any error
causes the entire file to be regurgitated as the "failing query".  Some
experimentation suggests a better rule: end the command string when we see
a semicolon immediately followed by two newlines, ie, an empty line after
a query.  This serves nicely to break up the existing examples such as
information_schema.sql and system_views.sql.  A limitation is that it's
no longer possible to write such a sequence within a string literal or
multiline comment in a file meant to be read with -j; but there are no
instances of such a problem within the data currently used by initdb.
(If someone does make such a mistake in future, it'll be obvious because
they'll get an unterminated-literal or unterminated-comment syntax error.)
Other than that, there shouldn't be any negative consequences; you're not
forced to end statements that way, it's just a better idea in most cases.

In passing, remove src/include/tcop/tcopdebug.h, which is dead code
because it's not included anywhere, and hasn't been for more than
ten years.  One of the debug-support symbols it purported to describe
has been unreferenced for at least the same amount of time, and the
other is removed by this commit on the grounds that it was useless:
forcing -j mode all the time would have broken initdb.  The lack of
complaints about that, or about the missing inclusion, shows that
no one has tried to use TCOP_DONTUSENEWLINE in many years.

doc/src/sgml/ref/postgres-ref.sgml

@@ -529,7 +529,9 @@ PostgreSQL documentation
     </indexterm>
 
     <para>
-     The following options only apply to the single-user mode.
+     The following options only apply to the single-user mode
+     (see <xref linkend="app-postgres-single-user"
+     endterm="app-postgres-single-user-title">).
     </para>
 
     <variablelist>
@@ -558,7 +560,7 @@ PostgreSQL documentation
       <term><option>-E</option></term>
       <listitem>
        <para>
-        Echo all commands.
+        Echo all commands to standard output before executing them.
        </para>
       </listitem>
      </varlistentry>
@@ -567,7 +569,8 @@ PostgreSQL documentation
       <term><option>-j</option></term>
       <listitem>
        <para>
-        Disables use of newline as a statement delimiter.
+        Use semicolon followed by two newlines, rather than just newline,
+        as the command entry terminator.
        </para>
       </listitem>
      </varlistentry>
@@ -760,8 +763,8 @@ PostgreSQL documentation
   </para>
  </refsect1>
 
- <refsect1>
-  <title>Usage</title>
+ <refsect1 id="app-postgres-single-user">
+  <title id="app-postgres-single-user-title">Single-User Mode</title>
 
    <para>
     To start a single-user mode server, use a command like
@@ -778,30 +781,40 @@ PostgreSQL documentation
     entry terminator; there is no intelligence about semicolons,
     as there is in <application>psql</>.  To continue a command
     across multiple lines, you must type backslash just before each
-    newline except the last one.
+    newline except the last one.  The backslash and adjacent newline are
+    both dropped from the input command.  Note that this will happen even
+    when within a string literal or comment.
    </para>
 
    <para>
-    But if you use the <option>-j</> command line switch, then newline does
-    not terminate command entry.  In this case, the server will read the standard input
-    until the end-of-file (<acronym>EOF</>) marker, then
-    process the input as a single command string.  Backslash-newline is not
-    treated specially in this case.
+    But if you use the <option>-j</> command line switch, a single newline
+    does not terminate command entry; instead, the sequence
+    semicolon-newline-newline does.  That is, type a semicolon immediately
+    followed by a completely empty line.  Backslash-newline is not
+    treated specially in this mode.  Again, there is no intelligence about
+    such a sequence appearing within a string literal or comment.
+   </para>
+
+   <para>
+    In either input mode, if you type a semicolon that is not just before or
+    part of a command entry terminator, it is considered a command separator.
+    When you do type a command entry terminator, the multiple statements
+    you've entered will be executed as a single transaction.
    </para>
 
    <para>
     To quit the session, type <acronym>EOF</acronym>
     (<keycombo action="simul"><keycap>Control</><keycap>D</></>, usually).
-    If you've
-    used <option>-j</>, two consecutive <acronym>EOF</>s are needed to exit.
+    If you've entered any text since the last command entry terminator,
+    then <acronym>EOF</acronym> will be taken as a command entry terminator,
+    and another <acronym>EOF</> will be needed to exit.
    </para>
 
    <para>
     Note that the single-user mode server does not provide sophisticated
     line-editing features (no command history, for example).
-    Single-User mode also does not do any background processing, like
-    automatic checkpoints.
-
+    Single-user mode also does not do any background processing, such as
+    automatic checkpoints or replication.
    </para>
  </refsect1>
 

src/backend/catalog/information_schema.sql

@@ -5,6 +5,14 @@
  * Copyright (c) 2003-2015, PostgreSQL Global Development Group
  *
  * src/backend/catalog/information_schema.sql
+ *
+ * Note: this file is read in single-user -j mode, which means that the
+ * command terminator is semicolon-newline-newline; whenever the backend
+ * sees that, it stops and executes what it's got.  If you write a lot of
+ * statements without empty lines between, they'll all get quoted to you
+ * in any error message about one of them, so don't do that.  Also, you
+ * cannot write a semicolon immediately followed by an empty line in a
+ * string literal (including a function body!) or a multiline comment.
  */
 
 /*

src/backend/catalog/system_views.sql

@@ -4,6 +4,14 @@
  * Copyright (c) 1996-2015, PostgreSQL Global Development Group
  *
  * src/backend/catalog/system_views.sql
+ *
+ * Note: this file is read in single-user -j mode, which means that the
+ * command terminator is semicolon-newline-newline; whenever the backend
+ * sees that, it stops and executes what it's got.  If you write a lot of
+ * statements without empty lines between, they'll all get quoted to you
+ * in any error message about one of them, so don't do that.  Also, you
+ * cannot write a semicolon immediately followed by an empty line in a
+ * string literal (including a function body!) or a multiline comment.
  */
 
 CREATE VIEW pg_roles AS

src/backend/snowball/snowball.sql.in

@@ -1,6 +1,22 @@
--- src/backend/snowball/snowball.sql.in$
+/*
+ * text search configuration for _LANGNAME_ language
+ *
+ * Copyright (c) 2007-2015, PostgreSQL Global Development Group
+ *
+ * src/backend/snowball/snowball.sql.in
+ *
+ * _LANGNAME_ and certain other macros are replaced for each language;
+ * see the Makefile for details.
+ *
+ * Note: this file is read in single-user -j mode, which means that the
+ * command terminator is semicolon-newline-newline; whenever the backend
+ * sees that, it stops and executes what it's got.  If you write a lot of
+ * statements without empty lines between, they'll all get quoted to you
+ * in any error message about one of them, so don't do that.  Also, you
+ * cannot write a semicolon immediately followed by an empty line in a
+ * string literal (including a function body!) or a multiline comment.
+ */
 
--- text search configuration for _LANGNAME_ language
 CREATE TEXT SEARCH DICTIONARY _DICTNAME_
 	(TEMPLATE = snowball, Language = _LANGNAME_ _STOPWORDS_);
 

src/backend/snowball/snowball_func.sql.in

@@ -1,4 +1,21 @@
--- src/backend/snowball/snowball_func.sql.in$
+/*
+ * Create underlying C functions for Snowball stemmers
+ *
+ * Copyright (c) 2007-2015, PostgreSQL Global Development Group
+ *
+ * src/backend/snowball/snowball_func.sql.in
+ *
+ * This file is combined with multiple instances of snowball.sql.in to
+ * build snowball_create.sql, which is executed during initdb.
+ *
+ * Note: this file is read in single-user -j mode, which means that the
+ * command terminator is semicolon-newline-newline; whenever the backend
+ * sees that, it stops and executes what it's got.  If you write a lot of
+ * statements without empty lines between, they'll all get quoted to you
+ * in any error message about one of them, so don't do that.  Also, you
+ * cannot write a semicolon immediately followed by an empty line in a
+ * string literal (including a function body!) or a multiline comment.
+ */
 
 SET search_path = pg_catalog;
 

src/backend/tcop/postgres.c

@@ -157,18 +157,8 @@ static CachedPlanSource *unnamed_stmt_psrc = NULL;
 
 /* assorted command-line switches */
 static const char *userDoption = NULL;	/* -D switch */
-
 static bool EchoQuery = false;	/* -E switch */
-
-/*
- * people who want to use EOF should #define DONTUSENEWLINE in
- * tcop/tcopdebug.h
- */
-#ifndef TCOP_DONTUSENEWLINE
-static int	UseNewLine = 1;		/* Use newlines query delimiters (the default) */
-#else
-static int	UseNewLine = 0;		/* Use EOF as query delimiters */
-#endif   /* TCOP_DONTUSENEWLINE */
+static bool UseSemiNewlineNewline = false;		/* -j switch */
 
 /* whether or not, and why, we were canceled by conflict with recovery */
 static bool RecoveryConflictPending = false;
@@ -219,8 +209,6 @@ static int
 InteractiveBackend(StringInfo inBuf)
 {
 	int			c;				/* character read from getc() */
-	bool		end = false;	/* end-of-input flag */
-	bool		backslashSeen = false;	/* have we seen a \ ? */
 
 	/*
 	 * display a prompt and obtain input from the user
@@ -230,55 +218,56 @@ InteractiveBackend(StringInfo inBuf)
 
 	resetStringInfo(inBuf);
 
-	if (UseNewLine)
-	{
 	/*
-		 * if we are using \n as a delimiter, then read characters until the
-		 * \n.
+	 * Read characters until EOF or the appropriate delimiter is seen.
 	 */
 	while ((c = interactive_getc()) != EOF)
 	{
 		if (c == '\n')
 		{
-				if (backslashSeen)
+			if (UseSemiNewlineNewline)
 			{
-					/* discard backslash from inBuf */
-					inBuf->data[--inBuf->len] = '\0';
-					backslashSeen = false;
-					continue;
-				}
-				else
+				/*
+				 * In -j mode, semicolon followed by two newlines ends the
+				 * command; otherwise treat newline as regular character.
+				 */
+				if (inBuf->len > 1 &&
+					inBuf->data[inBuf->len - 1] == '\n' &&
+					inBuf->data[inBuf->len - 2] == ';')
 				{
-					/* keep the newline character */
-					appendStringInfoChar(inBuf, '\n');
+					/* might as well drop the second newline */
 					break;
 				}
 			}
-			else if (c == '\\')
-				backslashSeen = true;
-			else
-				backslashSeen = false;
-
-			appendStringInfoChar(inBuf, (char) c);
-		}
-
-		if (c == EOF)
-			end = true;
-	}
 			else
 			{
 				/*
-		 * otherwise read characters until EOF.
+				 * In plain mode, newline ends the command unless preceded by
+				 * backslash.
 				 */
-		while ((c = interactive_getc()) != EOF)
-			appendStringInfoChar(inBuf, (char) c);
-
-		/* No input before EOF signal means time to quit. */
-		if (inBuf->len == 0)
-			end = true;
+				if (inBuf->len > 0 &&
+					inBuf->data[inBuf->len - 1] == '\\')
+				{
+					/* discard backslash from inBuf */
+					inBuf->data[--inBuf->len] = '\0';
+					/* discard newline too */
+					continue;
+				}
+				else
+				{
+					/* keep the newline character, but end the command */
+					appendStringInfoChar(inBuf, '\n');
+					break;
+				}
+			}
 		}
 
-	if (end)
+		/* Not newline, or newline treated as regular character */
+		appendStringInfoChar(inBuf, (char) c);
+	}
+
+	/* No input before EOF signal means time to quit. */
+	if (c == EOF && inBuf->len == 0)
 		return EOF;
 
 	/*
@@ -3391,7 +3380,7 @@ process_postgres_switches(int argc, char *argv[], GucContext ctx,
 
 			case 'j':
 				if (secure)
-					UseNewLine = 0;
+					UseSemiNewlineNewline = true;
 				break;
 
 			case 'k':

src/include/tcop/tcopdebug.h

@@ -1,44 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * tcopdebug.h
- *	  #defines governing debugging behaviour in the traffic cop
- *
- *
- * Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/include/tcop/tcopdebug.h
- *
- *-------------------------------------------------------------------------
- */
-#ifndef TCOPDEBUG_H
-#define TCOPDEBUG_H
-
-/* ----------------------------------------------------------------
- *		debugging defines.
- *
- *		If you want certain debugging behaviour, then #define
- *		the variable to 1, else #undef it. -cim 10/26/89
- * ----------------------------------------------------------------
- */
-
-/* ----------------
- *		TCOP_SHOWSTATS controls whether or not buffer and
- *		access method statistics are shown for each query.  -cim 2/9/89
- * ----------------
- */
-#undef TCOP_SHOWSTATS
-
-/* ----------------
- *		TCOP_DONTUSENEWLINE controls the default setting of
- *		the UseNewLine variable in postgres.c
- * ----------------
- */
-#undef TCOP_DONTUSENEWLINE
-
-/* ----------------------------------------------------------------
- *		#defines controlled by above definitions
- * ----------------------------------------------------------------
- */
-
-#endif   /* TCOPDEBUG_H */