postgresql/doc/TODO.detail/prepare

From owner-pgsql-hackers@hub.org Wed Nov 18 14:40:49 1998
Received: from hub.org (majordom@hub.org [209.47.148.200])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id OAA29743
	for <maillist@candle.pha.pa.us>; Wed, 18 Nov 1998 14:40:36 -0500 (EST)
Received: from localhost (majordom@localhost)
	by hub.org (8.9.1/8.9.1) with SMTP id OAA03716;
	Wed, 18 Nov 1998 14:37:04 -0500 (EST)
	(envelope-from owner-pgsql-hackers@hub.org)
Received: by hub.org (TLB v0.10a (1.23 tibbs 1997/01/09 00:29:32)); Wed, 18 Nov 1998 14:34:39 +0000 (EST)
Received: (from majordom@localhost)
	by hub.org (8.9.1/8.9.1) id OAA03395
	for pgsql-hackers-outgoing; Wed, 18 Nov 1998 14:34:37 -0500 (EST)
	(envelope-from owner-pgsql-hackers@postgreSQL.org)
Received: from orion.SAPserv.Hamburg.dsh.de (Tpolaris2.sapham.debis.de [53.2.131.8])
	by hub.org (8.9.1/8.9.1) with SMTP id OAA03381
	for <pgsql-hackers@hub.org>; Wed, 18 Nov 1998 14:34:31 -0500 (EST)
	(envelope-from wieck@sapserv.debis.de)
Received: by orion.SAPserv.Hamburg.dsh.de 
	for pgsql-hackers@hub.org 
	id m0zgDnj-000EBTC; Wed, 18 Nov 98 21:02 MET
Message-Id: <m0zgDnj-000EBTC@orion.SAPserv.Hamburg.dsh.de>
From: jwieck@debis.com (Jan Wieck)
Subject: Re: [HACKERS] PREPARE
To: meskes@usa.net (Michael Meskes)
Date: Wed, 18 Nov 1998 21:02:06 +0100 (MET)
Cc: pgsql-hackers@hub.org
Reply-To: jwieck@debis.com (Jan Wieck)
In-Reply-To: <19981118084843.B869@usa.net> from "Michael Meskes" at Nov 18, 98 08:48:43 am
X-Mailer: ELM [version 2.4 PL25]
Content-Type: text
Sender: owner-pgsql-hackers@postgreSQL.org
Precedence: bulk
Status: RO

Michael Meskes wrote:

>
> On Wed, Nov 18, 1998 at 03:23:30AM +0000, Thomas G. Lockhart wrote:
> > > I didn't get this one completly. What input do you mean?
> >
> > Just the original string/query to be prepared...
>
> I see. But wouldn't it be more useful to preprocess the query and store the
> resulting nodes instead? We don't want to parse the statement everytime a
> variable binding comes in.

    Right.  A real improvement would only be to have the prepared
    execution plan in the backend and just giving  the  parameter
    values.

    I can think of the following construct:

        PREPARE optimizable-statement;

    That one will run parser/rewrite/planner, create a new memory
    context with a unique identifier and  saves  the  querytree's
    and  plan's  in  it.   Parameter values are identified by the
    usual $n notation. The command returns the identifier.

        EXECUTE QUERY identifier [value [, ...]];

    then get's back the prepared plan and querytree  by  the  id,
    creates  an  executor  context  with  the given values in the
    parameter array and calls ExecutorRun() for them.

    The PREPARE needs to analyze the resulting parsetrees to  get
    the  datatypes  (and maybe atttypmod's) of the parameters, so
    EXECUTE QUERY can convert the values into Datum's  using  the
    types  input  functions.  And  the  EXECUTE has to be handled
    special in tcop (it's something between a regular  query  and
    an utility statement). But it's not too hard to implement.

    Finally a

        FORGET QUERY identifier;

    (don't  remember  how  the  others  named it) will remove the
    prepared plan etc. simply by destroying  the  memory  context
    and dropping the identifier from the id->mcontext+prepareinfo
    mapping.

    This all  restricts  the  usage  of  PREPARE  to  optimizable
    statements.  Is  it  required  to  be able to prepare utility
    statements (like CREATE TABLE or so) too?


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#======================================== jwieck@debis.com (Jan Wieck) #




From pgsql-hackers-owner+M67@postgresql.org Tue Oct 31 19:18:16 2000
Received: from mail.postgresql.org ([216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id TAA08916
	for <pgman@candle.pha.pa.us>; Tue, 31 Oct 2000 19:18:15 -0500 (EST)
Received: from mail.postgresql.org ([216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA10IOl60635;
	Tue, 31 Oct 2000 19:18:24 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M67@postgresql.org)
Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA10H8l60400
	for <pgsql-hackers@postgresql.org>; Tue, 31 Oct 2000 19:17:08 -0500 (EST)
	(envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
	by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id BAA32036;
	Wed, 1 Nov 2000 01:16:42 +0100
Date: Wed, 1 Nov 2000 01:16:42 +0100 (CET)
From: Karel Zak <zakkr@zf.jcu.cz>
To: Alfred Perlstein <bright@wintelcom.net>
cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Query cache import?
In-Reply-To: <20001031151144.F22110@fw.wintelcom.net>
Message-ID: <Pine.LNX.3.96.1001101005110.31713B-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


On Tue, 31 Oct 2000, Alfred Perlstein wrote:

> I never saw much traffic regarding Karel's work on making stored
> proceedures:
>
> http://people.freebsd.org/~alfred/karel-pgsql.txt
> 
> What happened with this?  It looked pretty interesting. :(

 It's probably a little about me :-) ... well,

  My query cache is in usable state and it's efficient for all 
things those motivate me to work on this.

 some basic features:

	- share parsed plans between backends in shared memory
	- store plans to private backend hash table
	- use parameters for stored queries
	- better design for SPI 
			- memory usage for saved plans
			- save plans "by key"

 
 The current query cache code depend on 7.1 memory management. After
official 7.1 release I prepare patch with query cache+SPI (if not
hit me over head, please ..)

 All what will doing next time not depend on me, *it's on code developers*.

 For example Jan has interesting idea about caching all plans which
processing backend. But it's far future and IMHO we must go by small
steps to Oracle's funeral :-) 

 If I need the query cache in the my work (typical for some web+pgsql) or 
will some public interest I will continue on this, if not I freeze it. 
(Exists more interesting work like http://mape.jcu.cz ... sorry of 
advertising :-)

					Karel






From pgsql-hackers-owner+M312@postgresql.org Mon Nov  6 03:27:32 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA28404
	for <pgman@candle.pha.pa.us>; Mon, 6 Nov 2000 03:27:32 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA68Pos51966;
	Mon, 6 Nov 2000 03:25:50 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M312@postgresql.org)
Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA68Fes50414
	for <pgsql-hackers@postgresql.org>; Mon, 6 Nov 2000 03:15:40 -0500 (EST)
	(envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
	by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id JAA20862;
	Mon, 6 Nov 2000 09:15:04 +0100
Date: Mon, 6 Nov 2000 09:15:04 +0100 (CET)
From: Karel Zak <zakkr@zf.jcu.cz>
To: Christof Petig <christof.petig@wtal.de>
cc: Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>,
        The Hermit Hacker <scrappy@hub.org>, pgsql-hackers@postgresql.org
Subject: Re: AW: [HACKERS] Re: [GENERAL] Query caching
In-Reply-To: <3A02DDFF.E8CBFCF3@wtal.de>
Message-ID: <Pine.LNX.3.96.1001106090801.20612C-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


On Fri, 3 Nov 2000, Christof Petig wrote:

> Karel Zak wrote:
> 
> > On Thu, 2 Nov 2000, Zeugswetter Andreas SB wrote:
> >
> > >
> > > > Well I can re-write and resubmit this patch. Add it as a
> > > > compile time option
> > > > is not bad idea. Second possibility is distribute it as patch
> > > > in the contrib
> > > > tree. And if it until not good tested not dirty with this main tree...
> > > >
> > > >  Ok, I next week prepare it...
> > >
> > > One thing that worries me though is, that it extends the sql language,
> > > and there has been no discussion about the chosen syntax.
> > >
> > > Imho the standard embedded SQL syntax (prepare ...) could be a
> > > starting point.
> >
> >  Yes, you are right... my PREPARE/EXECUTE is not too much ready to SQL92,
> > I some old letter I speculate about "SAVE/EXECUTE PLAN" instead
> > PREPARE/EXECUTE. But don't forget, it will *experimental* patch... we can
> > change it in future ..etc.
> >
> >                                         Karel
> 
> [Sorry, I didn't look into your patch, yet.]

 Please, read my old query cache and PREPARE/EXECUTE description...

> What about parameters? Normally you can prepare a statement and execute it

 We have in PG parameters, see SPI, but now it's used inside backend only
and not exist statement that allows to use this feature in be<->fe.

> using different parameters. AFAIK postgres' frontend-backend protocol is not
> designed to take parameters for statements (e.g. like result presents
> results). A very long road to go.
> By the way, I'm somewhat interested in getting this feature in. Perhaps it
> should be part of a protocol redesign (e.g. binary parameters/results).
> Handling endianness is one aspect, floats are harder (but float->ascii->float
> sometimes fails as well).

        PREPARE <name> AS <query>
                [ USING type, ... typeN ]
                [ NOSHARE | SHARE | GLOBAL ]

        EXECUTE <name>
                [ INTO [ TEMPORARY | TEMP ] [ TABLE ] new_table ]
                [ USING val, ... valN ]
                       [ NOSHARE | SHARE | GLOBAL ]

        DEALLOCATE PREPARE
                [ <name> [ NOSHARE | SHARE | GLOBAL ]]
                [ ALL | ALL INTERNAL ]


An example:


PREPARE chris_query AS SELECT * FROM pg_class WHERE relname = $1 USING text;

EXECUTE chris_query USING 'pg_shadow';


	Or mean you something other?
					Karel






From pgsql-hackers-owner+M444@postgresql.org Thu Nov  9 03:32:10 2000
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by candle.pha.pa.us (8.9.0/8.9.0) with ESMTP id DAA09953
	for <pgman@candle.pha.pa.us>; Thu, 9 Nov 2000 03:32:09 -0500 (EST)
Received: from mail.postgresql.org (webmail.postgresql.org [216.126.85.28])
	by mail.postgresql.org (8.11.1/8.11.1) with SMTP id eA98RSs11426;
	Thu, 9 Nov 2000 03:27:28 -0500 (EST)
	(envelope-from pgsql-hackers-owner+M444@postgresql.org)
Received: from ara.zf.jcu.cz (ara.zf.jcu.cz [160.217.161.4])
	by mail.postgresql.org (8.11.1/8.11.1) with ESMTP id eA98OPs11045;
	Thu, 9 Nov 2000 03:24:25 -0500 (EST)
	(envelope-from zakkr@zf.jcu.cz)
Received: from localhost (zakkr@localhost)
	by ara.zf.jcu.cz (8.9.3/8.9.3/Debian 8.9.3-21) with SMTP id JAA08951;
	Thu, 9 Nov 2000 09:23:41 +0100
Date: Thu, 9 Nov 2000 09:23:41 +0100 (CET)
From: Karel Zak <zakkr@zf.jcu.cz>
To: Christof Petig <christof.petig@wtal.de>
cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
        Michael Meskes <meskes@postgresql.org>,
        Zeugswetter Andreas SB <ZeugswetterA@wien.spardat.at>,
        The Hermit Hacker <scrappy@hub.org>
Subject: Re: AW: [HACKERS] Re: [GENERAL] Query caching
In-Reply-To: <3A096BCE.F9887955@wtal.de>
Message-ID: <Pine.LNX.3.96.1001109090739.8052B-100000@ara.zf.jcu.cz>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org
Status: OR


On Wed, 8 Nov 2000, Christof Petig wrote:

> Karel Zak wrote:
> 
> > > What about parameters? Normally you can prepare a statement and execute it
> >
> >  We have in PG parameters, see SPI, but now it's used inside backend only
> > and not exist statement that allows to use this feature in be<->fe.
> 
> Sad. Since ecpg would certainly benefit from this.
> 
> > > using different parameters. AFAIK postgres' frontend-backend protocol is not
> > > designed to take parameters for statements (e.g. like result presents
> > > results). A very long road to go.
> > > By the way, I'm somewhat interested in getting this feature in. Perhaps it
> > > should be part of a protocol redesign (e.g. binary parameters/results).
> > > Handling endianness is one aspect, floats are harder (but float->ascii->float
> > > sometimes fails as well).
> >
> >         PREPARE <name> AS <query>
> >                 [ USING type, ... typeN ]
> >                 [ NOSHARE | SHARE | GLOBAL ]
> >
> >         EXECUTE <name>
> >                 [ INTO [ TEMPORARY | TEMP ] [ TABLE ] new_table ]
> >                 [ USING val, ... valN ]
> >                        [ NOSHARE | SHARE | GLOBAL ]
> >
> >         DEALLOCATE PREPARE
> >                 [ <name> [ NOSHARE | SHARE | GLOBAL ]]
> >                 [ ALL | ALL INTERNAL ]
> >
> > An example:
> >
> > PREPARE chris_query AS SELECT * FROM pg_class WHERE relname = $1 USING text;
> 
> I would prefer '?' as a parameter name, since this is in the embedded sql standard
> (do you have a copy of the 94 draft? I can mail mine to you?)

 This not depend on query cache. The '$n' is PostgreSQL query parametr
keyword and is defined in standard parser. The PREPARE statement not parsing
query it's job for standard parser.

> Also the standard says a whole lot about guessing the parameter's type.
> 
> Also I vote for  ?::type or type(?) or sql's cast(...) (don't know it's syntax)
> instead of abusing the using keyword.

The postgresql executor expect types of parametrs in separate input (array).
I not sure how much expensive/executable is survey it from query.

> > EXECUTE chris_query USING 'pg_shadow';
> 
> Great idea of yours to implement this! Since I was thinking about implementing a
> more decent schema for ecpg but had no mind to touch the backend and be-fe
> protocol (yet).
> It would be desirable to do an 'execute immediate using', since using input
> parameters would take a lot of code away from ecpg.

By the way, PREPARE/EXECUTE is face only. More interesting in this period is
query-cache-kernel. SQL92 is really a little unlike my PREPARE/EXECUTE.

						Karel