Commit Graph

20661 Commits

Author SHA1 Message Date
Bruce Momjian
7f0b690334 Improve comments for AdjustIntervalForTypmod.
Blank line adjustments.
2005-07-12 16:05:12 +00:00
Bruce Momjian
62abb039df Change 5e0 to 5.0, for consistency. 2005-07-12 15:17:44 +00:00
Tom Lane
220e6bfc53 Fix plperl to do recursion safely, and fix a problem with array results.
Add suitable regression tests.  Andrew Dunstan
2005-07-12 01:16:22 +00:00
Tom Lane
a1a64bb7c3 Further tweaking of Win32-specific random code. Marko Kreen 2005-07-11 19:06:46 +00:00
Tom Lane
e82f7e9e7e I forgot to update win32 code when doing a renaming in random.c.
Marko Kreen
2005-07-11 17:00:03 +00:00
Tom Lane
067a5cdb35 Avoid bzero/bxopy in favor of more standard library routines.
Marko Kreen
2005-07-11 15:40:38 +00:00
Tom Lane
e997758cb6 More pgcrypto fixes: avoid bogus alignment assumptions in sha2,
be more wary about having a value for BYTE_ORDER, clean up randomly-
chosen ways of including Postgres core headers.
Marko Kreen and Tom Lane
2005-07-11 15:07:59 +00:00
Tom Lane
3c52d17fa6 Add support for AES cipher with older OpenSSL libraries.
Marko Kreen
2005-07-11 14:38:05 +00:00
Tom Lane
8826fe859f Missed adding extra argument to array_recv in a couple of places
(harmless, actually, but let's be tidy).
2005-07-10 21:36:21 +00:00
Tom Lane
d78397d301 Change typreceive function API so that receive functions get the same
optional arguments as text input functions, ie, typioparam OID and
atttypmod.  Make all the datatypes that use typmod enforce it the same
way in typreceive as they do in typinput.  This fixes a problem with
failure to enforce length restrictions during COPY FROM BINARY.
2005-07-10 21:14:00 +00:00
Bruce Momjian
2e330699fa > One more failure:
>
> I think this is because we don't have -lz in SHLIB_LINK.
> Following patch fixes it.

Marko Kreen
2005-07-10 18:32:55 +00:00
Bruce Momjian
21634e513f Add extra argument for new pg_regexec API. 2005-07-10 18:31:59 +00:00
Tom Lane
aacfeba61b Suppress compile warning. 2005-07-10 17:22:54 +00:00
Bruce Momjian
e8a3e6b8a0 Rename xmalloc to pg_malloc for consistency with psql usage.
Add missing plperl include.
2005-07-10 16:13:13 +00:00
Bruce Momjian
f86c63ab6a Reverse pg_malloc patch because psql/print.c is used in scripts files
that don't have pg_malloc.
2005-07-10 15:53:42 +00:00
Bruce Momjian
8c9393c640 Use failure-safe pg_malloc consistently in psql/print.c. 2005-07-10 15:48:14 +00:00
Bruce Momjian
6e1004f0fd Remove #include <openssl/bn.h> as compile fix.
Marko Kreen
2005-07-10 15:37:03 +00:00
Bruce Momjian
b135508c98 Following up a previous thought I had, yesterday I realised how to
return arays nicely without having to make the plperl programmer aware
of anything. The attached patch allows plperl to return an arrayref
where the function returns an array type. It silently calls a perl
function to stringify the array before passing it to the pg array
parser. Non-array returns are handled as before (i.e. passed through
this process) so it is backwards compatible. I will presently submit
regression tests and docs.

example:

andrew=# create or replace function blah() returns text[][] language
plperl as $$ return [['a"b','c,d'],['e\\f','g']]; $$;
CREATE FUNCTION
andrew=# select blah();
            blah
-----------------------------
 {{"a\"b","c,d"},{"e\\f",g}}


This would complete half of the TODO item:

  . Pass arrays natively instead of as text between plperl and postgres

(The other half is translating pg array arguments to perl arrays - that
will have to wait for 8.1).

Some of this patch is adapted from a previously submitted patch from
Sergej Sergeev. Both he and Abhijit Menon-Sen have looked it over
briefly and tentatively said it looks ok.

Andrew Dunstan
2005-07-10 15:32:47 +00:00
Bruce Momjian
6d92f2106f The attached patch implements spi_query() and spi_fetchrow() functions
for PL/Perl, to avoid loading the entire result set into memory as the
existing spi_exec_query() function does.

Here's how one might use the new functions:

    $x = spi_query("select ...");
    while (defined ($y = spi_fetchrow($x))) {
        ...
        return_next(...);
    }

The changes do not affect the spi_exec_query() interface in any way.

Abhijit Menon-Sen
2005-07-10 15:19:43 +00:00
Bruce Momjian
d1cffe2f77 Add --encoding to pg_dump.
The Problem:  Occassionally a DBA needs to dump a database to a new
encoding.   In instances where the current encoding, (or lack of an
encoding, like SQL_ASCII) is poorly supported on the target  database
server, it can be useful to dump into a particular  encoding.  But,
currently the only way to set the encoding of  a pg_dump file is to
change  client_encoding in postgresql.conf and restart postmaster.
This is more  than a little awkward for production systems.

Magnus Hagander
2005-07-10 15:08:52 +00:00
Bruce Momjian
0793108036 This patch implements putting language handlers for the optional PLs
into pg_catalog rather than public, and supports dumping languages whose
handlers are found there. This will make it easier to drop the public
schema if desired.

Unlike the previous patch, the comments have been updated and I have
reformatted some code to meet Alvarro's request to stick to 80 cols. (I
actually aghree with this - it makes printing the code much nicer).

I think I did the right thing w.r.t versions earlier than 7.3, but I
have no real way of checking, so that should be checked by someone with
more/older knowledge than me ;-)

Andrew Dunstan
2005-07-10 14:26:30 +00:00
Bruce Momjian
d51df91897 As Kris Jurka found out, pgcrypto does not work with
OpenSSL 0.9.6x.  The DES functions use the older 'des_'
API, but the newer 3DES functions use the 0.9.7x-only
'DES_' API.

I think I just used /usr/include/openssl/des.h for reference
when implementing them, and had upgraded OpenSSL in the
meantime.

Following patch converts DES also to newer API and provides
compatibility functions for OpenSSL < 0.9.7.

I chose this route because:

- openssl.c uses few DES functions.
- compatibility for old 'des_' API is going away at some point
  of time from OpenSSL.
- as seen from macros, new API is saner
- Thus pgcrypto supports any OpenSSL version from 0.9.5 to 1.0

Tested with OpenSSL 0.9.6c and 0.9.7e.

Marko Kreen
2005-07-10 13:54:34 +00:00
Bruce Momjian
e94dd6ab91 Add missing pgcrypto files from previous commit. 2005-07-10 13:46:29 +00:00
Bruce Momjian
42e7b0f02f Update.
< 	  computations should adjust based on the time zone rules, e.g.
< 	  adding 24 hours to a timestamp would yield a different result from
< 	  adding one day.
<
> 	  computations should adjust based on the time zone rules.
2005-07-10 05:06:42 +00:00
Bruce Momjian
37f11c3081 This patch addresses the problem mentioned in the "process crash
when a plpython function returns unicode" thread:

http://archives.postgresql.org/pgsql-bugs/2005-06/msg00105.php

In several places PL/Python was calling PyObject_Str() and then
PyString_AsString() without checking if the former had returned
NULL to indicate an error.  PyString_AsString() doesn't expect a
NULL argument, so passing one causes a segmentation fault.  This
patch adds checks for NULL and raises errors via PLy_elog(), which
prints details of the underlying Python exception.  The patch also
adds regression tests for these checks.  All tests pass on my
Solaris 9 box running HEAD and Python 2.4.1.

In one place the patch doesn't call PLy_elog() because that could
cause infinite recursion; see the comment I added.  I'm not sure
how to test that particular case or whether it's even possible to
get an error there: the value that the code should check is the
Python exception type, so I wonder if a NULL value "shouldn't
happen."  This patch converts NULL to "Unknown Exception" but I
wonder if an Assert() would be appropriate.

The patch is against HEAD but the same changes should be applied
to earlier versions because they have the same problem.  The patch
might not apply cleanly against earlier versions -- will the committer
take care of little differences or should I submit different versions
of the patch?

Michael Fuhr
2005-07-10 04:56:55 +00:00
Bruce Momjian
75a64eeb4b I made the patch that implements regexp_replace again.
The specification of this function is as follows.

regexp_replace(source text, pattern text, replacement text, [flags
text])
returns text

Replace string that matches to regular expression in source text to
replacement text.

 - pattern is regular expression pattern.
 - replacement is replace string that can use '\1'-'\9', and '\&'.
    '\1'-'\9': back reference to the n'th subexpression.
    '\&'     : entire matched string.
 - flags can use the following values:
    g: global (replace all)
    i: ignore case
    When the flags is not specified, case sensitive, replace the first
    instance only.

Atsushi Ogawa
2005-07-10 04:54:33 +00:00
Bruce Momjian
73e2431817 Major pgcrypto changes:
of password-based encryption from RFC2440 (OpenPGP).

The goal of this code is to be more featureful encryption solution
than current encrypt(), which only functionality is running cipher
over data.

Compared to encrypt(), pgp_encrypt() does following:

* It uses the equvialent of random Inital Vector to get cipher
  into random state before it processes user data
* Stores SHA-1 of the data into result so any modification
  will be detected.
* Remembers if data was text or binary - thus it can decrypt
  to/from text data.  This was a major nuisance for encrypt().
* Stores info about used algorithms with result, so user needs
  not remember them - more user friendly!
* Uses String2Key algorithms (similar to crypt()) with random salt
  to generate full-length binary key to be used for encrypting.
* Uses standard format for data - you can feed it to GnuPG, if needed.

Optional features (off by default):

* Can use separate session key - user data will be encrypted
  with totally random key, which will be encrypted with S2K
  generated key and attached to result.
* Data compression with zlib.
* Can convert between CRLF<->LF line-endings - to get fully
  RFC2440-compliant behaviour.  This is off by default as
  pgcrypto does not know the line-endings of user data.

Interface is simple:


    pgp_encrypt(data text, key text) returns bytea
    pgp_decrypt(data text, key text) returns text
    pgp_encrypt_bytea(data bytea, key text) returns bytea
    pgp_decrypt_bytea(data bytea, key text) returns bytea

To change parameters (cipher, compression, mdc):

    pgp_encrypt(data text, key text, parms text) returns bytea
    pgp_decrypt(data text, key text, parms text) returns text
    pgp_encrypt_bytea(data bytea, key text, parms text) returns bytea
    pgp_decrypt_bytea(data bytea, key text, parms text) returns bytea

Parameter names I lifted from gpg:

   pgp_encrypt('message', 'key', 'compress-algo=1,cipher-algo=aes256')

For text data, pgp_encrypt simply encrypts the PostgreSQL internal data.

This maps to RFC2440 data type 't' - 'extenally specified encoding'.
But this may cause problems if data is dumped and reloaded into database
which as different internal encoding.  My next goal is to implement data
type 'u' - which means data is in UTF-8 encoding by converting internal
encoding to UTF-8 and back.  And there wont be any compatibility
problems with current code, I think its ok to submit this without UTF-8
encoding by converting internal encoding to UTF-8 and back.  And there
wont be any compatibility problems with current code, I think its ok to
submit this without UTF-8 support.


Here is v4 of PGP encrypt.  This depends on previously sent
Fortuna-patch, as it uses the px_add_entropy function.

- New function: pgp_key_id() for finding key id's.
- Add SHA1 of user data and key into RNG pools.  We need to get
  randomness from somewhere, and it is in user best interests
  to contribute.
- Regenerate pgp-armor test for SQL_ASCII database.
- Cleanup the key handling so that the pubkey support is less
  hackish.

Marko Kreen
2005-07-10 03:57:55 +00:00
Bruce Momjian
4fcf8b11ff - Add Fortuna PRNG to pgcrypto.
- Move openssl random provider to openssl.c and builtin provider
  to internal.c
- Make px_random_bytes use Fortuna, instead of giving error.
- Retarget random.c to aquiring system randomness, for initial seeding
  of Fortuna.  There is ATM 2 functions for Windows,
  reader from /dev/urandom and the regular time()/getpid() silliness.

Marko Kreen
2005-07-10 03:55:28 +00:00
Bruce Momjian
248eeb82f0 This patch adds implementation of SHA2 to pgcrypto.
New hashes: SHA256, SHA384, SHA512.

Marko Kreen
2005-07-10 03:52:56 +00:00
Bruce Momjian
73a7c322c3 Add psql \pset numericsep to allow output numbers like 100,000.0 or
100.000,0.

Eugen Nedelcu
2005-07-10 03:46:13 +00:00
Tom Lane
be3aa30da3 Fix inadequate error checking: you can't assume that fcinfo->resultinfo
is a ReturnSetInfo unless you've tested it with IsA.
2005-07-09 01:53:22 +00:00
Tom Lane
40d091b818 Fix config file lexer to not barf if postgresql.conf ends with a comment
that has no terminating newline.  Per report from maps.on at gmx.net.
2005-07-08 18:41:40 +00:00
Tom Lane
0ed5f90d73 Try connecting to both postgres and template1 databases to do the initial
inspection of shared catalogs.  This allows pg_dumpall to continue to
work with pre-8.1 servers that likely won't have a database named postgres.
Also, suppress output of SYSID options for users and groups, since server
no longer does anything with these except emit a rude message.
There is much more to be done to update pg_dumpall for the roles feature,
but this at least makes it usable again.  Per gripe from Chris K-L.
2005-07-08 16:51:30 +00:00
Tom Lane
70f7da3e6e Make libpq_gettext save and restore errno in a Windows-compatible way.
Also, back-patch fix into back branches.
2005-07-08 15:24:41 +00:00
PostgreSQL Daemon
85f97937d2 testing activitymail 2005-07-08 15:13:09 +00:00
Neil Conway
86897f54a8 This patch updates the DDL for contrib/pgcrypto to create all
functions as STRICT, and all functions except gen_salt() as IMMUTABLE.
gen_salt() is VOLATILE.

Although the functions are now STRICT, I left their PG_ARGISNULL()
checks in place as a protective measure for users who install the
new code but use old (non-STRICT) catalog entries (e.g., restored
from a dump).  Per recent discussion in pgsql-hackers.

Patch from Michael Fuhr.
2005-07-08 04:27:49 +00:00
Neil Conway
40ffa1a14c Remove some dead code for handling XLOG_DBASE_CREATE_OLD and
XLOG_DBASE_DROP_OLD WAL records -- these records are no longer created in
current sources. Adjust numbering of XLOG_DBASE_CREATE and XLOG_DBASE_DROP
and bump the catversion. Patch from Gavin Sherry, adjusted by Neil Conway.
2005-07-08 04:12:27 +00:00
Tom Lane
d7207cfc6b Even though I'd like to see full_page_writes go away before 8.1,
a minimum requirement is that it not completely break the system
meanwhile.  Put the test in the right place.
2005-07-08 04:07:26 +00:00
Bruce Momjian
a4ffa38234 Done:
> * -Prevent dropping user that still owns objects, or auto-drop the objects
2005-07-08 02:44:37 +00:00
Tom Lane
59d1b3d99e Track dependencies on shared objects (which is to say, roles; we already
have adequate mechanisms for tracking the contents of databases and
tablespaces).  This solves the longstanding problem that you can drop a
user who still owns objects and/or has access permissions.
Alvaro Herrera, with some kibitzing from Tom Lane.
2005-07-07 20:40:02 +00:00
Bruce Momjian
442b59dd8b Update:
< 	   writer.
> 	   writer.  It might cause problems for applying WAL on recovery
> 	   into a partially-written page, but later the full page will be
> 	   replaced from WAL.
2005-07-07 16:02:06 +00:00
Bruce Momjian
f0a2a91918 Update:
>
> 	o  -Add ability to turn off full page writes
> 	o  When off, write CRC to WAL and check file system blocks
> 	   on recovery
> 	o  Write full pages during file system write and not when
> 	   the page is modified in the buffer cache
>
> 	   This allows most full page writes to happen in the background
> 	   writer.
2005-07-07 15:18:26 +00:00
Bruce Momjian
1c883366ef Fix plperl expected output.
Andrew Dunstan
2005-07-07 04:41:01 +00:00
Bruce Momjian
294de2dc01 pg_column_size() cleanup for messages and code cleanup.
Mark Kirkwood
2005-07-07 04:36:08 +00:00
Bruce Momjian
eefdbba062 Currently, nonfatal warnings are not trapped (as they should be) by
plperl - the attached small patch remedies that omission, and adds a
small regression test for error and warning output - the new regression
input and expected output are in separate attached files.

Andrew Dunstan
2005-07-06 22:44:49 +00:00
Bruce Momjian
77838f7380 Currently, nonfatal warnings are not trapped (as they should be) by
plperl - the attached small patch remedies that omission.

Andrew Dunstan
2005-07-06 22:33:39 +00:00
Bruce Momjian
970bb03c3c Complete zic patch backout by removing NO_PGPORT workaround. 2005-07-06 21:40:09 +00:00
Bruce Momjian
261ffd03f7 Reverse out because the lack of using pgport in timezone/ is causing
problems:

---------------------------------------------------------------------------

Support cross compilation by compiling "zic" with a native compiler.
This relies on the output of zic being platform independent, but that is
currently the case.
2005-07-06 21:04:14 +00:00
Bruce Momjian
a923602855 Add pg_column_size() to return storage size of a column, including
possible compression.

Mark Kirkwood
2005-07-06 19:02:54 +00:00
Tom Lane
b9cb132648 Sync dlopen error handling for the *BSDs ... seems to me I've done this
before, but they were out of sync again.  Per Kris Jurka.
2005-07-06 16:55:58 +00:00