mirror/postgresql
2
0
Fork 0
mirror of https://git.postgresql.org/git/postgresql.git synced 2024-12-15 08:20:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
e37d474f91
postgresql/contrib/pg_xlogdump/pg_xlogdump.c

998 lines
25 KiB
C
Raw Normal View History

Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
/*-------------------------------------------------------------------------
*
* pg_xlogdump.c - decode and display WAL
*
Update copyright for 2015 Backpatch certain files through 9.0
2015-01-07 00:43:47 +08:00
* Copyright (c) 2013-2015, PostgreSQL Global Development Group
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
*
* IDENTIFICATION
* contrib/pg_xlogdump/pg_xlogdump.c
*-------------------------------------------------------------------------
*/
#define FRONTEND 1
#include "postgres.h"
#include <dirent.h>
#include <unistd.h>
#include "access/xlogreader.h"
Move the backup-block logic from XLogInsert to a new file, xloginsert.c. xlog.c is huge, this makes it a little bit smaller, which is nice. Functions related to putting together the WAL record are in xloginsert.c, and the lower level stuff for managing WAL buffers and such are in xlog.c. Also move the definition of XLogRecord to a separate header file. This causes churn in the #includes of all the files that write WAL records, and redo routines, but it avoids pulling in xlog.h into most places. Reviewed by Michael Paquier, Alvaro Herrera, Andres Freund and Amit Kapila.
2014-11-06 19:52:08 +08:00
#include "access/xlogrecord.h"
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
#include "access/xlog_internal.h"
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
#include "access/transam.h"
#include "common/fe_memutils.h"
#include "getopt_long.h"
#include "rmgrdesc.h"
static const char *progname;
typedef struct XLogDumpPrivate
{
TimeLineID timeline;
char *inpath;
XLogRecPtr startptr;
XLogRecPtr endptr;
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
bool endptr_reached;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
} XLogDumpPrivate;
typedef struct XLogDumpConfig
{
/* display options */
bool bkp_details;
int stop_after_records;
int already_displayed_records;
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
bool follow;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
bool stats;
bool stats_per_record;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
/* filter options */
int filter_by_rmgr;
TransactionId filter_by_xid;
Fix copy-and-pasteo Harmless, but it's certainly better like this. Noticed by Andres Freund
2013-02-23 04:03:22 +08:00
bool filter_by_xid_enabled;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
} XLogDumpConfig;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
typedef struct Stats
{
uint64 count;
uint64 rec_len;
uint64 fpi_len;
} Stats;
#define MAX_XLINFO_TYPES 16
typedef struct XLogDumpStats
{
uint64 count;
Stats rmgr_stats[RM_NEXT_ID];
Stats record_stats[RM_NEXT_ID][MAX_XLINFO_TYPES];
} XLogDumpStats;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
static void
fatal_error(const char *fmt,...)
__attribute__((format(PG_PRINTF_ATTRIBUTE, 1, 2)));
/*
* Big red button to push when things go horribly wrong.
*/
static void
fatal_error(const char *fmt,...)
{
va_list args;
fflush(stdout);
fprintf(stderr, "%s: FATAL: ", progname);
va_start(args, fmt);
vfprintf(stderr, fmt, args);
va_end(args);
fputc('\n', stderr);
exit(EXIT_FAILURE);
}
static void
print_rmgr_list(void)
{
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-30 04:58:43 +08:00
int i;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
Fix off-by-one in pg_xlogdump -r option. Because of the bug, -r would not accept the rmgr with the highest ID.
2013-06-04 23:51:43 +08:00
for (i = 0; i <= RM_MAX_ID; i++)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
{
printf("%s\n", RmgrDescTable[i].rm_name);
}
}
/*
* Check whether directory exists and whether we can open it. Keep errno set so
* that the caller can report errors somewhat more accurately.
*/
static bool
verify_directory(const char *directory)
{
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-30 04:58:43 +08:00
DIR *dir = opendir(directory);
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
if (dir == NULL)
return false;
closedir(dir);
return true;
}
/*
* Split a pathname as dirname(1) and basename(1) would.
*
* XXX this probably doesn't do very well on Windows. We probably need to
* apply canonicalize_path(), at the very least.
*/
static void
split_path(const char *path, char **dir, char **fname)
{
char *sep;
/* split filepath into directory & filename */
sep = strrchr(path, '/');
/* directory path */
if (sep != NULL)
{
*dir = pg_strdup(path);
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-30 04:58:43 +08:00
(*dir)[(sep - path) + 1] = '\0'; /* no strndup */
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
*fname = pg_strdup(sep + 1);
}
/* local directory */
else
{
*dir = NULL;
*fname = pg_strdup(path);
}
}
/*
* Try to find the file in several places:
* if directory == NULL:
* fname
* XLOGDIR / fname
* $PGDATA / XLOGDIR / fname
* else
* directory / fname
* directory / XLOGDIR / fname
*
* return a read only fd
*/
static int
fuzzy_open_file(const char *directory, const char *fname)
{
int fd = -1;
char fpath[MAXPGPATH];
if (directory == NULL)
{
const char *datadir;
/* fname */
fd = open(fname, O_RDONLY | PG_BINARY, 0);
if (fd < 0 && errno != ENOENT)
return -1;
Fix off-by-one in pg_xlogdump's fuzzy_open_file(). In the unlikely case of stdin (fd 0) being closed, the off-by-one would lead to pg_xlogdump failing to open files. Spotted by Coverity. Backpatch to 9.3 where pg_xlogdump was introduced.
2015-01-04 22:35:46 +08:00
else if (fd >= 0)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
return fd;
/* XLOGDIR / fname */
snprintf(fpath, MAXPGPATH, "%s/%s",
XLOGDIR, fname);
fd = open(fpath, O_RDONLY | PG_BINARY, 0);
if (fd < 0 && errno != ENOENT)
return -1;
Fix off-by-one in pg_xlogdump's fuzzy_open_file(). In the unlikely case of stdin (fd 0) being closed, the off-by-one would lead to pg_xlogdump failing to open files. Spotted by Coverity. Backpatch to 9.3 where pg_xlogdump was introduced.
2015-01-04 22:35:46 +08:00
else if (fd >= 0)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
return fd;
datadir = getenv("PGDATA");
/* $PGDATA / XLOGDIR / fname */
if (datadir != NULL)
{
snprintf(fpath, MAXPGPATH, "%s/%s/%s",
datadir, XLOGDIR, fname);
fd = open(fpath, O_RDONLY | PG_BINARY, 0);
if (fd < 0 && errno != ENOENT)
return -1;
Fix off-by-one in pg_xlogdump's fuzzy_open_file(). In the unlikely case of stdin (fd 0) being closed, the off-by-one would lead to pg_xlogdump failing to open files. Spotted by Coverity. Backpatch to 9.3 where pg_xlogdump was introduced.
2015-01-04 22:35:46 +08:00
else if (fd >= 0)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
return fd;
}
}
else
{
/* directory / fname */
snprintf(fpath, MAXPGPATH, "%s/%s",
directory, fname);
fd = open(fpath, O_RDONLY | PG_BINARY, 0);
if (fd < 0 && errno != ENOENT)
return -1;
Fix off-by-one in pg_xlogdump's fuzzy_open_file(). In the unlikely case of stdin (fd 0) being closed, the off-by-one would lead to pg_xlogdump failing to open files. Spotted by Coverity. Backpatch to 9.3 where pg_xlogdump was introduced.
2015-01-04 22:35:46 +08:00
else if (fd >= 0)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
return fd;
/* directory / XLOGDIR / fname */
snprintf(fpath, MAXPGPATH, "%s/%s/%s",
directory, XLOGDIR, fname);
fd = open(fpath, O_RDONLY | PG_BINARY, 0);
if (fd < 0 && errno != ENOENT)
return -1;
Fix off-by-one in pg_xlogdump's fuzzy_open_file(). In the unlikely case of stdin (fd 0) being closed, the off-by-one would lead to pg_xlogdump failing to open files. Spotted by Coverity. Backpatch to 9.3 where pg_xlogdump was introduced.
2015-01-04 22:35:46 +08:00
else if (fd >= 0)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
return fd;
}
return -1;
}
/*
* Read count bytes from a segment file in the specified directory, for the
* given timeline, containing the specified record pointer; store the data in
* the passed buffer.
*/
static void
XLogDumpXLogRead(const char *directory, TimeLineID timeline_id,
XLogRecPtr startptr, char *buf, Size count)
{
char *p;
XLogRecPtr recptr;
Size nbytes;
static int sendFile = -1;
static XLogSegNo sendSegNo = 0;
static uint32 sendOff = 0;
p = buf;
recptr = startptr;
nbytes = count;
while (nbytes > 0)
{
uint32 startoff;
int segbytes;
int readbytes;
startoff = recptr % XLogSegSize;
if (sendFile < 0 || !XLByteInSeg(recptr, sendSegNo))
{
char fname[MAXFNAMELEN];
/* Switch to another logfile segment */
if (sendFile >= 0)
close(sendFile);
XLByteToSeg(recptr, sendSegNo);
XLogFileName(fname, timeline_id, sendSegNo);
sendFile = fuzzy_open_file(directory, fname);
if (sendFile < 0)
fatal_error("could not find file \"%s\": %s",
fname, strerror(errno));
sendOff = 0;
}
/* Need to seek in the file? */
if (sendOff != startoff)
{
if (lseek(sendFile, (off_t) startoff, SEEK_SET) < 0)
{
int err = errno;
char fname[MAXPGPATH];
XLogFileName(fname, timeline_id, sendSegNo);
fatal_error("could not seek in log segment %s to offset %u: %s",
fname, startoff, strerror(err));
}
sendOff = startoff;
}
/* How many bytes are within this segment? */
if (nbytes > (XLogSegSize - startoff))
segbytes = XLogSegSize - startoff;
else
segbytes = nbytes;
readbytes = read(sendFile, p, segbytes);
if (readbytes <= 0)
{
int err = errno;
char fname[MAXPGPATH];
XLogFileName(fname, timeline_id, sendSegNo);
fatal_error("could not read from log segment %s, offset %d, length %d: %s",
fname, sendOff, segbytes, strerror(err));
}
/* Update state for read */
recptr += readbytes;
sendOff += readbytes;
nbytes -= readbytes;
p += readbytes;
}
}
/*
* XLogReader read_page callback
*/
static int
XLogDumpReadPage(XLogReaderState *state, XLogRecPtr targetPagePtr, int reqLen,
XLogRecPtr targetPtr, char *readBuff, TimeLineID *curFileTLI)
{
XLogDumpPrivate *private = state->private_data;
int count = XLOG_BLCKSZ;
if (private->endptr != InvalidXLogRecPtr)
{
if (targetPagePtr + XLOG_BLCKSZ <= private->endptr)
count = XLOG_BLCKSZ;
else if (targetPagePtr + reqLen <= private->endptr)
count = private->endptr - targetPagePtr;
else
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
{
private->endptr_reached = true;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
return -1;
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
}
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
}
XLogDumpXLogRead(private->inpath, private->timeline, targetPagePtr,
readBuff, count);
return count;
}
/*
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
* Store per-rmgr and per-record statistics for a given record.
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
*/
static void
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
XLogDumpCountRecord(XLogDumpConfig *config, XLogDumpStats *stats,
XLogReaderState *record)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
{
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
RmgrId rmid;
uint8 recid;
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
uint32 rec_len;
uint32 fpi_len;
Fix pg_xlogdump's calculation of full-page image data. The old formula was completely bogus with the new WAL record format.
2014-12-05 17:39:40 +08:00
int block_id;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
stats->count++;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
rmid = XLogRecGetRmid(record);
rec_len = XLogRecGetDataLen(record) + SizeOfXLogRecord;
Fix pg_xlogdump's calculation of full-page image data. The old formula was completely bogus with the new WAL record format.
2014-12-05 17:39:40 +08:00
/*
* Calculate the amount of FPI data in the record. Each backup block
* takes up BLCKSZ bytes, minus the "hole" length.
*
* XXX: We peek into xlogreader's private decoded backup blocks for the
* hole_length. It doesn't seem worth it to add an accessor macro for
* this.
*/
fpi_len = 0;
for (block_id = 0; block_id <= record->max_block_id; block_id++)
{
if (XLogRecHasBlockImage(record, block_id))
fpi_len += BLCKSZ - record->blocks[block_id].hole_length;
}
/* Update per-rmgr statistics */
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
stats->rmgr_stats[rmid].count++;
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
stats->rmgr_stats[rmid].rec_len += rec_len;
stats->rmgr_stats[rmid].fpi_len += fpi_len;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
/*
* Update per-record statistics, where the record is identified by a
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
* combination of the RmgrId and the four bits of the xl_info field that
* are the rmgr's domain (resulting in sixteen possible entries per
* RmgrId).
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
*/
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
recid = XLogRecGetInfo(record) >> 4;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
stats->record_stats[rmid][recid].count++;
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
stats->record_stats[rmid][recid].rec_len += rec_len;
stats->record_stats[rmid][recid].fpi_len += fpi_len;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
}
/*
* Print a record to stdout
*/
static void
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record)
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
{
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
const char *id;
const RmgrDescData *desc = &RmgrDescTable[XLogRecGetRmid(record)];
RelFileNode rnode;
ForkNumber forknum;
BlockNumber blk;
int block_id;
uint8 info = XLogRecGetInfo(record);
XLogRecPtr xl_prev = XLogRecGetPrev(record);
id = desc->rm_identify(info);
Improve code around the recently added rm_identify rmgr callback. There are four weaknesses in728f152e07f998d2cb4fe5f24ec8da2c3bda98f2: * append_init() in heapdesc.c was ugly and required that rm_identify return values are only valid till the next call. Instead just add a couple more switch() cases for the INIT_PAGE cases. Now the returned value will always be valid. * a couple rm_identify() callbacks missed masking xl_info with ~XLR_INFO_MASK. * pg_xlogdump didn't map a NULL rm_identify to UNKNOWN or a similar string. * append_init() was called when id=NULL - which should never actually happen. But it's better to be careful.
2014-09-22 22:48:14 +08:00
if (id == NULL)
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
id = psprintf("UNKNOWN (%x)", info & ~XLR_INFO_MASK);
Improve code around the recently added rm_identify rmgr callback. There are four weaknesses in728f152e07f998d2cb4fe5f24ec8da2c3bda98f2: * append_init() in heapdesc.c was ugly and required that rm_identify return values are only valid till the next call. Instead just add a couple more switch() cases for the INIT_PAGE cases. Now the returned value will always be valid. * a couple rm_identify() callbacks missed masking xl_info with ~XLR_INFO_MASK. * pg_xlogdump didn't map a NULL rm_identify to UNKNOWN or a similar string. * append_init() was called when id=NULL - which should never actually happen. But it's better to be careful.
2014-09-22 22:48:14 +08:00
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
printf("rmgr: %-11s len (rec/tot): %6u/%6u, tx: %10u, lsn: %X/%08X, prev %X/%08X, ",
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
desc->rm_name,
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
XLogRecGetDataLen(record), XLogRecGetTotalLen(record),
XLogRecGetXid(record),
(uint32) (record->ReadRecPtr >> 32), (uint32) record->ReadRecPtr,
(uint32) (xl_prev >> 32), (uint32) xl_prev);
printf("desc: %s ", id);
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
/* the desc routine will printf the description directly to stdout */
Change the signature of rm_desc so that it's passed a XLogRecord. Just feels more natural, and is more consistent with rm_redo.
2014-06-14 15:46:48 +08:00
desc->rm_desc(NULL, record);
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
if (!config->bkp_details)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
{
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
/* print block references (short format) */
for (block_id = 0; block_id <= record->max_block_id; block_id++)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
{
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
if (!XLogRecHasBlockRef(record, block_id))
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
continue;
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
XLogRecGetBlockTag(record, block_id, &rnode, &forknum, &blk);
if (forknum != MAIN_FORKNUM)
printf(", blkref #%u: rel %u/%u/%u fork %s blk %u",
block_id,
rnode.spcNode, rnode.dbNode, rnode.relNode,
forkNames[forknum],
blk);
else
printf(", blkref #%u: rel %u/%u/%u blk %u",
block_id,
rnode.spcNode, rnode.dbNode, rnode.relNode,
blk);
if (XLogRecHasBlockImage(record, block_id))
printf(" FPW");
}
putchar('\n');
}
else
{
/* print block references (detailed format) */
putchar('\n');
for (block_id = 0; block_id <= record->max_block_id; block_id++)
{
if (!XLogRecHasBlockRef(record, block_id))
continue;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
XLogRecGetBlockTag(record, block_id, &rnode, &forknum, &blk);
printf("\tblkref #%u: rel %u/%u/%u fork %s blk %u",
block_id,
rnode.spcNode, rnode.dbNode, rnode.relNode,
forkNames[forknum],
blk);
if (XLogRecHasBlockImage(record, block_id))
{
printf(" (FPW); hole: offset: %u, length: %u\n",
record->blocks[block_id].hole_offset,
record->blocks[block_id].hole_length);
}
putchar('\n');
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
}
}
}
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
/*
* Display a single row of record counts and sizes for an rmgr or record.
*/
static void
XLogDumpStatsRow(const char *name,
uint64 n, double n_pct,
uint64 rec_len, double rec_len_pct,
uint64 fpi_len, double fpi_len_pct,
uint64 total_len, double total_len_pct)
{
printf("%-27s "
"%20" INT64_MODIFIER "u (%6.02f) "
"%20" INT64_MODIFIER "u (%6.02f) "
"%20" INT64_MODIFIER "u (%6.02f) "
"%20" INT64_MODIFIER "u (%6.02f)\n",
name, n, n_pct, rec_len, rec_len_pct, fpi_len, fpi_len_pct,
total_len, total_len_pct);
}
/*
* Display summary statistics about the records seen so far.
*/
static void
XLogDumpDisplayStats(XLogDumpConfig *config, XLogDumpStats *stats)
{
int ri, rj;
uint64 total_count = 0;
uint64 total_rec_len = 0;
uint64 total_fpi_len = 0;
uint64 total_len = 0;
/* ---
* Make a first pass to calculate column totals:
* count(*),
* sum(xl_len+SizeOfXLogRecord),
* sum(xl_tot_len-xl_len-SizeOfXLogRecord), and
* sum(xl_tot_len).
* These are used to calculate percentages for each record type.
* ---
*/
for (ri = 0; ri < RM_NEXT_ID; ri++)
{
total_count += stats->rmgr_stats[ri].count;
total_rec_len += stats->rmgr_stats[ri].rec_len;
total_fpi_len += stats->rmgr_stats[ri].fpi_len;
}
total_len = total_rec_len+total_fpi_len;
/*
* 27 is strlen("Transaction/COMMIT_PREPARED"),
* 20 is strlen(2^64), 8 is strlen("(100.00%)")
*/
printf("%-27s %20s %8s %20s %8s %20s %8s %20s %8s\n"
"%-27s %20s %8s %20s %8s %20s %8s %20s %8s\n",
"Type", "N", "(%)", "Record size", "(%)", "FPI size", "(%)", "Combined size", "(%)",
"----", "-", "---", "-----------", "---", "--------", "---", "-------------", "---");
for (ri = 0; ri < RM_NEXT_ID; ri++)
{
uint64 count, rec_len, fpi_len, tot_len;
const RmgrDescData *desc = &RmgrDescTable[ri];
if (!config->stats_per_record)
{
count = stats->rmgr_stats[ri].count;
rec_len = stats->rmgr_stats[ri].rec_len;
fpi_len = stats->rmgr_stats[ri].fpi_len;
tot_len = rec_len + fpi_len;
XLogDumpStatsRow(desc->rm_name,
count, 100 * (double) count / total_count,
rec_len, 100 * (double) rec_len / total_rec_len,
fpi_len, 100 * (double) fpi_len / total_fpi_len,
tot_len, 100 * (double) tot_len / total_len);
}
else
{
for (rj = 0; rj < MAX_XLINFO_TYPES; rj++)
{
const char *id;
count = stats->record_stats[ri][rj].count;
rec_len = stats->record_stats[ri][rj].rec_len;
fpi_len = stats->record_stats[ri][rj].fpi_len;
tot_len = rec_len + fpi_len;
/* Skip undefined combinations and ones that didn't occur */
if (count == 0)
continue;
/* the upper four bits in xl_info are the rmgr's */
id = desc->rm_identify(rj << 4);
if (id == NULL)
id = psprintf("UNKNOWN (%x)", rj << 4);
XLogDumpStatsRow(psprintf("%s/%s", desc->rm_name, id),
count, 100 * (double) count / total_count,
rec_len, 100 * (double) rec_len / total_rec_len,
fpi_len, 100 * (double) fpi_len / total_fpi_len,
tot_len, 100 * (double) tot_len / total_len);
}
}
}
printf("%-27s %20s %8s %20s %8s %20s %8s %20s\n",
"", "--------", "", "--------", "", "--------", "", "--------");
/*
* The percentages in earlier rows were calculated against the
* column total, but the ones that follow are against the row total.
* Note that these are displayed with a % symbol to differentiate
* them from the earlier ones, and are thus up to 9 characters long.
*/
printf("%-27s "
"%20" INT64_MODIFIER "u %-9s"
"%20" INT64_MODIFIER "u %-9s"
"%20" INT64_MODIFIER "u %-9s"
"%20" INT64_MODIFIER "u %-6s\n",
"Total", stats->count, "",
total_rec_len, psprintf("[%.02f%%]", 100 * (double)total_rec_len / total_len),
total_fpi_len, psprintf("[%.02f%%]", 100 * (double)total_fpi_len / total_len),
total_len, "[100%]");
}
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
static void
usage(void)
{
printf("%s decodes and displays PostgreSQL transaction logs for debugging.\n\n",
progname);
printf("Usage:\n");
pg_xlogdump: Improve --help output
2013-05-12 09:55:37 +08:00
printf(" %s [OPTION]... [STARTSEG [ENDSEG]] \n", progname);
printf("\nOptions:\n");
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
printf(" -b, --bkp-details output detailed information about backup blocks\n");
printf(" -e, --end=RECPTR stop reading at log position RECPTR\n");
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
printf(" -f, --follow keep retrying after reaching end of WAL\n");
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
printf(" -n, --limit=N number of records to display\n");
printf(" -p, --path=PATH directory in which to find log segment files\n");
printf(" (default: ./pg_xlog)\n");
printf(" -r, --rmgr=RMGR only show records generated by resource manager RMGR\n");
printf(" use --rmgr=list to list valid resource manager names\n");
Fix typo in "pg_xlogdump --help" and error message. Fujii Masao and me.
2013-02-28 03:27:01 +08:00
printf(" -s, --start=RECPTR start reading at log position RECPTR\n");
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
printf(" -t, --timeline=TLI timeline from which to read log records\n");
printf(" (default: 1 or the value used in STARTSEG)\n");
pg_xlogdump: Improve --help output
2013-05-12 09:55:37 +08:00
printf(" -V, --version output version information, then exit\n");
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
printf(" -x, --xid=XID only show records with TransactionId XID\n");
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
printf(" -z, --stats[=record] show statistics instead of records\n");
printf(" (optionally, show per-record statistics)\n");
pg_xlogdump: Improve --help output
2013-05-12 09:55:37 +08:00
printf(" -?, --help show this help, then exit\n");
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
}
int
main(int argc, char **argv)
{
uint32 xlogid;
uint32 xrecoff;
XLogReaderState *xlogreader_state;
XLogDumpPrivate private;
XLogDumpConfig config;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
XLogDumpStats stats;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
XLogRecord *record;
XLogRecPtr first_record;
char *errormsg;
static struct option long_options[] = {
{"bkp-details", no_argument, NULL, 'b'},
{"end", required_argument, NULL, 'e'},
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
{"follow", no_argument, NULL, 'f'},
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
{"help", no_argument, NULL, '?'},
{"limit", required_argument, NULL, 'n'},
{"path", required_argument, NULL, 'p'},
{"rmgr", required_argument, NULL, 'r'},
{"start", required_argument, NULL, 's'},
{"timeline", required_argument, NULL, 't'},
{"xid", required_argument, NULL, 'x'},
{"version", no_argument, NULL, 'V'},
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
{"stats", optional_argument, NULL, 'z'},
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
{NULL, 0, NULL, 0}
};
int option;
int optindex = 0;
progname = get_progname(argv[0]);
memset(&private, 0, sizeof(XLogDumpPrivate));
memset(&config, 0, sizeof(XLogDumpConfig));
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
memset(&stats, 0, sizeof(XLogDumpStats));
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
private.timeline = 1;
private.startptr = InvalidXLogRecPtr;
private.endptr = InvalidXLogRecPtr;
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
private.endptr_reached = false;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
config.bkp_details = false;
config.stop_after_records = -1;
config.already_displayed_records = 0;
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
config.follow = false;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
config.filter_by_rmgr = -1;
config.filter_by_xid = InvalidTransactionId;
config.filter_by_xid_enabled = false;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
config.stats = false;
config.stats_per_record = false;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
if (argc <= 1)
{
fprintf(stderr, "%s: no arguments specified\n", progname);
goto bad_argument;
}
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
while ((option = getopt_long(argc, argv, "be:?fn:p:r:s:t:Vx:z",
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
long_options, &optindex)) != -1)
{
switch (option)
{
case 'b':
config.bkp_details = true;
break;
case 'e':
if (sscanf(optarg, "%X/%X", &xlogid, &xrecoff) != 2)
{
fprintf(stderr, "%s: could not parse end log position \"%s\"\n",
progname, optarg);
goto bad_argument;
}
private.endptr = (uint64) xlogid << 32 | xrecoff;
break;
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
case 'f':
config.follow = true;
break;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
case '?':
usage();
exit(EXIT_SUCCESS);
break;
case 'n':
if (sscanf(optarg, "%d", &config.stop_after_records) != 1)
{
fprintf(stderr, "%s: could not parse limit \"%s\"\n",
progname, optarg);
goto bad_argument;
}
break;
case 'p':
private.inpath = pg_strdup(optarg);
break;
case 'r':
{
int i;
if (pg_strcasecmp(optarg, "list") == 0)
{
print_rmgr_list();
exit(EXIT_SUCCESS);
}
Fix off-by-one in pg_xlogdump -r option. Because of the bug, -r would not accept the rmgr with the highest ID.
2013-06-04 23:51:43 +08:00
for (i = 0; i <= RM_MAX_ID; i++)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
{
if (pg_strcasecmp(optarg, RmgrDescTable[i].rm_name) == 0)
{
config.filter_by_rmgr = i;
break;
}
}
if (config.filter_by_rmgr == -1)
{
fprintf(stderr, "%s: resource manager \"%s\" does not exist\n",
progname, optarg);
goto bad_argument;
}
}
break;
case 's':
if (sscanf(optarg, "%X/%X", &xlogid, &xrecoff) != 2)
{
Fix typo in "pg_xlogdump --help" and error message. Fujii Masao and me.
2013-02-28 03:27:01 +08:00
fprintf(stderr, "%s: could not parse start log position \"%s\"\n",
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
progname, optarg);
goto bad_argument;
}
else
private.startptr = (uint64) xlogid << 32 | xrecoff;
break;
case 't':
if (sscanf(optarg, "%d", &private.timeline) != 1)
{
fprintf(stderr, "%s: could not parse timeline \"%s\"\n",
progname, optarg);
goto bad_argument;
}
break;
case 'V':
puts("pg_xlogdump (PostgreSQL) " PG_VERSION);
exit(EXIT_SUCCESS);
break;
case 'x':
if (sscanf(optarg, "%u", &config.filter_by_xid) != 1)
{
fprintf(stderr, "%s: could not parse \"%s\" as a valid xid\n",
progname, optarg);
goto bad_argument;
}
config.filter_by_xid_enabled = true;
break;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
case 'z':
config.stats = true;
config.stats_per_record = false;
if (optarg)
{
if (strcmp(optarg, "record") == 0)
config.stats_per_record = true;
else if (strcmp(optarg, "rmgr") != 0)
{
fprintf(stderr, "%s: unrecognised argument to --stats: %s\n",
progname, optarg);
goto bad_argument;
}
}
break;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
default:
goto bad_argument;
}
}
if ((optind + 2) < argc)
{
fprintf(stderr,
"%s: too many command-line arguments (first is \"%s\")\n",
progname, argv[optind + 2]);
goto bad_argument;
}
if (private.inpath != NULL)
{
/* validate path points to directory */
if (!verify_directory(private.inpath))
{
fprintf(stderr,
"%s: path \"%s\" cannot be opened: %s",
progname, private.inpath, strerror(errno));
goto bad_argument;
}
}
/* parse files as start/end boundaries, extract path if not specified */
if (optind < argc)
{
char *directory = NULL;
char *fname = NULL;
int fd;
XLogSegNo segno;
split_path(argv[optind], &directory, &fname);
if (private.inpath == NULL && directory != NULL)
{
private.inpath = directory;
if (!verify_directory(private.inpath))
fatal_error("cannot open directory \"%s\": %s",
private.inpath, strerror(errno));
}
fd = fuzzy_open_file(private.inpath, fname);
if (fd < 0)
fatal_error("could not open file \"%s\"", fname);
close(fd);
/* parse position from file */
XLogFromFileName(fname, &private.timeline, &segno);
if (XLogRecPtrIsInvalid(private.startptr))
XLogSegNoOffsetToRecPtr(segno, 0, private.startptr);
else if (!XLByteInSeg(private.startptr, segno))
{
fprintf(stderr,
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-30 04:58:43 +08:00
"%s: start log position %X/%X is not inside file \"%s\"\n",
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
progname,
(uint32) (private.startptr >> 32),
(uint32) private.startptr,
fname);
goto bad_argument;
}
/* no second file specified, set end position */
if (!(optind + 1 < argc) && XLogRecPtrIsInvalid(private.endptr))
XLogSegNoOffsetToRecPtr(segno + 1, 0, private.endptr);
/* parse ENDSEG if passed */
if (optind + 1 < argc)
{
XLogSegNo endsegno;
/* ignore directory, already have that */
split_path(argv[optind + 1], &directory, &fname);
fd = fuzzy_open_file(private.inpath, fname);
if (fd < 0)
fatal_error("could not open file \"%s\"", fname);
close(fd);
/* parse position from file */
XLogFromFileName(fname, &private.timeline, &endsegno);
if (endsegno < segno)
fatal_error("ENDSEG %s is before STARTSEG %s",
argv[optind + 1], argv[optind]);
if (XLogRecPtrIsInvalid(private.endptr))
XLogSegNoOffsetToRecPtr(endsegno + 1, 0, private.endptr);
/* set segno to endsegno for check of --end */
segno = endsegno;
}
if (!XLByteInSeg(private.endptr, segno) &&
private.endptr != (segno + 1) * XLogSegSize)
{
fprintf(stderr,
"%s: end log position %X/%X is not inside file \"%s\"\n",
progname,
(uint32) (private.endptr >> 32),
(uint32) private.endptr,
argv[argc - 1]);
goto bad_argument;
}
}
/* we don't know what to print */
if (XLogRecPtrIsInvalid(private.startptr))
{
fprintf(stderr, "%s: no start log position given in range mode.\n", progname);
goto bad_argument;
}
/* done with argument parsing, do the actual work */
/* we have everything we need, start reading */
xlogreader_state = XLogReaderAllocate(XLogDumpReadPage, &private);
if (!xlogreader_state)
fatal_error("out of memory");
/* first find a valid recptr to start from */
first_record = XLogFindNextRecord(xlogreader_state, private.startptr);
if (first_record == InvalidXLogRecPtr)
fatal_error("could not find a valid record after %X/%X",
(uint32) (private.startptr >> 32),
(uint32) private.startptr);
/*
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-30 04:58:43 +08:00
* Display a message that we're skipping data if `from` wasn't a pointer
* to the start of a record and also wasn't a pointer to the beginning of
* a segment (e.g. we were used in file mode).
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
*/
if (first_record != private.startptr && (private.startptr % XLogSegSize) != 0)
printf("first record is after %X/%X, at %X/%X, skipping over %u bytes\n",
(uint32) (private.startptr >> 32), (uint32) private.startptr,
(uint32) (first_record >> 32), (uint32) first_record,
(uint32) (first_record - private.startptr));
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
for (;;)
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
{
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
/* try to read the next record */
record = XLogReadRecord(xlogreader_state, first_record, &errormsg);
if (!record)
{
if (!config.follow || private.endptr_reached)
break;
else
{
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-07 00:12:18 +08:00
pg_usleep(1000000L); /* 1 second */
Add -f/--follow option to pg_xlogdump. This is useful for seeing what WAL records are inserted in real-time, by pointing pg_xlogdump to a live server.
2014-03-26 19:48:20 +08:00
continue;
}
}
/* after reading the first record, continue at next one */
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
first_record = InvalidXLogRecPtr;
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
/* apply all specified filters */
if (config.filter_by_rmgr != -1 &&
config.filter_by_rmgr != record->xl_rmid)
continue;
if (config.filter_by_xid_enabled &&
config.filter_by_xid != record->xl_xid)
continue;
/* process the record */
if (config.stats == true)
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
XLogDumpCountRecord(&config, &stats, xlogreader_state);
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
else
Revamp the WAL record format. Each WAL record now carries information about the modified relation and block(s) in a standardized format. That makes it easier to write tools that need that information, like pg_rewind, prefetching the blocks to speed up recovery, etc. There's a whole new API for building WAL records, replacing the XLogRecData chains used previously. The new API consists of XLogRegister* functions, which are called for each buffer and chunk of data that is added to the record. The new API also gives more control over when a full-page image is written, by passing flags to the XLogRegisterBuffer function. This also simplifies the XLogReadBufferForRedo() calls. The function can dig the relation and block number from the WAL record, so they no longer need to be passed as arguments. For the convenience of redo routines, XLogReader now disects each WAL record after reading it, copying the main data part and the per-block data into MAXALIGNed buffers. The data chunks are not aligned within the WAL record, but the redo routines can assume that the pointers returned by XLogRecGet* functions are. Redo routines are now passed the XLogReaderState, which contains the record in the already-disected format, instead of the plain XLogRecord. The new record format also makes the fixed size XLogRecord header smaller, by removing the xl_len field. The length of the "main data" portion is now stored at the end of the WAL record, and there's a separate header after XLogRecord for it. The alignment padding at the end of XLogRecord is also removed. This compansates for the fact that the new format would otherwise be more bulky than the old format. Reviewed by Andres Freund, Amit Kapila, Michael Paquier, Alvaro Herrera, Fujii Masao.
2014-11-20 23:56:26 +08:00
XLogDumpDisplayRecord(&config, xlogreader_state);
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
/* check whether we printed enough */
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
config.already_displayed_records++;
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
if (config.stop_after_records > 0 &&
config.already_displayed_records >= config.stop_after_records)
break;
}
Add the capability to display summary statistics to pg_xlogdump. The new --stats/--stats=record options to pg_xlogdump display per rmgr/per record statistics about the parsed WAL. This is useful to understand what the WAL primarily consists of, to allow targeted optimizations on application, configuration, and core code level. It is likely that we will want to fine tune the statistics further, but the feature already is quite helpful. Author: Abhijit Menon-Sen, slightly editorialized by me Reviewed-By: Andres Freund, Dilip Kumar and Furuya Osamu Discussion: 20140604104716.GA3989@toroid.org
2014-09-19 22:33:16 +08:00
if (config.stats == true)
XLogDumpDisplayStats(&config, &stats);
Add pg_xlogdump contrib program This program relies on rm_desc backend routines and the xlogreader infrastructure to emit human-readable rendering of WAL records. Author: Andres Freund, with many reworks by Álvaro Reviewed (in a much earlier version) by Peter Eisentraut
2013-02-23 03:46:24 +08:00
if (errormsg)
fatal_error("error in WAL record at %X/%X: %s\n",
(uint32) (xlogreader_state->ReadRecPtr >> 32),
(uint32) xlogreader_state->ReadRecPtr,
errormsg);
XLogReaderFree(xlogreader_state);
return EXIT_SUCCESS;
bad_argument:
fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
return EXIT_FAILURE;
}
Reference in New Issue Copy Permalink