pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedWAL size many times data size?
2+ messages / 2 participants
[nested] [flat]
* WAL size many times data size?
@ 2016-11-10 14:17 Craig James <cjames@emolecules.com>
2016-11-10 14:57 ` Re: WAL size many times data size? Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 2+ messages in thread
From: Craig James @ 2016-11-10 14:17 UTC (permalink / raw)
To: pgsql-admin
Our streaming replication and WAL archiving keep having trouble because of
the sheer size of the WAL files. The number of bytes in the WAL files seems
to be a large multiplier, like 5x or 10x, the amount of data that we load.
It's hard to know exactly, because in addition to the size of the actual
data files, there are indexes and auxiliary tables of crunched data. But
these shouldn't even double the total data.
The other day we loaded about 5-8 GB of data, and the WAL directory ended
up with 75GB in it. Streaming replication to our standby server broke
because the network couldn't keep up, so it had to fall back to the
secondary stream of WAL files.
Is this expected?
Thanks,
Craig
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: WAL size many times data size?
2016-11-10 14:17 WAL size many times data size? Craig James <cjames@emolecules.com>
@ 2016-11-10 14:57 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 2+ messages in thread
From: Tom Lane @ 2016-11-10 14:57 UTC (permalink / raw)
To: Craig James <cjames@emolecules.com>; +Cc: pgsql-admin
Craig James <cjames@emolecules.com> writes:
> Our streaming replication and WAL archiving keep having trouble because of
> the sheer size of the WAL files. The number of bytes in the WAL files seems
> to be a large multiplier, like 5x or 10x, the amount of data that we load.
My first instinct would be to check how often checkpoints are happening,
and increase the checkpoint control parameters so that they're at least
5 to 15 minutes apart. A short checkpoint interval leads directly to
WAL bloat because it means more full-page images get inserted into WAL.
(The first change of any given page after a checkpoint requires an FPI.)
regards, tom lane
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2016-11-10 14:57 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2016-11-10 14:17 WAL size many times data size? Craig James <cjames@emolecules.com>
2016-11-10 14:57 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox