From: ktm@rice.edu <ktm@rice.edu>
To: Albe Laurenz <laurenz.albe@wien.gv.at>
Cc: Jeff *EXTERN* <jeff@jefftrout.com>
Cc: Jeff Janes <jeff.janes@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Replaying 48 WAL files takes 80 minutes
Date: Tue, 30 Oct 2012 08:41:24 -0500
Message-ID: <20121030134124.GL2872@aart.rice.edu> (raw)
In-Reply-To: <D960CB61B694CF459DCFB4B0128514C2089A6216@exadv11.host.magwien.gv.at>
References: <D960CB61B694CF459DCFB4B0128514C2089A5F80@exadv11.host.magwien.gv.at>
<CAMkU=1z_GYXM+cn_TuVLx78_FAeFVw0+H0KDjFVf+rooiWYebw@mail.gmail.com>
<9677809C-5FBC-4E8A-9A7E-5F5A3FDAFA0B@jefftrout.com>
<D960CB61B694CF459DCFB4B0128514C2089A60EC@exadv11.host.magwien.gv.at>
<20121030130533.GJ2872@aart.rice.edu>
<D960CB61B694CF459DCFB4B0128514C2089A6216@exadv11.host.magwien.gv.at>
On Tue, Oct 30, 2012 at 02:16:57PM +0100, Albe Laurenz wrote:
> ktm@rice.edu wrote:
> >>> If you do not have good random io performance log replay is nearly
> >>> unbearable.
> >>>
> >>> also, what io scheduler are you using? if it is cfq change that to
> >>> deadline or noop.
> >>> that can make a huge difference.
> >>
> >> We use the noop scheduler.
> >> As I said, an identical system performed well in load tests.
>
> > The load tests probably had the "important" data already cached.
> Processing
> > a WAL file would involve bringing all the data back into memory using
> a
> > random I/O pattern.
>
> The database is way too big (1 TB) to fit into cache.
>
> What are "all the data" that have to be brought back?
> Surely only the database blocks that are modified by the WAL,
> right?
>
> Yours,
> Laurenz Albe
>
Right, it would only read the blocks that are modified.
Regards,
Ken
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: ktm@rice.edu, laurenz.albe@wien.gv.at, jeff@jefftrout.com, jeff.janes@gmail.com
Subject: Re: Replaying 48 WAL files takes 80 minutes
In-Reply-To: <20121030134124.GL2872@aart.rice.edu>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox