From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Fujii Masao <masao.fujii@gmail.com>
Cc: Martijn van Oosterhout <kleptog@svana.org>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: WAL logging problem in 9.4.3?
Date: Fri, 10 Jul 2015 10:59:32 +0200
Message-ID: <20150710085932.GJ10242@alap3.anarazel.de> (raw)
In-Reply-To: <29916.1436483171@sss.pgh.pa.us>
References: <CAHGQGwGzxAyKLUm+z2n43-ee6u976jLmQAz=pPBVzSueD3CAig@mail.gmail.com>
<20150703164931.GI3291@awork2.anarazel.de>
<20150703170229.GJ3291@awork2.anarazel.de>
<20150703172605.GM3291@awork2.anarazel.de>
<CAHGQGwGGM2OBh4WOTd8wWqxKzRy+WdJX+-fxxPCWNz4eeJ1aQQ@mail.gmail.com>
<27532.1436195680@sss.pgh.pa.us>
<20150706152123.GK8902@alap3.anarazel.de>
<28415.1436197794@sss.pgh.pa.us>
<20150709182315.GG10242@alap3.anarazel.de>
<29916.1436483171@sss.pgh.pa.us>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 2015-07-09 19:06:11 -0400, Tom Lane wrote:
> What evidence have you got to base that value judgement on?
>
> cab9a0656c36739f was based on an actual user complaint, so we have good
> evidence that there are people out there who care about the cost of
> truncating a table many times in one transaction. On the other hand,
> I know of no evidence that anyone's depending on multiple sequential
> COPYs, nor intermixed COPY and INSERT, to be fast. The original argument
> for having this COPY optimization at all was to make restoring pg_dump
> scripts in a single transaction fast; and that use-case doesn't care
> about anything but a single COPY into a virgin table.
Well, you'll hardly have heard complaints about COPY, given that we
behaved like currently for a long while.
I definitely know of ETL like processes that have relied on subsequent
COPYs into truncates relations being cheaper. Can't remember the same
for intermixed COPY and INSERT, but it'd not surprise me if somebody
mixed COPY and UPDATEs rather freely for ETL.
> I think you're worrying about exactly the wrong case.
>
> > My tentative guess is that the best course is to
> > a) Make heap_truncate_one_rel() create a new relfeilnode. That fixes the
> > truncation replay issue.
> > b) Force new pages to be used when using the heap_sync mode in
> > COPY. That avoids the INIT danger you found. It seems rather
> > reasonable to avoid using pages that have already been the target of
> > WAL logging here in general.
>
> And what reason is there to think that this would fix all the
> problems?
Yea, that's the big problem.
> Again, the only known field usage for the COPY optimization is the pg_dump
> scenario; were that not so, we'd have noticed the problem long since.
> So I don't have any faith that this is a well-tested area.
You need to crash in the right moment. I don't think that's that
frequently exercised...
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: andres@anarazel.de, tgl@sss.pgh.pa.us, masao.fujii@gmail.com, kleptog@svana.org
Subject: Re: WAL logging problem in 9.4.3?
In-Reply-To: <20150710085932.GJ10242@alap3.anarazel.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox