From: Noah Misch <noah@leadboat.com>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: robertmhaas@gmail.com
Cc: pgsql-hackers@postgresql.org
Cc: 9erthalion6@gmail.com
Cc: andrew.dunstan@2ndquadrant.com
Cc: hlinnaka@iki.fi
Cc: michael@paquier.xyz
Subject: Re: [HACKERS] WAL logging problem in 9.4.3?
Date: Sun, 24 Nov 2019 22:08:39 -0500
Message-ID: <20191125030839.GA51906@gust.leadboat.com> (raw)
In-Reply-To: <20191125.110854.1439648354665518969.horikyota.ntt@gmail.com>
References: <20191118045434.GA1173436@rfd.leadboat.com>
<20191121.160107.1405593316918593372.horikyota.ntt@gmail.com>
<20191123212136.GA41522@gust.leadboat.com>
<20191125.110854.1439648354665518969.horikyota.ntt@gmail.com>
On Mon, Nov 25, 2019 at 11:08:54AM +0900, Kyotaro Horiguchi wrote:
> At Sat, 23 Nov 2019 16:21:36 -0500, Noah Misch <noah@leadboat.com> wrote in
> > This benchmark procedure may help:
> >
> > 1. Determine $DDL_COUNT, a number of DDL transactions that take about one
> > minute when done via syncs.
> > 2. Start "pgbench -rP1 --progress-timestamp -T180 -c10 -j10".
> > 3. Wait 10s.
> > 4. Start one DDL backend that runs $DDL_COUNT transactions.
> > 5. Save DDL start timestamp, DDL end timestamp, and pgbench output.
> >
> > I would compare pgbench tps and latency between the seconds when DDL is and is
> > not running. As you did in earlier tests, I would repeat it using various
> > page counts, with and without sync.
>
> I understood the "DDL" is not pure DDLs but a kind of
> define-then-load, like "CREATE TABLE AS" , "CREATE TABLE" then "COPY
> FROM".
When I wrote "DDL", I meant the four-command transaction that you already used
in benchmarks.
> > I noticed an additional defect:
> >
> > BEGIN;
> > CREATE TABLE t (c) AS SELECT 1;
> > CHECKPOINT; -- write and fsync the table's one page
> > TRUNCATE t; -- no WAL
> > COMMIT; -- no FPI, just the commit record
> >
> > If we crash after the COMMIT and before the next fsync or OS-elected sync of
> > the table's file, the table will stay on disk with its pre-TRUNCATE content.
>
> The TRUNCATE replaces relfilenode in the catalog
No, it does not. Since the relation is new in the transaction, the TRUNCATE
uses the heap_truncate_one_rel() strategy.
> Since the file has no pages, it's right that no FPI emitted.
Correct.
> If we don't rely on such
> behavior, we can make sure thhat by turning the zero-pages case from
> WAL into file sync. I'll do that in the next version.
The zero-pages case is not special. Here's an example of the problem with a
nonzero size:
BEGIN;
CREATE TABLE t (c) AS SELECT * FROM generate_series(1,100000);
CHECKPOINT; -- write and fsync the table's many pages
TRUNCATE t; -- no WAL
INSERT INTO t VALUES (0); -- no WAL
COMMIT; -- FPI for one page; nothing removes the additional pages
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: noah@leadboat.com, horikyota.ntt@gmail.com, robertmhaas@gmail.com, 9erthalion6@gmail.com, andrew.dunstan@2ndquadrant.com, hlinnaka@iki.fi, michael@paquier.xyz
Subject: Re: [HACKERS] WAL logging problem in 9.4.3?
In-Reply-To: <20191125030839.GA51906@gust.leadboat.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox