pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Christopher Petrilli <petrilli@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Greg Stark <gsstark@mit.edu>
Cc: jd@commandprompt.com, "Marc G. Fournier" <scrappy@postgresql.org>
Cc: pgsql-performance@postgresql.org
Subject: Re: Sustained inserts per sec ... ?
Date: Tue, 5 Apr 2005 00:16:27 -0400
Message-ID: <59d991c405040421164aeea3fb@mail.gmail.com> (raw)
In-Reply-To: <7438.1112673460@sss.pgh.pa.us>
References: <20050401140421.P18194@ganymede.hub.org>
	<59d991c4050401125956d6069d@mail.gmail.com>
	<59d991c40504040648457808e9@mail.gmail.com>
	<26454.1112629979@sss.pgh.pa.us>
	<59d991c405040409094e7bbd17@mail.gmail.com>
	<26669.1112631807@sss.pgh.pa.us>
	<59d991c40504041846bc39a6@mail.gmail.com>
	<5217.1112668589@sss.pgh.pa.us>
	<87y8bxophw.fsf@stark.xeocode.com>
	<7438.1112673460@sss.pgh.pa.us>

On Apr 4, 2005 11:57 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Greg Stark <gsstark@mit.edu> writes:
> > All this is happening within a single transaction too, right? So there hasn't
> > been an fsync the entire time. It's entirely up to the kernel when to decide
> > to start writing data.
> 
> No ... there's a commit every 500 records.  However, I think Chris said
> he was running with fsync off; so you're right that the kernel is at
> liberty to write stuff to disk when it feels like.  It could be that
> those outlier points are transactions that occurred in the middle of
> periodic syncer-driven mass writes.  Maybe fsync off is
> counterproductive for this situation?

Looking at preliminary results from running with shared_buffers at
16000, it seems this may be correct.  Performance was flatter for a
BIT longer, but slammed right into the wall and started hitting the
3-30 second range per COPY.  I've restarted the run, with fsync turned
on (fdatasync), and we'll see.

My fear is that it's some bizarre situation interacting with both
issues, and one that might not be solvable.  Does anyone else have
much experience with this sort of sustained COPY?

Chris
-- 
| Christopher Petrilli
| petrilli@gmail.com



view thread (44+ messages)  latest in thread

Message-ID: <59d991c405040421164aeea3fb@mail.gmail.com>
Permalink:  ../59d991c405040421164aeea3fb@mail.gmail.com/
Also on:    postgresql.org/message-id/59d991c405040421164aeea3fb@mail.gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: petrilli@gmail.com, tgl@sss.pgh.pa.us, gsstark@mit.edu, scrappy@postgresql.org
  Subject: Re: Sustained inserts per sec ... ?
  In-Reply-To: <59d991c405040421164aeea3fb@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox