pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Christopher Petrilli <petrilli@gmail.com>
To: jd@commandprompt.com
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Marc G. Fournier <scrappy@postgresql.org>
Cc: pgsql-performance@postgresql.org
Subject: Re: Sustained inserts per sec ... ?
Date: Mon, 4 Apr 2005 09:48:47 -0400
Message-ID: <59d991c40504040648457808e9@mail.gmail.com> (raw)
In-Reply-To: <59d991c4050401125956d6069d@mail.gmail.com>
References: <20050401140421.P18194@ganymede.hub.org>
	<59d991c4050401113857086517@mail.gmail.com>
	<28353.1112388128@sss.pgh.pa.us>
	<59d991c4050401124638c4ab5d@mail.gmail.com>
	<1112388818.3487.34.camel@jd2.commandprompt.com>
	<59d991c4050401125956d6069d@mail.gmail.com>

On Apr 1, 2005 3:59 PM, Christopher Petrilli <petrilli@gmail.com> wrote:
> On Apr 1, 2005 3:53 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
> >
> > > What seems to happen is it slams into a "wall" of some sort, the
> > > system goes into disk write frenzy (wait=90% CPU), and eventually
> > > recovers and starts running for a while at a more normal speed.  What
> > > I need though, is to not have that wall happen.  It is easier for me
> > > to accept a constant degredation of 5%, rather than a 99% degredation
> > > for short periods, as it can cause cascade problems in the system.
> >
> > Could this possibly be a checkpoint happening?
> >
> > Also how many checkpoint segments do you have?
> 
> Changes to the postgresql.conf file from "default":
> 
>     maintenance_work_mem = 131072
>     fsync = false
>     checkpoint_segments = 32

I've now had a chance to run a couple more tests, and here's two
graphs of the time required to insert (via COPY from a file) 500
records at a time:

http://www.amber.org/~petrilli/diagrams/pgsql_copy500.png
http://www.amber.org/~petrilli/diagrams/pgsql_copy500_bgwriter.png

The first is with the above changes, the second contains two
additional modificiations to the configuration:

    bgwriter_percent = 25
    bgwriter_maxpages = 1000 

To my, likely faulty, intuition, it would seem that there is a backup
happening in the moving of data from the WAL to the final resting
place, and that by increasing these I could pull that forward.  As you
can see from the charts, that doesn't seem to have any major impact. 
The point, in the rough middle, is where the program begins inserting
into a new table (inherited). The X axis is the "total" number of rows
inserted. The table has:

    * 21 columns (nothing too strange)
    * No OIDS
    * 5 indexes, including the primary key on a string

They are created by creating a main table, then doing:

    CREATE TABLE foo001 INHERITS (foos);

And then recreating all the indexes.

Thoughts? Any advice would be more than appreciated. 

Chris
-- 
| Christopher Petrilli
| petrilli@gmail.com



view thread (44+ messages)  latest in thread

Message-ID: <59d991c40504040648457808e9@mail.gmail.com>
Permalink:  ../59d991c40504040648457808e9@mail.gmail.com/
Also on:    postgresql.org/message-id/59d991c40504040648457808e9@mail.gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: petrilli@gmail.com, jd@commandprompt.com, tgl@sss.pgh.pa.us, scrappy@postgresql.org
  Subject: Re: Sustained inserts per sec ... ?
  In-Reply-To: <59d991c40504040648457808e9@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox