pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Bruce Momjian <bruce@momjian.us>
To: Heikki Linnakangas <heikki@enterprisedb.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Craig Ringer <craig@postnewspapers.com.au>
Cc: pgsql-patches <pgsql-patches@postgresql.org>
Subject: Re: [PERFORM] Very slow (2 tuples/second) sequential scan after bulk insert; speed returns to ~500 tuples/second after commit
Date: Sun, 16 Mar 2008 22:21:24 -0400 (EDT)
Message-ID: <200803170221.m2H2LOp12205@momjian.us> (raw)
In-Reply-To: <47D7DE12.80700@enterprisedb.com>


This has been applied by Tom.

---------------------------------------------------------------------------

Heikki Linnakangas wrote:
> Tom Lane wrote:
> > "Heikki Linnakangas" <heikki@enterprisedb.com> writes:
> >> Elsewhere in our codebase where we use arrays that are enlarged as 
> >> needed, we keep track of the "allocated" size and the "used" size of the 
> >> array separately, and only call repalloc when the array fills up, and 
> >> repalloc a larger than necessary array when it does. I chose to just 
> >> call repalloc every time instead, as repalloc is smart enough to fall 
> >> out quickly if the chunk the allocation was made in is already larger 
> >> than the new size. There might be some gain avoiding the repeated 
> >> repalloc calls, but I doubt it's worth the code complexity, and calling 
> >> repalloc with a larger than necessary size can actually force it to 
> >> unnecessarily allocate a new, larger chunk instead of reusing the old 
> >> one. Thoughts on that?
> > 
> > Seems like a pretty bad idea to me, as the behavior you're counting on
> > only applies to chunks up to 8K or thereabouts. 
> 
> Oh, you're right. Though I'm sure libc realloc has all kinds of smarts 
> as well, it does seem better to not rely too much on that.
> 
> > In a situation where
> > you are subcommitting lots of XIDs one at a time, this is likely to have
> > quite awful behavior (or at least, you're at the mercy of the local
> > malloc library as to how bad it is).  I'd go with the same
> > double-it-each-time-needed approach we use elsewhere.
> 
> Yep, patch attached. I also changed xactGetCommittedChildren to return 
> the original array instead of copying it, as Alvaro suggested.
> 
> -- 
>    Heikki Linnakangas
>    EnterpriseDB   http://www.enterprisedb.com


> 
> -- 
> Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-patches

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +



view thread (30+ messages)

Message-ID: <200803170221.m2H2LOp12205@momjian.us>
Permalink:  ../200803170221.m2H2LOp12205@momjian.us/
Also on:    postgresql.org/message-id/200803170221.m2H2LOp12205@momjian.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: bruce@momjian.us, heikki@enterprisedb.com, tgl@sss.pgh.pa.us, craig@postnewspapers.com.au, pgsql-patches@postgresql.org
  Subject: Re: [PERFORM] Very slow (2 tuples/second) sequential scan after bulk insert; speed returns to ~500 tuples/second after commit
  In-Reply-To: <200803170221.m2H2LOp12205@momjian.us>
  References: <47D7DE12.80700@enterprisedb.com> <200803170221.m2H2LOp12205@momjian.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox