pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Scott Marlowe <scott.marlowe@gmail.com>
To: Mark Stosberg <mark@summersault.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: Re: best practice for moving millions of rows to child table when setting up partitioning?
Date: Wed, 27 Apr 2011 18:26:14 -0600
Message-ID: <BANLkTimihm0e4DewUxkSZ_YNs4wkK8VpZA@mail.gmail.com> (raw)
In-Reply-To: <ip9tkc$jek$1@dough.gmane.org>
References: <ip9abr$nk8$1@dough.gmane.org>
	<ip9tkc$jek$1@dough.gmane.org>

I had a similar problem about a year ago,  The parent table had about
1.5B rows each with a unique ID from a bigserial.  My approach was to
create all the child tables needed for the past and the next month or
so.  Then, I simple did something like:

begin;
insert into table select * from only table where id between 1 and 10000000;
delete from only table where id between 1 and 10000000;
-- first few times check to make sure it's working of course
commit;
begin;
insert into table select * from only table where id between 10000001
and 20000000;
delete from only table where id between 10000001 and 20000000;
commit;

and so on.  New entries were already going into the child tables as
they showed up, old entries were migrating 10M rows at a time.  This
kept the moves small enough so as not to run the machine out of any
resource involved in moving 1.5B rows at once.



view thread (15+ messages)  latest in thread

Message-ID: <BANLkTimihm0e4DewUxkSZ_YNs4wkK8VpZA@mail.gmail.com>
Permalink:  ../BANLkTimihm0e4DewUxkSZ_YNs4wkK8VpZA@mail.gmail.com/
Also on:    postgresql.org/message-id/BANLkTimihm0e4DewUxkSZ_YNs4wkK8VpZA@mail.gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-admin@postgresql.org
  Cc: scott.marlowe@gmail.com, mark@summersault.com
  Subject: Re: Re: best practice for moving millions of rows to child table when setting up partitioning?
  In-Reply-To: <BANLkTimihm0e4DewUxkSZ_YNs4wkK8VpZA@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox