From: Jim C. Nasby <jnasby@pervasive.com>
To: Michael Artz <mlartz@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Bulk loading/merging
Date: Mon, 5 Jun 2006 10:04:25 -0500
Message-ID: <20060605150425.GS53487@pervasive.com> (raw)
In-Reply-To: <e9c163070606011104w1d0cf7a6jca8cf5b36f2d8de3@mail.gmail.com>
References: <ce4072df0605261148g4ec22267gc54b6b445e47c855@mail.gmail.com>
<20060531010922.GC59464@pervasive.com>
<e9c163070606011104w1d0cf7a6jca8cf5b36f2d8de3@mail.gmail.com>
On Thu, Jun 01, 2006 at 02:04:46PM -0400, Michael Artz wrote:
> On 5/30/06, Jim C. Nasby <jnasby@pervasive.com> wrote:
>
> >Your best bet is to do this as a single, bulk operation if possible.
> >That way you can simply do an UPDATE ... WHERE EXISTS followed by an
> >INSERT ... SELECT ... WHERE NOT EXISTS.
>
>
>
> hmm, I don't quite understand what you are saying and I think my
> basic misunderstanding is how to use the UPDATE ... WHERE EXISTS to merge
> data in bulk. Assuming that I bulk COPYed the data into a temporary
> table, I'd need to issue an UPDATE for each row in the newly created table,
> right?
>
> For example, for a slightly different key,count schema:
>
> CREATE TABLE kc (key integer, count integer);
>
> and wanting to merge the following data by just updating the count for a
> given key to the equivalent of OLD.count + NEW.count:
>
> 1,10
> 2,15
> 3,45
> 1,30
>
> How would I go about using UPDATE ... WHERE EXISTS to update the "master" kc
> table from a (temporary) table loaded with the above data?
CREATE TEMP TABLE moo () LIKE kc;
COPY ... moo;
BEGIN;
UPDATE kc
SET count=kc.count + moo.count
FROM moo
WHERE moo.key = kc.key
;
INSERT INTO kc(key, count)
SELECT key, count
FROM moo
WHERE NOT EXISTS (
SELECT 1
FROM kc
WHERE kc.key = moo.key
)
;
COMMIT;
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: jnasby@pervasive.com, mlartz@gmail.com
Subject: Re: Bulk loading/merging
In-Reply-To: <20060605150425.GS53487@pervasive.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox