From: Joe Conway <mail@joeconway.com>
To: Chris Hoover <revoohc@sermonaudio.com>
Cc: PostgreSQL Admin <pgsql-admin@postgresql.org>
Subject: Re: Copying large tables with DBLink
Date: Thu, 24 Mar 2005 11:21:10 -0800
Message-ID: <42431326.2010908@joeconway.com> (raw)
In-Reply-To: <42430E20.1010600@sermonaudio.com>
References: <42430E20.1010600@sermonaudio.com>
Chris Hoover wrote:
> Has anyone had problems with memory exhaustion and dblink? We were
> trying to use dblink to convert our databases to our new layout, and had
> our test server lock up several times when trying to copy a table that
> was significantly larger than our memory and swap.
> Basically where were doing an insert into <table> select * from
> dblink('dbname=olddb','select * from large_table) as t_large_table(table
> column listing);
>
> Does anyone know of a way around this?
dblink just uses libpq, and libpq reads the entire result into memory.
There is no direct way around that that I'm aware of. You could,
however, use a cursor, and fetch/manipulate rows in more reasonably
sized groups.
HTH,
Joe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: mail@joeconway.com, revoohc@sermonaudio.com
Subject: Re: Copying large tables with DBLink
In-Reply-To: <42431326.2010908@joeconway.com>
References: <42430E20.1010600@sermonaudio.com> <42431326.2010908@joeconway.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox