pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Database Loading
7+ messages / 6 participants
[nested] [flat]

* Database Loading
@ 2008-02-19 18:47  Carol Walter <walterc@indiana.edu>
  0 siblings, 3 replies; 7+ messages in thread

From: Carol Walter @ 2008-02-19 18:47 UTC (permalink / raw)
  To: pgsql-admin

Hello,

I have to load a very large database.  Is there a command that can  
load data in bulk faster than the copy command does?

Carol Walter



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Database Loading
@ 2008-02-19 19:49  Andrew Sullivan <ajs@crankycanuck.ca>
  parent: Carol Walter <walterc@indiana.edu>
  2 siblings, 1 reply; 7+ messages in thread

From: Andrew Sullivan @ 2008-02-19 19:49 UTC (permalink / raw)
  To: pgsql-admin

On Tue, Feb 19, 2008 at 01:47:01PM -0500, Carol Walter wrote:
> I have to load a very large database.  Is there a command that can  
> load data in bulk faster than the copy command does?

No.

A




^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Database Loading
@ 2008-02-19 20:09  Spiegelberg, Greg <gspiegelberg@isodxsolutions.com>
  parent: Andrew Sullivan <ajs@crankycanuck.ca>
  0 siblings, 0 replies; 7+ messages in thread

From: Spiegelberg, Greg @ 2008-02-19 20:09 UTC (permalink / raw)
  To: pgsql-admin

> Andrew Sullivan wrote:
> 
> On Tue, Feb 19, 2008 at 01:47:01PM -0500, Carol Walter wrote:
> > I have to load a very large database.  Is there a command that can 
> > load data in bulk faster than the copy command does?
> 
> No.
> 

There was a thread a while back regarding this.

http://archives.postgresql.org/pgsql-performance/2005-07/msg00372.php

I don't recall though if an answer was found.

Good luck.

CONFIDENTIALITY NOTICE: This e-mail communication and any attachments may contain confidential and privileged information for the use of the designated recipients named above.  If you are not the intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited.  If you have received this communication in error; please notify Cranel Incorporated immediately by telephone at 614-431-8000 or 800-288-3475 and destroy all copies of this communication and any attachments.




^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Database Loading
@ 2008-02-19 20:19  Scott Marlowe <scott.marlowe@gmail.com>
  parent: Carol Walter <walterc@indiana.edu>
  2 siblings, 1 reply; 7+ messages in thread

From: Scott Marlowe @ 2008-02-19 20:19 UTC (permalink / raw)
  To: Carol Walter <walterc@indiana.edu>; +Cc: pgsql-admin

On Feb 19, 2008 12:47 PM, Carol Walter <walterc@indiana.edu> wrote:
> Hello,
>
> I have to load a very large database.  Is there a command that can
> load data in bulk faster than the copy command does?

pg_bulkload

http://pgfoundry.org/projects/pgbulkload/

I've not used it, but have seen it mentioned several times.



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Database Loading
@ 2008-02-19 21:04  Carol Walter <walterc@indiana.edu>
  parent: Scott Marlowe <scott.marlowe@gmail.com>
  0 siblings, 1 reply; 7+ messages in thread

From: Carol Walter @ 2008-02-19 21:04 UTC (permalink / raw)
  To: pgsql-admin

I went to the page.  It said the OS is Linux.  Is there anything for  
Solaris. I've got Solaris 10.

Carol
On Feb 19, 2008, at 3:19 PM, Scott Marlowe wrote:

> On Feb 19, 2008 12:47 PM, Carol Walter <walterc@indiana.edu> wrote:
>> Hello,
>>
>> I have to load a very large database.  Is there a command that can
>> load data in bulk faster than the copy command does?
>
> pg_bulkload
>
> http://pgfoundry.org/projects/pgbulkload/
>
> I've not used it, but have seen it mentioned several times.
>
> ---------------------------(end of  
> broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster




^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Database Loading
@ 2008-02-19 22:06  Peter Koczan <pjkoczan@gmail.com>
  parent: Carol Walter <walterc@indiana.edu>
  0 siblings, 0 replies; 7+ messages in thread

From: Peter Koczan @ 2008-02-19 22:06 UTC (permalink / raw)
  To: Carol Walter <walterc@indiana.edu>; +Cc: pgsql-admin

On Feb 19, 2008 3:04 PM, Carol Walter <walterc@indiana.edu> wrote:
> I went to the page.  It said the OS is Linux.  Is there anything for
> Solaris. I've got Solaris 10.

It looks like it's just some C code, a shell file, and some sql
scripts. It should compile and run fine on any postgresql supported
platform (the operative word being "should"). They probably just
listed Linux as the OS as it's likely the primary OS on which they
wrote and tested it.

Peter



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Database Loading
@ 2008-02-20 09:17  Dimitri Fontaine <dfontaine@hi-media.com>
  parent: Carol Walter <walterc@indiana.edu>
  2 siblings, 0 replies; 7+ messages in thread

From: Dimitri Fontaine @ 2008-02-20 09:17 UTC (permalink / raw)
  To: pgsql-admin; +Cc: Carol Walter <walterc@indiana.edu>

Hi,

Le mardi 19 février 2008, Carol Walter a écrit :
> I have to load a very large database.  Is there a command that can
> load data in bulk faster than the copy command does?

The next version of pgloader has been asked to implement some new parallel 
loading of a single file, which could speed up things by having several 
backends importing data at once. The implementation of this is now done and 
ready to get some tests, but still at a beta or release candidate status.

If you want to try it (and report about your tests?), here's the current 
version of it:
 http://pgloader.projects.postgresql.org/dev/
 http://pgloader.projects.postgresql.org/dev/pgloader.1.html#_parallel_loading

Regards,
-- 
dim

Attachments:

  signature.asc (application/pgp-signature, 188B) download

^ permalink  raw  reply  [nested|flat] 7+ messages in thread


end of thread, other threads:[~2008-02-20 09:17 UTC | newest]

Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2008-02-19 18:47 Database Loading Carol Walter <walterc@indiana.edu>
2008-02-19 19:49 ` Andrew Sullivan <ajs@crankycanuck.ca>
2008-02-19 20:09   ` Spiegelberg, Greg <gspiegelberg@isodxsolutions.com>
2008-02-19 20:19 ` Scott Marlowe <scott.marlowe@gmail.com>
2008-02-19 21:04   ` Carol Walter <walterc@indiana.edu>
2008-02-19 22:06     ` Peter Koczan <pjkoczan@gmail.com>
2008-02-20 09:17 ` Dimitri Fontaine <dfontaine@hi-media.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox