From: Rural Hunter <ruralhunter@gmail.com>
To: Julio Leyva <jcleyva@hotmail.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: plsql gets "out of memory"
Date: Mon, 29 Aug 2011 21:37:08 +0800
Message-ID: <4E5B9604.5070602@gmail.com> (raw)
In-Reply-To: <BAY154-W12A115156AED066911581DA1140@phx.gbl>
References: <4E5B8FF7.4040007@gmail.com>
<BAY154-W12A115156AED066911581DA1140@phx.gbl>
well, thank you for the quick reply but actually I'm not concerning the
performance as of now. My problem is related to the bulk insert of
client side program psql. Or, it's a problem of migrating vast data from
mysql to pgsql.
于2011年8月29日 21:25:29,Julio Leyva写到:
>
> check this out
> http://www.revsys.com/writings/postgresql-performance.html
>
>
> > Date: Mon, 29 Aug 2011 21:11:19 +0800
> > From: ruralhunter@gmail.com
> > To: pgsql-admin@postgresql.org
> > Subject: [ADMIN] plsql gets "out of memory"
> >
> > Hi all,
> > I'm a newbie here. I'm trying to test pgsql with my mysql data. If the
> > performance is good, I will migrate from mysql to pgsql.
> > I installed pgsql 9.1rc on my Ubuntu server. I'm trying to import a
> > large sql file dumped from mysql into pgsql with 'plsql -f'. The file is
> > around 30G with bulk insert commands in it. It rans several hours and
> > then aborted with an "out of memory" error. This is the tail of the log
> > I got:
> > INSERT 0 280
> > INSERT 0 248
> > INSERT 0 210
> > INSERT 0 199
> > invalid command \n
> > out of memory
> >
> > On server side, I only found these errors related to invalid UTF-8
> > characters which is related to escape characters when exported from
> mysql.
> > 2011-08-29 19:19:29 CST ERROR: invalid byte sequence for encoding
> > "UTF8": 0x00
> > 2011-08-29 19:55:35 CST LOG: unexpected EOF on client connection
> >
> > My understanding is this is a client side issue and not related to any
> > server memory setting. But how can ajust the memory setting of the psql
> > program?
> >
> > To handle the escape character '\' which is default in mysql but not in
> > pgsql, I have already made some rough modification to the exported sql
> > dump file:
> > sed "s/,'/,E'/g" |sed 's/\\0/ /g'. I guess there might still be some
> > characters missing handling and that might cause the insert command to
> > be split to several invalid pgsql commands. Would that be the cause of
> > the "out of memory" error?
> >
> >
> > --
> > Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-admin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: ruralhunter@gmail.com, jcleyva@hotmail.com
Subject: Re: plsql gets "out of memory"
In-Reply-To: <4E5B9604.5070602@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox