pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedOut of memory running 560 MB query
4+ messages / 3 participants
[nested] [flat]
* Out of memory running 560 MB query
@ 2014-09-15 00:18 Morgan Ramsay <morgan.ramsay@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Morgan Ramsay @ 2014-09-15 00:18 UTC (permalink / raw)
To: pgsql-admin
I have 560 MB worth of INSERT statements in a .sql file. I'm trying to run
that query but I'm getting an out of memory error.
Any idea how to successfully run a query that large? What's the size limit
of a query?
-Morgan
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Out of memory running 560 MB query
@ 2014-09-15 00:28 Rosser Schwarz <rosser.schwarz@gmail.com>
parent: Morgan Ramsay <morgan.ramsay@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Rosser Schwarz @ 2014-09-15 00:28 UTC (permalink / raw)
To: Morgan Ramsay <morgan.ramsay@gmail.com>; +Cc: pgsql-admin
On Sun, Sep 14, 2014 at 5:18 PM, Morgan Ramsay <morgan.ramsay@gmail.com>
wrote:
> Any idea how to successfully run a query that large? What’s the size limit
> of a query?
>
Without seeing your actual query (or a syntactically similar one, if there
are private details you can't disclose), it's difficult to say. In
general, though, a couple of thoughts come to mind:
Can you run it as a single COPY statement, instead of however many
individual INSERTS? Failing that, can you use the multi-row VALUES syntax
for your INSERT?
As an aside, this type of question is probably more suited for the
pgsql-sql list, rather than -admin.
rls
--
:wq
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Out of memory running 560 MB query
@ 2014-09-15 00:53 Morgan Ramsay <morgan.ramsay@gmail.com>
parent: Rosser Schwarz <rosser.schwarz@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Morgan Ramsay @ 2014-09-15 00:53 UTC (permalink / raw)
To: 'Rosser Schwarz' <rosser.schwarz@gmail.com>; +Cc: pgsql-admin
Sorry, I was running the query on what appeared to be a cached copy of the old table—or something.
After a reboot, everything seems to be working. The admin client wasn’t responding but now all of my data is there.
With regard to the list, I wasn’t sure whether there was a .conf setting that limited the size of queries or whether there was a query size limit.
—Morgan
From: Rosser Schwarz [mailto:rosser.schwarz@gmail.com]
Sent: Sunday, September 14, 2014 5:28 PM
To: Morgan Ramsay
Cc: PgSQL ADMIN
Subject: Re: [ADMIN] Out of memory running 560 MB query
On Sun, Sep 14, 2014 at 5:18 PM, Morgan Ramsay <morgan.ramsay@gmail.com> wrote:
Any idea how to successfully run a query that large? What’s the size limit of a query?
Without seeing your actual query (or a syntactically similar one, if there are private details you can't disclose), it's difficult to say. In general, though, a couple of thoughts come to mind:
Can you run it as a single COPY statement, instead of however many individual INSERTS? Failing that, can you use the multi-row VALUES syntax for your INSERT?
As an aside, this type of question is probably more suited for the pgsql-sql list, rather than -admin.
rls
--
:wq
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Out of memory running 560 MB query
@ 2014-09-15 04:47 Rajesh Madiwale <rajeshmadiwale65@gmail.com>
parent: Morgan Ramsay <morgan.ramsay@gmail.com>
0 siblings, 0 replies; 4+ messages in thread
From: Rajesh Madiwale @ 2014-09-15 04:47 UTC (permalink / raw)
To: Morgan Ramsay <morgan.ramsay@gmail.com>; +Cc: Rosser Schwarz <rosser.schwarz@gmail.com>; pgsql-admin
On Mon, Sep 15, 2014 at 6:23 AM, Morgan Ramsay <morgan.ramsay@gmail.com>
wrote:
> Sorry, I was running the query on what appeared to be a cached copy of the
> old table—or something.
>
>
>
> After a reboot, everything seems to be working. The admin client wasn’t
> responding but now all of my data is there.
>
>
>
> With regard to the list, I wasn’t sure whether there was a .conf setting
> that limited the size of queries or whether there was a query size limit.
>
>
Hi,
I suspect this issue occured due to no proper configuration of parameter(
shared_buffer and maintenance_work_mem) in postgresql.conf file.
For greater insert/copy operation shared_buffer and maintenance_work_mem
parameters should be configured to resolve the issue.
Regards,
Rajesh Madiwale,
Database Administrator,
Shreeyansh Technologies.
>
>
> —Morgan
>
>
>
>
>
> *From:* Rosser Schwarz [mailto:rosser.schwarz@gmail.com]
> *Sent:* Sunday, September 14, 2014 5:28 PM
> *To:* Morgan Ramsay
> *Cc:* PgSQL ADMIN
> *Subject:* Re: [ADMIN] Out of memory running 560 MB query
>
>
>
> On Sun, Sep 14, 2014 at 5:18 PM, Morgan Ramsay <morgan.ramsay@gmail.com>
> wrote:
>
> Any idea how to successfully run a query that large? What’s the size limit
> of a query?
>
>
>
> Without seeing your actual query (or a syntactically similar one, if there
> are private details you can't disclose), it's difficult to say. In
> general, though, a couple of thoughts come to mind:
>
>
>
> Can you run it as a single COPY statement, instead of however many
> individual INSERTS? Failing that, can you use the multi-row VALUES syntax
> for your INSERT?
>
>
>
> As an aside, this type of question is probably more suited for the
> pgsql-sql list, rather than -admin.
>
>
>
> rls
>
>
>
> --
> :wq
>
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2014-09-15 04:47 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-09-15 00:18 Out of memory running 560 MB query Morgan Ramsay <morgan.ramsay@gmail.com>
2014-09-15 00:28 ` Rosser Schwarz <rosser.schwarz@gmail.com>
2014-09-15 00:53 ` Morgan Ramsay <morgan.ramsay@gmail.com>
2014-09-15 04:47 ` Rajesh Madiwale <rajeshmadiwale65@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox