pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedpartition query on multiple cores
6+ messages / 5 participants
[nested] [flat]
* partition query on multiple cores
@ 2011-05-10 04:32 Didik Prasetyo <prasetyodidik62@yahoo.com>
0 siblings, 1 reply; 6+ messages in thread
From: Didik Prasetyo @ 2011-05-10 04:32 UTC (permalink / raw)
To: pgsql-performance
I have 8-core server, I wanted to ask whether a query can be divided for
multiple processors or cores, if it could be what to do in postgresql
Thanks
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: partition query on multiple cores
@ 2011-05-10 15:06 Maciek Sakrejda <msakrejda@truviso.com>
parent: Didik Prasetyo <prasetyodidik62@yahoo.com>
0 siblings, 1 reply; 6+ messages in thread
From: Maciek Sakrejda @ 2011-05-10 15:06 UTC (permalink / raw)
To: Didik Prasetyo <prasetyodidik62@yahoo.com>; +Cc: pgsql-performance
> I have 8-core server, I wanted to ask whether a query can be divided for
> multiple processors or cores, if it could be what to do in postgresql
No, at this time (and for the foreseeable future), a single query will
run on a single core.
---
Maciek Sakrejda | System Architect | Truviso
1065 E. Hillsdale Blvd., Suite 215
Foster City, CA 94404
(650) 242-3500 Main
www.truviso.com
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: partition query on multiple cores
@ 2011-05-10 16:22 Shaun Thomas <sthomas@peak6.com>
parent: Maciek Sakrejda <msakrejda@truviso.com>
0 siblings, 2 replies; 6+ messages in thread
From: Shaun Thomas @ 2011-05-10 16:22 UTC (permalink / raw)
To: Maciek Sakrejda <msakrejda@truviso.com>; +Cc: Didik Prasetyo <prasetyodidik62@yahoo.com>; pgsql-performance
On 05/10/2011 10:06 AM, Maciek Sakrejda wrote:
>> I have 8-core server, I wanted to ask whether a query can be divided for
>> multiple processors or cores, if it could be what to do in postgresql
>
> No, at this time (and for the foreseeable future), a single query will
> run on a single core.
It can *kinda* be done. Take a look at GridSQL. It's really good for
splitting up reporting-like queries that benefit from parallel access of
large tables. It's not exactly Hadoop, but I ran a test on a single
system with two separate instances of PostgreSQL, and a single query
over those two nodes cut execution time in half.
It's meant for server parallelism, so I wouldn't necessarily recommend
splitting your data up across nodes on the same server. But it seems to
deliver as promised when used in the right circumstances.
--
Shaun Thomas
OptionsHouse | 141 W. Jackson Blvd. | Suite 800 | Chicago IL, 60604
312-676-8870
sthomas@peak6.com
______________________________________________
See http://www.peak6.com/email_disclaimer.php
for terms and conditions related to this email
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: partition query on multiple cores
@ 2011-05-10 18:57 Tomas Vondra <tv@fuzzy.cz>
parent: Shaun Thomas <sthomas@peak6.com>
1 sibling, 1 reply; 6+ messages in thread
From: Tomas Vondra @ 2011-05-10 18:57 UTC (permalink / raw)
To: pgsql-performance
Dne 10.5.2011 18:22, Shaun Thomas napsal(a):
> On 05/10/2011 10:06 AM, Maciek Sakrejda wrote:
>
>>> I have 8-core server, I wanted to ask whether a query can be divided for
>>> multiple processors or cores, if it could be what to do in postgresql
>>
>> No, at this time (and for the foreseeable future), a single query will
>> run on a single core.
>
> It can *kinda* be done. Take a look at GridSQL.
Or pgpool-II, that can give you something similar.
http://pgpool.projects.postgresql.org/
regards
Tomas
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: partition query on multiple cores
@ 2011-05-11 11:56 Mason S <masonlists@gmail.com>
parent: Shaun Thomas <sthomas@peak6.com>
1 sibling, 0 replies; 6+ messages in thread
From: Mason S @ 2011-05-11 11:56 UTC (permalink / raw)
To: sthomas@peak6.com; +Cc: Maciek Sakrejda <msakrejda@truviso.com>; Didik Prasetyo <prasetyodidik62@yahoo.com>; pgsql-performance
On Tue, May 10, 2011 at 12:22 PM, Shaun Thomas <sthomas@peak6.com> wrote:
> On 05/10/2011 10:06 AM, Maciek Sakrejda wrote:
>
> I have 8-core server, I wanted to ask whether a query can be divided for
>>> multiple processors or cores, if it could be what to do in postgresql
>>>
>>
>> No, at this time (and for the foreseeable future), a single query will
>> run on a single core.
>>
>
> It can *kinda* be done. Take a look at GridSQL. It's really good for
> splitting up reporting-like queries that benefit from parallel access of
> large tables. It's not exactly Hadoop, but I ran a test on a single system
> with two separate instances of PostgreSQL, and a single query over those two
> nodes cut execution time in half.
>
> It's meant for server parallelism, so I wouldn't necessarily recommend
> splitting your data up across nodes on the same server. But it seems to
> deliver as promised when used in the right circumstances.
>
>
Yes, GridSQL is useful even in multi-core scenarios on a single server for
query parallelism. You can also use the same PostgreSQL instance (cluster),
as the virtual node databases are named distinctly, which simplifies
configuration.
Mason
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: partition query on multiple cores
@ 2011-05-11 12:04 Mason S <masonlists@gmail.com>
parent: Tomas Vondra <tv@fuzzy.cz>
0 siblings, 0 replies; 6+ messages in thread
From: Mason S @ 2011-05-11 12:04 UTC (permalink / raw)
To: Tomas Vondra <tv@fuzzy.cz>; +Cc: pgsql-performance
On Tue, May 10, 2011 at 2:57 PM, Tomas Vondra <tv@fuzzy.cz> wrote:
> Dne 10.5.2011 18:22, Shaun Thomas napsal(a):
> > On 05/10/2011 10:06 AM, Maciek Sakrejda wrote:
> >
> >>> I have 8-core server, I wanted to ask whether a query can be divided
> for
> >>> multiple processors or cores, if it could be what to do in postgresql
> >>
> >> No, at this time (and for the foreseeable future), a single query will
> >> run on a single core.
> >
> > It can *kinda* be done. Take a look at GridSQL.
>
> Or pgpool-II, that can give you something similar.
>
> http://pgpool.projects.postgresql.org/
>
>
Last time I tested parallelism in pgpool-II, I saw that if your query is
fairly simple, pgpool-II will help. If it is more complex with joins and
aggregates, GridSQL will typically outperform it. GridSQL pushes down joins
as much as possible, minimizes row shipping, and parallelizes aggregates and
grouping.
Mason Sharp
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2011-05-11 12:04 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2011-05-10 04:32 partition query on multiple cores Didik Prasetyo <prasetyodidik62@yahoo.com>
2011-05-10 15:06 ` Maciek Sakrejda <msakrejda@truviso.com>
2011-05-10 16:22 ` Shaun Thomas <sthomas@peak6.com>
2011-05-10 18:57 ` Tomas Vondra <tv@fuzzy.cz>
2011-05-11 12:04 ` Mason S <masonlists@gmail.com>
2011-05-11 11:56 ` Mason S <masonlists@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox