pg.ddx.io pgsql-cluster-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedPostgresql Write cluster needed
8+ messages / 5 participants
[nested] [flat]
* Postgresql Write cluster needed
@ 2014-11-21 11:29 fahad basheer <fahad.a.basheer@gmail.com>
0 siblings, 1 reply; 8+ messages in thread
From: fahad basheer @ 2014-11-21 11:29 UTC (permalink / raw)
To: pgsql-cluster-hackers
Hi ALL,
I need to scale postgresql with write capabilities, please help me.
Regards
Fahad
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql Write cluster needed
@ 2014-11-21 11:38 Mario Splivalo <mario@splivalo.hr>
parent: fahad basheer <fahad.a.basheer@gmail.com>
0 siblings, 1 reply; 8+ messages in thread
From: Mario Splivalo @ 2014-11-21 11:38 UTC (permalink / raw)
To: fahad basheer <fahad.a.basheer@gmail.com>; pgsql-cluster-hackers
Unless you shard (partition) your data, writes can't be scaled.
Mario
On November 21, 2014 12:29:38 PM CET, fahad basheer <fahad.a.basheer@gmail.com> wrote:
>Hi ALL,
>
>I need to scale postgresql with write capabilities, please help me.
>
>Regards
>Fahad
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql Write cluster needed
@ 2014-11-21 11:56 Sergei Sheinin <sergei.sheinin@gmail.com>
parent: Mario Splivalo <mario@splivalo.hr>
0 siblings, 1 reply; 8+ messages in thread
From: Sergei Sheinin @ 2014-11-21 11:56 UTC (permalink / raw)
To: Mario Splivalo <mario@splivalo.hr>; +Cc: fahad basheer <fahad.a.basheer@gmail.com>; pgsql-cluster-hackers
It may be possible to write to text files then use COPY command for bulk
inserts.
On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
> Unless you shard (partition) your data, writes can't be scaled.
>
> Mario
>
>
> On November 21, 2014 12:29:38 PM CET, fahad basheer <
> fahad.a.basheer@gmail.com> wrote:
>>
>> Hi ALL,
>>
>> I need to scale postgresql with write capabilities, please help me.
>>
>> Regards
>> Fahad
>>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql Write cluster needed
@ 2014-11-21 12:15 desmodemone <desmodemone@gmail.com>
parent: Sergei Sheinin <sergei.sheinin@gmail.com>
0 siblings, 1 reply; 8+ messages in thread
From: desmodemone @ 2014-11-21 12:15 UTC (permalink / raw)
To: Sergei Sheinin <sergei.sheinin@gmail.com>; +Cc: Mario Splivalo <mario@splivalo.hr>; fahad basheer <fahad.a.basheer@gmail.com>; pgsql-cluster-hackers
2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:
> It may be possible to write to text files then use COPY command for bulk
> inserts.
>
> On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
>
>> Unless you shard (partition) your data, writes can't be scaled.
>>
>> Mario
>>
>>
>> On November 21, 2014 12:29:38 PM CET, fahad basheer <
>> fahad.a.basheer@gmail.com> wrote:
>>>
>>> Hi ALL,
>>>
>>> I need to scale postgresql with write capabilities, please help me.
>>>
>>> Regards
>>> Fahad
>>>
>>
>> --
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>
>
Hi,
I think we need more informations about this problem and how it's
the enviroment (OLTP, dwh ? ) . For example if some tables not need to be
full ACID you could use unlogged table or for example you could relax the
durability of the commit , disabling synchronous commit .
Regards
--
Matteo Durighetto
- - - - - - - - - - - - - - - - - - - - - - -
Italian PostgreSQL User Group <http://www.itpug.org/index.it.html;
Italian Community for Geographic Free/Open-Source Software
<http://www.gfoss.it;
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql Write cluster needed
@ 2014-11-24 00:28 Koichi Suzuki <koichi.dbms@gmail.com>
parent: desmodemone <desmodemone@gmail.com>
0 siblings, 1 reply; 8+ messages in thread
From: Koichi Suzuki @ 2014-11-24 00:28 UTC (permalink / raw)
To: desmodemone <desmodemone@gmail.com>; +Cc: Sergei Sheinin <sergei.sheinin@gmail.com>; Mario Splivalo <mario@splivalo.hr>; fahad basheer <fahad.a.basheer@gmail.com>; pgsql-cluster-hackers
Write scalability for OLTP workload is available with Postgres-XC.
---
Koichi Suzuki
2014-11-21 21:15 GMT+09:00 desmodemone <desmodemone@gmail.com>:
>
>
> 2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:
>>
>> It may be possible to write to text files then use COPY command for bulk
>> inserts.
>>
>> On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
>>>
>>> Unless you shard (partition) your data, writes can't be scaled.
>>>
>>> Mario
>>>
>>>
>>> On November 21, 2014 12:29:38 PM CET, fahad basheer
>>> <fahad.a.basheer@gmail.com> wrote:
>>>>
>>>> Hi ALL,
>>>>
>>>> I need to scale postgresql with write capabilities, please help me.
>>>>
>>>> Regards
>>>> Fahad
>>>
>>>
>>> --
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>>
>
>
> Hi,
> I think we need more informations about this problem and how it's
> the enviroment (OLTP, dwh ? ) . For example if some tables not need to be
> full ACID you could use unlogged table or for example you could relax the
> durability of the commit , disabling synchronous commit .
>
> Regards
>
>
> --
> Matteo Durighetto
>
> - - - - - - - - - - - - - - - - - - - - - - -
>
> Italian PostgreSQL User Group
> Italian Community for Geographic Free/Open-Source Software
--
Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-cluster-hackers
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql Write cluster needed
@ 2014-11-24 09:30 fahad basheer <fahad.a.basheer@gmail.com>
parent: Koichi Suzuki <koichi.dbms@gmail.com>
0 siblings, 1 reply; 8+ messages in thread
From: fahad basheer @ 2014-11-24 09:30 UTC (permalink / raw)
To: Koichi Suzuki <koichi.dbms@gmail.com>; +Cc: desmodemone <desmodemone@gmail.com>; Sergei Sheinin <sergei.sheinin@gmail.com>; Mario Splivalo <mario@splivalo.hr>; pgsql-cluster-hackers
Hi All,
Thanks for the fast response
Database is used as OLTP database.
Frontend application used is OpenERP.
I have tried postgresql xc,but its not supported with OpenERP.
On Mon, Nov 24, 2014 at 5:58 AM, Koichi Suzuki <koichi.dbms@gmail.com>
wrote:
> Write scalability for OLTP workload is available with Postgres-XC.
> ---
> Koichi Suzuki
>
>
> 2014-11-21 21:15 GMT+09:00 desmodemone <desmodemone@gmail.com>:
> >
> >
> > 2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:
> >>
> >> It may be possible to write to text files then use COPY command for bulk
> >> inserts.
> >>
> >> On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr>
> wrote:
> >>>
> >>> Unless you shard (partition) your data, writes can't be scaled.
> >>>
> >>> Mario
> >>>
> >>>
> >>> On November 21, 2014 12:29:38 PM CET, fahad basheer
> >>> <fahad.a.basheer@gmail.com> wrote:
> >>>>
> >>>> Hi ALL,
> >>>>
> >>>> I need to scale postgresql with write capabilities, please help me.
> >>>>
> >>>> Regards
> >>>> Fahad
> >>>
> >>>
> >>> --
> >>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> >>
> >>
> >
> >
> > Hi,
> > I think we need more informations about this problem and how
> it's
> > the enviroment (OLTP, dwh ? ) . For example if some tables not need to be
> > full ACID you could use unlogged table or for example you could relax
> the
> > durability of the commit , disabling synchronous commit .
> >
> > Regards
> >
> >
> > --
> > Matteo Durighetto
> >
> > - - - - - - - - - - - - - - - - - - - - - - -
> >
> > Italian PostgreSQL User Group
> > Italian Community for Geographic Free/Open-Source Software
>
Regards,
Fahad
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql Write cluster needed
@ 2014-11-24 10:21 Mario Splivalo <mario@splivalo.hr>
parent: fahad basheer <fahad.a.basheer@gmail.com>
0 siblings, 1 reply; 8+ messages in thread
From: Mario Splivalo @ 2014-11-24 10:21 UTC (permalink / raw)
To: pgsql-cluster-hackers
On 11/24/2014 10:30 AM, fahad basheer wrote:
> Hi All,
>
> Thanks for the fast response
>
> Database is used as OLTP database.
>
> Frontend application used is OpenERP.
>
> I have tried postgresql xc,but its not supported with OpenERP.
>
>
I don't see why OpenERP wouldn't work with pg-xc, but I'm also not sure
if you would benefit from it as deciding on what to partition your data
on is not a simple task.
What is your main performance issue?
In theory, if you want to scale writes you need to scale your data. So,
you'd, for instance, put all the even IDs on server A and all the odd
IDs on server B. That way you double the writing throughput. But, when
querying your data you need to somehow 'merge' both partitions, the one
on server A and the one on server B.
This is, actually, where Postgres-XC comes in - it will allow you to
create partitions of your data in a way that is oblivious to the
application itself. But for OpenERP I'm not sure if you will gain
anything, and your setup will be quite complicated.
You either need better server (with better I/O performance - add more
spindles if you're on RAID10, or switch to ssd), or partition your data.
If you have multiple tenants in your OpenERP maybe you could move each
of them to their separate servers?
Mario
--
Mario Splivalo
mario@splivalo.hr
"I can do it quick, I can do it cheap, I can do it well. Pick any two."
--
Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-cluster-hackers
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Postgresql Write cluster needed
@ 2014-11-27 01:40 Koichi Suzuki <koichi.dbms@gmail.com>
parent: Mario Splivalo <mario@splivalo.hr>
0 siblings, 0 replies; 8+ messages in thread
From: Koichi Suzuki @ 2014-11-27 01:40 UTC (permalink / raw)
To: Mario Splivalo <mario@splivalo.hr>; +Cc: pgsql-cluster-hackers
I hope you've already visited some of PGXC sites for scaling. If you
haven't, please try to visit
http://sourceforge.net/p/postgres-xc/xc-wiki/Presentation_Materials/
and other materials which describes how PGXC scales both read and
write.
Regards;
---
Koichi Suzuki
2014-11-24 19:21 GMT+09:00 Mario Splivalo <mario@splivalo.hr>:
> On 11/24/2014 10:30 AM, fahad basheer wrote:
>> Hi All,
>>
>> Thanks for the fast response
>>
>> Database is used as OLTP database.
>>
>> Frontend application used is OpenERP.
>>
>> I have tried postgresql xc,but its not supported with OpenERP.
>>
>>
>
> I don't see why OpenERP wouldn't work with pg-xc, but I'm also not sure
> if you would benefit from it as deciding on what to partition your data
> on is not a simple task.
>
> What is your main performance issue?
>
> In theory, if you want to scale writes you need to scale your data. So,
> you'd, for instance, put all the even IDs on server A and all the odd
> IDs on server B. That way you double the writing throughput. But, when
> querying your data you need to somehow 'merge' both partitions, the one
> on server A and the one on server B.
> This is, actually, where Postgres-XC comes in - it will allow you to
> create partitions of your data in a way that is oblivious to the
> application itself. But for OpenERP I'm not sure if you will gain
> anything, and your setup will be quite complicated.
>
> You either need better server (with better I/O performance - add more
> spindles if you're on RAID10, or switch to ssd), or partition your data.
> If you have multiple tenants in your OpenERP maybe you could move each
> of them to their separate servers?
>
> Mario
> --
> Mario Splivalo
> mario@splivalo.hr
>
> "I can do it quick, I can do it cheap, I can do it well. Pick any two."
>
>
> --
> Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-cluster-hackers
--
Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-cluster-hackers
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2014-11-27 01:40 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
2014-11-21 11:38 ` Mario Splivalo <mario@splivalo.hr>
2014-11-21 11:56 ` Sergei Sheinin <sergei.sheinin@gmail.com>
2014-11-21 12:15 ` desmodemone <desmodemone@gmail.com>
2014-11-24 00:28 ` Koichi Suzuki <koichi.dbms@gmail.com>
2014-11-24 09:30 ` fahad basheer <fahad.a.basheer@gmail.com>
2014-11-24 10:21 ` Mario Splivalo <mario@splivalo.hr>
2014-11-27 01:40 ` Koichi Suzuki <koichi.dbms@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox