pg.ddx.io  pgsql-cluster-hackers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Postgresql Write cluster needed
8+ messages / 5 participants
[nested] [flat]

* Postgresql Write cluster needed
@ 2014-11-21 11:29 fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-21 11:38 ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
  0 siblings, 1 reply; 8+ messages in thread

From: fahad basheer @ 2014-11-21 11:29 UTC (permalink / raw)
  To: pgsql-cluster-hackers

Hi ALL,

I need to scale postgresql with write capabilities, please help me.

Regards
Fahad

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Postgresql Write cluster needed
  2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
@ 2014-11-21 11:38 ` Mario Splivalo <mario@splivalo.hr>
  2014-11-21 11:56   ` Re: Postgresql Write cluster needed Sergei Sheinin <sergei.sheinin@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Mario Splivalo @ 2014-11-21 11:38 UTC (permalink / raw)
  To: fahad basheer <fahad.a.basheer@gmail.com>; pgsql-cluster-hackers

Unless you shard (partition) your data, writes can't be scaled.

Mario

On November 21, 2014 12:29:38 PM CET, fahad basheer <fahad.a.basheer@gmail.com> wrote:
>Hi ALL,
>
>I need to scale postgresql with write capabilities, please help me.
>
>Regards
>Fahad

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Postgresql Write cluster needed
  2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-21 11:38 ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
@ 2014-11-21 11:56   ` Sergei Sheinin <sergei.sheinin@gmail.com>
  2014-11-21 12:15     ` Re: Postgresql Write cluster needed desmodemone <desmodemone@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Sergei Sheinin @ 2014-11-21 11:56 UTC (permalink / raw)
  To: Mario Splivalo <mario@splivalo.hr>; +Cc: fahad basheer <fahad.a.basheer@gmail.com>; pgsql-cluster-hackers

It may be possible to write to text files then use COPY command for bulk
inserts.

On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:

> Unless you shard (partition) your data, writes can't be scaled.
>
> Mario
>
>
> On November 21, 2014 12:29:38 PM CET, fahad basheer <
> fahad.a.basheer@gmail.com> wrote:
>>
>> Hi ALL,
>>
>> I need to scale postgresql with write capabilities, please help me.
>>
>> Regards
>> Fahad
>>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Postgresql Write cluster needed
  2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-21 11:38 ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
  2014-11-21 11:56   ` Re: Postgresql Write cluster needed Sergei Sheinin <sergei.sheinin@gmail.com>
@ 2014-11-21 12:15     ` desmodemone <desmodemone@gmail.com>
  2014-11-24 00:28       ` Re: Postgresql Write cluster needed Koichi Suzuki <koichi.dbms@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: desmodemone @ 2014-11-21 12:15 UTC (permalink / raw)
  To: Sergei Sheinin <sergei.sheinin@gmail.com>; +Cc: Mario Splivalo <mario@splivalo.hr>; fahad basheer <fahad.a.basheer@gmail.com>; pgsql-cluster-hackers

2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:

> It may be possible to write to text files then use COPY command for bulk
> inserts.
>
> On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
>
>> Unless you shard (partition) your data, writes can't be scaled.
>>
>> Mario
>>
>>
>> On November 21, 2014 12:29:38 PM CET, fahad basheer <
>> fahad.a.basheer@gmail.com> wrote:
>>>
>>> Hi ALL,
>>>
>>> I need to scale postgresql with write capabilities, please help me.
>>>
>>> Regards
>>> Fahad
>>>
>>
>> --
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>
>

Hi,
         I think we need more informations about this problem and how it's
the enviroment (OLTP, dwh ? ) . For example if some tables not need to be
full ACID you could use  unlogged table or  for example you could relax the
durability of the commit , disabling synchronous commit  .

Regards


-- 
Matteo Durighetto

- - - - - - - - - - - - - - - - - - - - - - -

Italian PostgreSQL User Group <http://www.itpug.org/index.it.html;
Italian Community for Geographic Free/Open-Source Software
<http://www.gfoss.it;

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Postgresql Write cluster needed
  2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-21 11:38 ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
  2014-11-21 11:56   ` Re: Postgresql Write cluster needed Sergei Sheinin <sergei.sheinin@gmail.com>
  2014-11-21 12:15     ` Re: Postgresql Write cluster needed desmodemone <desmodemone@gmail.com>
@ 2014-11-24 00:28       ` Koichi Suzuki <koichi.dbms@gmail.com>
  2014-11-24 09:30         ` Re: Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Koichi Suzuki @ 2014-11-24 00:28 UTC (permalink / raw)
  To: desmodemone <desmodemone@gmail.com>; +Cc: Sergei Sheinin <sergei.sheinin@gmail.com>; Mario Splivalo <mario@splivalo.hr>; fahad basheer <fahad.a.basheer@gmail.com>; pgsql-cluster-hackers

Write scalability for OLTP workload is available with Postgres-XC.
---
Koichi Suzuki


2014-11-21 21:15 GMT+09:00 desmodemone <desmodemone@gmail.com>:
>
>
> 2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:
>>
>> It may be possible to write to text files then use COPY command for bulk
>> inserts.
>>
>> On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr> wrote:
>>>
>>> Unless you shard (partition) your data, writes can't be scaled.
>>>
>>> Mario
>>>
>>>
>>> On November 21, 2014 12:29:38 PM CET, fahad basheer
>>> <fahad.a.basheer@gmail.com> wrote:
>>>>
>>>> Hi ALL,
>>>>
>>>> I need to scale postgresql with write capabilities, please help me.
>>>>
>>>> Regards
>>>> Fahad
>>>
>>>
>>> --
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>>
>
>
> Hi,
>          I think we need more informations about this problem and how it's
> the enviroment (OLTP, dwh ? ) . For example if some tables not need to be
> full ACID you could use  unlogged table or  for example you could relax the
> durability of the commit , disabling synchronous commit  .
>
> Regards
>
>
> --
> Matteo Durighetto
>
> - - - - - - - - - - - - - - - - - - - - - - -
>
> Italian PostgreSQL User Group
> Italian Community for Geographic Free/Open-Source Software


-- 
Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-cluster-hackers



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Postgresql Write cluster needed
  2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-21 11:38 ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
  2014-11-21 11:56   ` Re: Postgresql Write cluster needed Sergei Sheinin <sergei.sheinin@gmail.com>
  2014-11-21 12:15     ` Re: Postgresql Write cluster needed desmodemone <desmodemone@gmail.com>
  2014-11-24 00:28       ` Re: Postgresql Write cluster needed Koichi Suzuki <koichi.dbms@gmail.com>
@ 2014-11-24 09:30         ` fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-24 10:21           ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
  0 siblings, 1 reply; 8+ messages in thread

From: fahad basheer @ 2014-11-24 09:30 UTC (permalink / raw)
  To: Koichi Suzuki <koichi.dbms@gmail.com>; +Cc: desmodemone <desmodemone@gmail.com>; Sergei Sheinin <sergei.sheinin@gmail.com>; Mario Splivalo <mario@splivalo.hr>; pgsql-cluster-hackers

Hi All,

Thanks for the fast response

Database is used as OLTP database.

Frontend application used is  OpenERP.

I have tried postgresql xc,but its not supported with OpenERP.




On Mon, Nov 24, 2014 at 5:58 AM, Koichi Suzuki <koichi.dbms@gmail.com>
wrote:

> Write scalability for OLTP workload is available with Postgres-XC.
> ---
> Koichi Suzuki
>
>
> 2014-11-21 21:15 GMT+09:00 desmodemone <desmodemone@gmail.com>:
> >
> >
> > 2014-11-21 12:56 GMT+01:00 Sergei Sheinin <sergei.sheinin@gmail.com>:
> >>
> >> It may be possible to write to text files then use COPY command for bulk
> >> inserts.
> >>
> >> On Fri, Nov 21, 2014 at 6:38 PM, Mario Splivalo <mario@splivalo.hr>
> wrote:
> >>>
> >>> Unless you shard (partition) your data, writes can't be scaled.
> >>>
> >>> Mario
> >>>
> >>>
> >>> On November 21, 2014 12:29:38 PM CET, fahad basheer
> >>> <fahad.a.basheer@gmail.com> wrote:
> >>>>
> >>>> Hi ALL,
> >>>>
> >>>> I need to scale postgresql with write capabilities, please help me.
> >>>>
> >>>> Regards
> >>>> Fahad
> >>>
> >>>
> >>> --
> >>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> >>
> >>
> >
> >
> > Hi,
> >          I think we need more informations about this problem and how
> it's
> > the enviroment (OLTP, dwh ? ) . For example if some tables not need to be
> > full ACID you could use  unlogged table or  for example you could relax
> the
> > durability of the commit , disabling synchronous commit  .
> >
> > Regards
> >
> >
> > --
> > Matteo Durighetto
> >
> > - - - - - - - - - - - - - - - - - - - - - - -
> >
> > Italian PostgreSQL User Group
> > Italian Community for Geographic Free/Open-Source Software
>


Regards,
Fahad

^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Postgresql Write cluster needed
  2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-21 11:38 ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
  2014-11-21 11:56   ` Re: Postgresql Write cluster needed Sergei Sheinin <sergei.sheinin@gmail.com>
  2014-11-21 12:15     ` Re: Postgresql Write cluster needed desmodemone <desmodemone@gmail.com>
  2014-11-24 00:28       ` Re: Postgresql Write cluster needed Koichi Suzuki <koichi.dbms@gmail.com>
  2014-11-24 09:30         ` Re: Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
@ 2014-11-24 10:21           ` Mario Splivalo <mario@splivalo.hr>
  2014-11-27 01:40             ` Re: Postgresql Write cluster needed Koichi Suzuki <koichi.dbms@gmail.com>
  0 siblings, 1 reply; 8+ messages in thread

From: Mario Splivalo @ 2014-11-24 10:21 UTC (permalink / raw)
  To: pgsql-cluster-hackers

On 11/24/2014 10:30 AM, fahad basheer wrote:
> Hi All,
> 
> Thanks for the fast response
> 
> Database is used as OLTP database.
> 
> Frontend application used is  OpenERP.
> 
> I have tried postgresql xc,but its not supported with OpenERP.
> 
> 

I don't see why OpenERP wouldn't work with pg-xc, but I'm also not sure
if you would benefit from it as deciding on what to partition your data
on is not a simple task.

What is your main performance issue?

In theory, if you want to scale writes you need to scale your data. So,
you'd, for instance, put all the even IDs on server A and all the odd
IDs on server B. That way you double the writing throughput. But, when
querying your data you need to somehow 'merge' both partitions, the one
on server A and the one on server B.
This is, actually, where Postgres-XC comes in - it will allow you to
create partitions of your data in a way that is oblivious to the
application itself. But for OpenERP I'm not sure if you will gain
anything, and your setup will be quite complicated.

You either need better server (with better I/O performance - add more
spindles if you're on RAID10, or switch to ssd), or partition your data.
If you have multiple tenants in your OpenERP maybe you could move each
of them to their separate servers?

	Mario
-- 
Mario Splivalo
mario@splivalo.hr

"I can do it quick, I can do it cheap, I can do it well. Pick any two."


-- 
Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-cluster-hackers



^ permalink  raw  reply  [nested|flat] 8+ messages in thread

* Re: Postgresql Write cluster needed
  2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-21 11:38 ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
  2014-11-21 11:56   ` Re: Postgresql Write cluster needed Sergei Sheinin <sergei.sheinin@gmail.com>
  2014-11-21 12:15     ` Re: Postgresql Write cluster needed desmodemone <desmodemone@gmail.com>
  2014-11-24 00:28       ` Re: Postgresql Write cluster needed Koichi Suzuki <koichi.dbms@gmail.com>
  2014-11-24 09:30         ` Re: Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
  2014-11-24 10:21           ` Re: Postgresql Write cluster needed Mario Splivalo <mario@splivalo.hr>
@ 2014-11-27 01:40             ` Koichi Suzuki <koichi.dbms@gmail.com>
  0 siblings, 0 replies; 8+ messages in thread

From: Koichi Suzuki @ 2014-11-27 01:40 UTC (permalink / raw)
  To: Mario Splivalo <mario@splivalo.hr>; +Cc: pgsql-cluster-hackers

I hope you've already visited some of PGXC sites for scaling.   If you
haven't, please try to visit
http://sourceforge.net/p/postgres-xc/xc-wiki/Presentation_Materials/
and other materials which describes how PGXC scales both read and
write.

Regards;
---
Koichi Suzuki


2014-11-24 19:21 GMT+09:00 Mario Splivalo <mario@splivalo.hr>:
> On 11/24/2014 10:30 AM, fahad basheer wrote:
>> Hi All,
>>
>> Thanks for the fast response
>>
>> Database is used as OLTP database.
>>
>> Frontend application used is  OpenERP.
>>
>> I have tried postgresql xc,but its not supported with OpenERP.
>>
>>
>
> I don't see why OpenERP wouldn't work with pg-xc, but I'm also not sure
> if you would benefit from it as deciding on what to partition your data
> on is not a simple task.
>
> What is your main performance issue?
>
> In theory, if you want to scale writes you need to scale your data. So,
> you'd, for instance, put all the even IDs on server A and all the odd
> IDs on server B. That way you double the writing throughput. But, when
> querying your data you need to somehow 'merge' both partitions, the one
> on server A and the one on server B.
> This is, actually, where Postgres-XC comes in - it will allow you to
> create partitions of your data in a way that is oblivious to the
> application itself. But for OpenERP I'm not sure if you will gain
> anything, and your setup will be quite complicated.
>
> You either need better server (with better I/O performance - add more
> spindles if you're on RAID10, or switch to ssd), or partition your data.
> If you have multiple tenants in your OpenERP maybe you could move each
> of them to their separate servers?
>
>         Mario
> --
> Mario Splivalo
> mario@splivalo.hr
>
> "I can do it quick, I can do it cheap, I can do it well. Pick any two."
>
>
> --
> Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-cluster-hackers


-- 
Sent via pgsql-cluster-hackers mailing list (pgsql-cluster-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-cluster-hackers



^ permalink  raw  reply  [nested|flat] 8+ messages in thread


end of thread, other threads:[~2014-11-27 01:40 UTC | newest]

Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-11-21 11:29 Postgresql Write cluster needed fahad basheer <fahad.a.basheer@gmail.com>
2014-11-21 11:38 ` Mario Splivalo <mario@splivalo.hr>
2014-11-21 11:56   ` Sergei Sheinin <sergei.sheinin@gmail.com>
2014-11-21 12:15     ` desmodemone <desmodemone@gmail.com>
2014-11-24 00:28       ` Koichi Suzuki <koichi.dbms@gmail.com>
2014-11-24 09:30         ` fahad basheer <fahad.a.basheer@gmail.com>
2014-11-24 10:21           ` Mario Splivalo <mario@splivalo.hr>
2014-11-27 01:40             ` Koichi Suzuki <koichi.dbms@gmail.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox