Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id A4FD49FB1F0 for ; Mon, 30 Oct 2006 09:33:08 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 10815-10 for ; Mon, 30 Oct 2006 09:33:04 -0400 (AST) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.188]) by postgresql.org (Postfix) with ESMTP id 369079F9EB3 for ; Mon, 30 Oct 2006 09:33:02 -0400 (AST) Received: by nf-out-0910.google.com with SMTP id g2so2135996nfe for ; Mon, 30 Oct 2006 05:33:01 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=lJzLY75vMYKYK8VuN0DkeyY45wNGTS5CzvlIHnHwjqditYk7GhEPGN+8pxp1/vyPsORRgL788Oot5jaPMhR9wHuOPCBMli7qgTTd3SndcvR6O0DYQzlMPqky2Y0OO1g6R+s0Qt5wdqVAZR/aUlpKz0/CTLeC+RzBIuyNiABHe6Y= Received: by 10.82.136.4 with SMTP id j4mr420152bud; Mon, 30 Oct 2006 05:33:00 -0800 (PST) Received: by 10.82.129.11 with HTTP; Mon, 30 Oct 2006 05:33:00 -0800 (PST) Message-ID: Date: Mon, 30 Oct 2006 18:33:00 +0500 From: "Shoaib Mir" To: "Moiz Kothari" Subject: Re: 100% failover + replication solution Cc: "Ben Suffolk" , pgsql-admin@postgresql.org In-Reply-To: <82e1a9bd0610300511u35a3e156u22b3c600c69edd24@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_68739_12655727.1162215180930" References: <82e1a9bd0610300147m7e214893ue1306f1637bb489f@mail.gmail.com> <7CA68A94-ED1E-49C9-ACF2-2A34936333DC@vanilla.net> <82e1a9bd0610300253y1c999f31k5ba6ac086bcfb802@mail.gmail.com> <82e1a9bd0610300511u35a3e156u22b3c600c69edd24@mail.gmail.com> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200610/210 X-Sequence-Number: 23513 ------=_Part_68739_12655727.1162215180930 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Moiz, Have you tried PGPool? as that comes with a built-in load balancer as well. For PITR in HA scenario, I dont remember where I read but some one did it like this: - Make base backup for the primary server say five time a day (depends on the transactions happening on the db server) - Automate these base backups in a way that these backup are always made on the secondary server - Keep the archiving enabled with your archives been saved directly to a shared disk - Now have the recovery.conf placed in the $PGDATA of secondary server - When you want to switch servers just run the postmaster with recovery file and that way it will make it come up to date with the primary server - Once you have switched to seconday now mark it primary and the primary as seconday and keep on doing the same in a loop You can automate this easily with a few scripts. There is a new feature in 8.2 that let you set the archive_timeout so that after a specific amount of time a new WAL archive is made which makes it easy for low transaction systems where the WAL archive size is not reached that easily so it can be copied it to the archive folder. Hope this helps in your case.... Thank you, ---------- Shoaib Mir EnterpriseDB (www.enterprisedb.com) On 10/30/06, Moiz Kothari wrote: > > Shoaib, > > I agree that PGCluster might be a better option, i dont want to go with > Slony because of primary key constraints. But PGCluster is a good option, > the only concerns are : > > 1) It might slow down the process a bit. as confirmation happens after > transaction gets comitted to all the nodes. > 2) Its difficult to convince, as it is an external project and if support > for the same stops or future versions of postgres does not work, it might be > a problem. > > Can you elaborate more the way PITR for HA being used for primary and > secondary servers, maybe u can light a bulb in me for me to go ahead with > the approach. I like the idea of using WAL logs because its postgres > internal and secondly it would be fastest way of keeping databases in sync > without slowing down other servers. > > Awaiting your reply. > > Regards, > Moiz Kothari > > On 10/30/06, Shoaib Mir wrote: > > > > Hi Moiz, > > > > If I had to choose for your case where you want to direct your selects > > to slave node and inserts/updates on master, I would have opted for Slony or > > PGCluster. > > > > Using PITR for HA can be a good option if you want to switch between > > primary and secondary server, where you can store the archive files on a > > shared disk and place a recovery file with in $PGDATA and automate the > > process where it can run the process of recovery on each primary and > > seconday like for example 5 times a day as it all depends on the number of > > transactions happening on the db server. I have seen a few users doing this > > for routine VACUUM FULL process as a maintanence activity. > > > > Thanks, > > --------- > > Shoaib Mir > > EnterpriseDB (www.enterprisedb.com) > > > > On 10/30/06, Moiz Kothari < moizpostgres@gmail.com> wrote: > > > > > > Shoaib, > > > > > > It sure does, i saw PGCLUSTER, but 3 reasons for having a postgres > > > specific solution. > > > > > > 1) If pgcluster stops further development, it would be lot more hassel > > > when upgrading to a different version of postgres. > > > 2) Postgres specific solution would help alot going ahead in future. > > > 3) Also architecture of pgcluster might make things slower as it > > > updates complete cluster before confirming the request. > > > > > > There are lots of them available in market, but i think WAL solution > > > should be available, if not then the thought process should be there going > > > ahead. I am expecting a solution out of WAL logs. Let me know if you have > > > any thoughts about it. > > > > > > Regards, > > > Moiz Kothari > > > > > > On 10/30/06, Shoaib Mir < shoaibmir@gmail.com> wrote: > > > > > > > > There is this project which actually is not released yet, but > > > > something that you want to achieve :) > > > > > > > > http://pgfoundry.org/projects/pgpitrha > > > > > > > > Regards, > > > > ------- > > > > Shoaib Mir > > > > EnterpriseDB (www.enterprisedb.com) > > > > > > > > On 10/30/06, Ben Suffolk < ben@vanilla.net> wrote: > > > > > > > > > > Guys, > > > > > > > > > > > > I have been thinking about this and wanted to see if it can be > > > > > > achived. I wanted to make a 100% failover solution for my > > > > > postgres > > > > > > databases. The first thing that comes to my mind is doing it > > > > > using > > > > > > WAL logs. Am attaching the diagram for which i will write more > > > > > here. > > > > > > > > > > While its not the solution you were looking at, have you seen > > > > > PGCluser :- > > > > > > > > > > http://pgcluster.projects.postgresql.org/index.html > > > > > > > > > > I have not tried it, but was looking the other week at various > > > > > fail- > > > > > over type solutions and came across it. It seems to be able to do > > > > > what you want. > > > > > > > > > > Ben > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ---------------------------(end of > > > > > broadcast)--------------------------- > > > > > TIP 9: In versions below 8.0, the planner will ignore your desire > > > > > to > > > > > choose an index scan if your joining column's datatypes do > > > > > not > > > > > match > > > > > > > > > > > > > > > > > > > ------=_Part_68739_12655727.1162215180930 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Moiz,

Have you tried PGPool? as that comes with a built-in load balancer as well.

For PITR in HA scenario, I dont remember where I read but some one did it like this:

- Make base backup for the primary server say five time a day (depends on the transactions happening on the db server)
- Automate these base backups in a way that these backup are always made on the secondary server
- Keep the archiving enabled with your archives been saved directly to a shared disk
- Now have the recovery.conf placed in the $PGDATA of secondary server
- When you want to switch servers just run the postmaster with recovery file and that way it will make it come up to date with the primary server
- Once you have switched to seconday now mark it primary and the primary as seconday and keep on doing the same in a loop

You can automate this easily with a few scripts.

There is a new feature in 8.2 that let you set the archive_timeout so that after a specific amount of time a new WAL archive is made which makes it easy for low transaction systems where the WAL archive size is not reached that easily so it can be copied it to the archive folder.

Hope this helps in your case....

Thank you,
----------
Shoaib Mir
EnterpriseDB (www.enterprisedb.com)

On 10/30/06, Moiz Kothari <moizpostgres@gmail.com> wrote:
Shoaib,

I agree that PGCluster might be a better option, i dont want to go with Slony because of primary key constraints. But PGCluster is a good option, the only concerns are :

1) It might slow down the process a bit. as confirmation happens after transaction gets comitted to all the nodes.
2) Its difficult to convince, as it is an external project and if support for the same stops or future versions of postgres does not work, it might be a problem.

Can you elaborate more the way PITR for HA being used for primary and secondary servers, maybe u can light a bulb in me for me to go ahead with the approach. I like the idea of using WAL logs because its postgres internal and secondly it would be fastest way of keeping databases in sync without slowing down other servers.

Awaiting your reply.


Regards,
Moiz Kothari

On 10/30/06, Shoaib Mir < shoaibmir@gmail.com> wrote:
Hi Moiz,

If I had to choose for your case where you want to direct your selects to slave node and inserts/updates on master, I would have opted for Slony or PGCluster.

Using PITR for HA can be a good option if you want to switch between primary and secondary server, where you can store the archive files on a shared disk and place a recovery file with in $PGDATA and automate the process where it can run the process of recovery on each primary and seconday like for example 5 times a day as it all depends on the number of transactions happening on the db server. I have seen a few users doing this for routine VACUUM FULL process as a maintanence activity.

Thanks,
---------
Shoaib Mir
EnterpriseDB (www.enterprisedb.com)

On 10/30/06, Moiz Kothari < moizpostgres@gmail.com> wrote:
Shoaib,

It sure does, i saw PGCLUSTER, but 3 reasons for having a postgres specific solution.

1) If pgcluster stops further development, it would be lot more hassel when upgrading to a different version of postgres.
2) Postgres specific solution would help alot going ahead in future.
3) Also architecture of pgcluster might make things slower as it updates complete cluster before confirming the request.

There are lots of them available in market, but i think WAL solution should be available, if not then the thought process should be there going ahead. I am expecting a solution out of WAL logs. Let me know if you have any thoughts about it.

Regards,
Moiz Kothari


On 10/30/06, Shoaib Mir < shoaibmir@gmail.com> wrote:
There is this project which actually is not released yet, but something that you want to achieve :)

http://pgfoundry.org/projects/pgpitrha

Regards,
-------
Shoaib Mir
EnterpriseDB (www.enterprisedb.com)

On 10/30/06, Ben Suffolk < ben@vanilla.net> wrote:
> Guys,
>
> I have been thinking about this and wanted to see if it can be
> achived. I wanted to make a 100% failover solution for my postgres
> databases. The first thing that comes to my mind is doing it using
> WAL logs. Am attaching the diagram for which i will write more here.

While its not the solution you were looking at, have you seen
PGCluser :-

http://pgcluster.projects.postgresql.org/index.html

I have not tried it, but was looking the other week at various fail-
over type solutions and came across it. It seems to be able to do
what you want.

Ben






---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
       choose an index scan if your joining column's datatypes do not
       match





------=_Part_68739_12655727.1162215180930--