pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedNeed assistance for replication sync issue
7+ messages / 6 participants
[nested] [flat]
* Need assistance for replication sync issue
@ 2023-10-19 10:08 Veerendra Pulapa <veerendra.pulapa@ashnik.com>
0 siblings, 3 replies; 7+ messages in thread
From: Veerendra Pulapa @ 2023-10-19 10:08 UTC (permalink / raw)
To: pgsql-admin
Hello,
I need some help with a database issue we’re facing. We have a situation where our primary data center and our disaster recovery site are far apart, and this long distance is causing delays in keeping our data synchronized. It’s become a problem as we want our data to be as up-to-date as possible.
Do you have any suggestions for dealing with this kind of situation? We’ve tried a few things, but we’re still struggling with the replication delays caused by the high latency between the sites.
Note: Bandwidth is very low and latency is very high.
I’d appreciate any advice you can provide.
Thanks in advance.
Get Outlook for iOS<https://aka.ms/o0ukef;
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Need assistance for replication sync issue
@ 2023-10-19 10:38 Licio Matos <licio.matos@gmail.com>
parent: Veerendra Pulapa <veerendra.pulapa@ashnik.com>
2 siblings, 1 reply; 7+ messages in thread
From: Licio Matos @ 2023-10-19 10:38 UTC (permalink / raw)
To: Veerendra Pulapa <veerendra.pulapa@ashnik.com>; +Cc: pgsql-admin
Hi Veerendra,
Is your servers, primary and secondary, the same size? CPU, Memory etc.
Licio Matos
Em qui., 19 de out. de 2023 às 07:12, Veerendra Pulapa <
veerendra.pulapa@ashnik.com> escreveu:
> Hello,
>
> I need some help with a database issue we’re facing. We have a situation
> where our primary data center and our disaster recovery site are far apart,
> and this long distance is causing delays in keeping our data synchronized.
> It’s become a problem as we want our data to be as up-to-date as possible.
>
> Do you have any suggestions for dealing with this kind of situation? We’ve
> tried a few things, but we’re still struggling with the replication delays
> caused by the high latency between the sites.
>
> Note: Bandwidth is very low and latency is very high.
>
> I’d appreciate any advice you can provide.
>
> Thanks in advance.
>
> Get Outlook for iOS <https://aka.ms/o0ukef;
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Need assistance for replication sync issue
@ 2023-10-19 10:55 Laurenz Albe <laurenz.albe@cybertec.at>
parent: Veerendra Pulapa <veerendra.pulapa@ashnik.com>
2 siblings, 0 replies; 7+ messages in thread
From: Laurenz Albe @ 2023-10-19 10:55 UTC (permalink / raw)
To: Veerendra Pulapa <veerendra.pulapa@ashnik.com>; pgsql-admin
On Thu, 2023-10-19 at 10:08 +0000, Veerendra Pulapa wrote:
> I need some help with a database issue we’re facing. We have a situation where
> our primary data center and our disaster recovery site are far apart, and this
> long distance is causing delays in keeping our data synchronized. It’s become
> a problem as we want our data to be as up-to-date as possible.
>
> Do you have any suggestions for dealing with this kind of situation?
Yes: don't use synchronous replication.
You are fighting with physics (speed of light), not with an implementation
deficiency. You cannot win that fight.
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Need assistance for replication sync issue
@ 2023-10-19 12:36 Ron <ronljohnsonjr@gmail.com>
parent: Veerendra Pulapa <veerendra.pulapa@ashnik.com>
2 siblings, 0 replies; 7+ messages in thread
From: Ron @ 2023-10-19 12:36 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
On 10/19/23 05:08, Veerendra Pulapa wrote:
> Hello,
>
> I need some help with a database issue we’re facing. We have a situation
> where our primary data center and our disaster recovery site are far
> apart, and this long distance is causing delays in keeping our data
> synchronized. It’s become a problem as we want our data to be as
> up-to-date as possible.
>
> Do you have any suggestions for dealing with this kind of situation? We’ve
> tried a few things, but we’re still struggling with the replication delays
> caused by the high latency between the sites.
>
> Note: Bandwidth is very low and latency is very high.
>
> I’d appreciate any advice you can provide.
/Never/ use /synchronous/ replication over a WAN. Not only because
"physics", as Laurenz mentions, but also because WAN links can be /slow/,
and break more often than LAN links.
Thus, use "async" replication instead.
--
Born in Arizona, moved to Babylonia.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Need assistance for replication sync issue
@ 2023-10-19 13:53 Jwiencek3 <jwiencek3@comcast.net>
parent: Licio Matos <licio.matos@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Jwiencek3 @ 2023-10-19 13:53 UTC (permalink / raw)
To: Licio Matos <licio.matos@gmail.com>; +Cc: Veerendra Pulapa <veerendra.pulapa@ashnik.com>; pgsql-admin
--Apple-Mail-C0AF5AFC-E6E4-4D53-AF38-D805816B1B77
Content-Type: text/html;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto">Look at using synchronous replication.<br><=
br><div dir=3D"ltr">Sent from my iPad</div><div dir=3D"ltr"><br><blockquote t=
ype=3D"cite">On Oct 19, 2023, at 5:38 AM, Licio Matos <licio.matos@gmail.=
com> wrote:<br><br></blockquote></div><blockquote type=3D"cite"><div dir=3D=
"ltr">=EF=BB=BF<div dir=3D"auto">Hi Veerendra,</div><div dir=3D"auto"><br></=
div><div dir=3D"auto">Is your servers, primary and secondary, the same size?=
CPU, Memory etc.</div><div dir=3D"auto"><br></div><div dir=3D"auto"><br cle=
ar=3D"all"><div dir=3D"auto"><div dir=3D"ltr" class=3D"gmail_signature" data=
-smartmail=3D"gmail_signature">Licio Matos</div></div></div><div><br></div><=
div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">Em q=
ui., 19 de out. de 2023 =C3=A0s 07:12, Veerendra Pulapa <<a href=3D"mailt=
o:veerendra.pulapa@ashnik.com">veerendra.pulapa@ashnik.com</a>> escreveu:=
<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde=
r-left:1px #ccc solid;padding-left:1ex">
<div>
<div dir=3D"ltr">
<div></div>
<div>
<div>Hello,
<div dir=3D"ltr"><br>
</div>
<div>I need some help with a database issue we=E2=80=99re facing. We have a s=
ituation where our primary data center and our disaster recovery site are fa=
r apart, and this long distance is causing delays in keeping our data synchr=
onized. It=E2=80=99s become a problem as we
want our data to be as up-to-date as possible.</div>
<div dir=3D"ltr"><br>
</div>
<div>Do you have any suggestions for dealing with this kind of situation? We=
=E2=80=99ve tried a few things, but we=E2=80=99re still struggling with the r=
eplication delays caused by the high latency between the sites.</div>
<div dir=3D"ltr"><br>
</div>
<div dir=3D"ltr">Note: Bandwidth is very low and latency is very high.</div>=
<div dir=3D"ltr"><br>
</div>
<div>I=E2=80=99d appreciate any advice you can provide.</div>
<div dir=3D"ltr"><br>
</div>
<div>Thanks in advance.</div>
</div>
<div id=3D"m_-39603247587684154ms-outlook-mobile-signature">
<div><br>
</div>
Get <a href=3D"https://aka.ms/o0ukef"; target=3D"_blank">Outlook for iOS</a><=
/div>
</div>
</div>
</div>
</blockquote></div></div>
</div></blockquote></body></html>=
--Apple-Mail-C0AF5AFC-E6E4-4D53-AF38-D805816B1B77--
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Need assistance for replication sync issue
@ 2023-10-20 04:10 Veerendra Pulapa <veerendra.pulapa@ashnik.com>
parent: Jwiencek3 <jwiencek3@comcast.net>
0 siblings, 1 reply; 7+ messages in thread
From: Veerendra Pulapa @ 2023-10-20 04:10 UTC (permalink / raw)
To: Jwiencek3 <jwiencek3@comcast.net>; laurenz.albe@cybertec.at, ronljohnsonjr@gmail.com, Licio Matos <licio.matos@gmail.com>; pgsql-admin
Dears,
Thank you all for your response. Below are the general details about our
setup.
Our setup is DC & DR.
On DC: Three nodes(one master, Two replicas) and replication mode is
asynchronous between DC.
On DR: One replica and replication mode is asynchronous between DC & DR.
All four nodes have the same configuration such as CPU,RAM and Disks.
In DC no replication & in DR yes it's always replication lag due to this
our replication got broken.
The setup runs on on-prem RHEL 7.9 and PG 12.x.
Is there a way to perform an incremental data restore if replication breaks
at a certain point? The data size is substantial, and when replication
breaks between the DC and DR, it takes 3 to 4 days to rebuild the replica.
Br,
Veerendra Pulapa | Technical Consultant
M: +91-9949349894 | www.ashnik.com
<https://www.ashnik.com/;
<https://www.linkedin.com/company/ashnik-pte-ltd/;
<https://www.facebook.com/AshnikBiz;
<https://www.youtube.com/user/ashnikbiz;
<https://www.instagram.com/ashnikbiz/; <https://twitter.com/Ashnikbiz;
On Thu, Oct 19, 2023 at 7:23 PM Jwiencek3 <jwiencek3@comcast.net> wrote:
> Look at using synchronous replication.
>
> Sent from my iPad
>
> On Oct 19, 2023, at 5:38 AM, Licio Matos <licio.matos@gmail.com> wrote:
>
>
> Hi Veerendra,
>
> Is your servers, primary and secondary, the same size? CPU, Memory etc.
>
>
> Licio Matos
>
>
> Em qui., 19 de out. de 2023 às 07:12, Veerendra Pulapa <
> veerendra.pulapa@ashnik.com> escreveu:
>
>> Hello,
>>
>> I need some help with a database issue we’re facing. We have a situation
>> where our primary data center and our disaster recovery site are far apart,
>> and this long distance is causing delays in keeping our data synchronized.
>> It’s become a problem as we want our data to be as up-to-date as possible.
>>
>> Do you have any suggestions for dealing with this kind of situation?
>> We’ve tried a few things, but we’re still struggling with the replication
>> delays caused by the high latency between the sites.
>>
>> Note: Bandwidth is very low and latency is very high.
>>
>> I’d appreciate any advice you can provide.
>>
>> Thanks in advance.
>>
>> Get Outlook for iOS <https://aka.ms/o0ukef;
>>
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Need assistance for replication sync issue
@ 2023-10-23 11:58 Tomek <tomekphotos@gmail.com>
parent: Veerendra Pulapa <veerendra.pulapa@ashnik.com>
0 siblings, 0 replies; 7+ messages in thread
From: Tomek @ 2023-10-23 11:58 UTC (permalink / raw)
To: Veerendra Pulapa <veerendra.pulapa@ashnik.com>; +Cc: pgsql-admin
As understanded you have DR for backup purposes only.
I think you should not use stream replication to the DR - use only sending
of WALs. Definitely you will have lag but it is better to have a lag than
break and a need to make pg_dumpall from the beginning.
pt., 20 paź 2023 o 06:17 Veerendra Pulapa <veerendra.pulapa@ashnik.com>
napisał(a):
> Dears,
>
> Thank you all for your response. Below are the general details about our
> setup.
>
> Our setup is DC & DR.
> On DC: Three nodes(one master, Two replicas) and replication mode is
> asynchronous between DC.
> On DR: One replica and replication mode is asynchronous between DC & DR.
> All four nodes have the same configuration such as CPU,RAM and Disks.
> In DC no replication & in DR yes it's always replication lag due to this
> our replication got broken.
> The setup runs on on-prem RHEL 7.9 and PG 12.x.
>
> Is there a way to perform an incremental data restore if replication
> breaks at a certain point? The data size is substantial, and when
> replication breaks between the DC and DR, it takes 3 to 4 days to rebuild
> the replica.
>
> Br,
> Veerendra Pulapa | Technical Consultant
> M: +91-9949349894 | www.ashnik.com
>
> <https://www.ashnik.com/;
>
>
> <https://www.linkedin.com/company/ashnik-pte-ltd/;
> <https://www.facebook.com/AshnikBiz;
> <https://www.youtube.com/user/ashnikbiz;
> <https://www.instagram.com/ashnikbiz/; <https://twitter.com/Ashnikbiz;
>
>
> On Thu, Oct 19, 2023 at 7:23 PM Jwiencek3 <jwiencek3@comcast.net> wrote:
>
>> Look at using synchronous replication.
>>
>> Sent from my iPad
>>
>> On Oct 19, 2023, at 5:38 AM, Licio Matos <licio.matos@gmail.com> wrote:
>>
>>
>> Hi Veerendra,
>>
>> Is your servers, primary and secondary, the same size? CPU, Memory etc.
>>
>>
>> Licio Matos
>>
>>
>> Em qui., 19 de out. de 2023 às 07:12, Veerendra Pulapa <
>> veerendra.pulapa@ashnik.com> escreveu:
>>
>>> Hello,
>>>
>>> I need some help with a database issue we’re facing. We have a situation
>>> where our primary data center and our disaster recovery site are far apart,
>>> and this long distance is causing delays in keeping our data synchronized.
>>> It’s become a problem as we want our data to be as up-to-date as possible.
>>>
>>> Do you have any suggestions for dealing with this kind of situation?
>>> We’ve tried a few things, but we’re still struggling with the replication
>>> delays caused by the high latency between the sites.
>>>
>>> Note: Bandwidth is very low and latency is very high.
>>>
>>> I’d appreciate any advice you can provide.
>>>
>>> Thanks in advance.
>>>
>>> Get Outlook for iOS <https://aka.ms/o0ukef;
>>>
>>
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2023-10-23 11:58 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-10-19 10:08 Need assistance for replication sync issue Veerendra Pulapa <veerendra.pulapa@ashnik.com>
2023-10-19 10:38 ` Licio Matos <licio.matos@gmail.com>
2023-10-19 13:53 ` Jwiencek3 <jwiencek3@comcast.net>
2023-10-20 04:10 ` Veerendra Pulapa <veerendra.pulapa@ashnik.com>
2023-10-23 11:58 ` Tomek <tomekphotos@gmail.com>
2023-10-19 10:55 ` Laurenz Albe <laurenz.albe@cybertec.at>
2023-10-19 12:36 ` Ron <ronljohnsonjr@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox