pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedOptimize streaming replication because of network latency
3+ messages / 2 participants
[nested] [flat]
* Optimize streaming replication because of network latency
@ 2020-12-10 17:41 Mai Peng <maily.peng@webedia-group.com>
0 siblings, 1 reply; 3+ messages in thread
From: Mai Peng @ 2020-12-10 17:41 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Hello,
We’ve added a streaming standby on a gcp container. There’s a lot of network latency between our primary in EU and this standby in South America.
I did a big update on a table ( 4 millions rows) and it generates a lot of lag : more than 5 hours.
The standby has great hardware ressources: cpu, ssd disk and had no load during this update.
How could we handle this lag and have less impact, less lag ?
Basic PG Config:
shared_buffers = 8GB
work_mem = 128MB
max_connections = 600
wal_keep_segments = 1000
wal_sender_timeout = 0
replication_timeout=(not set)
wal_receiver_status_interval=10s
max_wal_senders=20
Checkpoint_timeout = 10min
Max_wal_size= 2GB
min_wal_size= 1GB
Thank you
Mai=
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Optimize streaming replication because of network latency
@ 2020-12-10 18:00 Ron <ronljohnsonjr@gmail.com>
parent: Mai Peng <maily.peng@webedia-group.com>
0 siblings, 1 reply; 3+ messages in thread
From: Ron @ 2020-12-10 18:00 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
On 12/10/20 11:41 AM, Mai Peng wrote:
> Hello,
>
> We’ve added a streaming standby on a gcp container. There’s a lot of
> network latency between our primary in EU and this standby in South America.
> I did a big update on a table ( 4 millions rows) and it generates a lot of
> lag : more than 5 hours.
> The standby has great hardware ressources: cpu, ssd disk and had no load
> during this update.
> How could we handle this lag and have less impact, less lag ?
How effectively fast is the pipe between the two servers? Where are the
bottlenecks? Is the pipe shared by other users?
> Basic PG Config:
> shared_buffers = 8GB
> work_mem = 128MB
> max_connections = 600
> wal_keep_segments = 1000
> wal_sender_timeout = 0
> replication_timeout=(not set)
> wal_receiver_status_interval=10s
> max_wal_senders=20
> Checkpoint_timeout = 10min
> Max_wal_size= 2GB
> min_wal_size= 1GB
>
>
> Thank you
> Mai
--
Angular momentum makes the world go 'round.
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Optimize streaming replication because of network latency
@ 2020-12-11 09:51 Mai Peng <maily.peng@webedia-group.com>
parent: Ron <ronljohnsonjr@gmail.com>
0 siblings, 0 replies; 3+ messages in thread
From: Mai Peng @ 2020-12-11 09:51 UTC (permalink / raw)
To: Ron <ronljohnsonjr@gmail.com>; +Cc: pgsql-admin@lists.postgresql.org
Hello Ron,
We have 200 ms of latency between primary and standby.
The interconnexion is a VPN and the connexion is shared by several type of data flow : backup, application, replication, database servers.
Thanks for your help
> Le 10 déc. 2020 à 19:00, Ron <ronljohnsonjr@gmail.com> a écrit :
>
> On 12/10/20 11:41 AM, Mai Peng wrote:
>> Hello,
>>
>> We’ve added a streaming standby on a gcp container. There’s a lot of network latency between our primary in EU and this standby in South America.
>> I did a big update on a table ( 4 millions rows) and it generates a lot of lag : more than 5 hours.
>> The standby has great hardware ressources: cpu, ssd disk and had no load during this update.
>> How could we handle this lag and have less impact, less lag ?
>
> How effectively fast is the pipe between the two servers? Where are the bottlenecks? Is the pipe shared by other users?
>
>> Basic PG Config:
>> shared_buffers = 8GB
>> work_mem = 128MB
>> max_connections = 600
>> wal_keep_segments = 1000
>> wal_sender_timeout = 0
>> replication_timeout=(not set)
>> wal_receiver_status_interval=10s
>> max_wal_senders=20
>> Checkpoint_timeout = 10min
>> Max_wal_size= 2GB
>> min_wal_size= 1GB
>>
>>
>> Thank you
>> Mai
>
> --
> Angular momentum makes the world go 'round.
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2020-12-11 09:51 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-12-10 17:41 Optimize streaming replication because of network latency Mai Peng <maily.peng@webedia-group.com>
2020-12-10 18:00 ` Ron <ronljohnsonjr@gmail.com>
2020-12-11 09:51 ` Mai Peng <maily.peng@webedia-group.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox