Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1vEPVU-00BNRg-Nb for pgsql-admin@arkaria.postgresql.org; Thu, 30 Oct 2025 10:03:48 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1vEPVS-007A5o-93 for pgsql-admin@arkaria.postgresql.org; Thu, 30 Oct 2025 10:03:45 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1vEPVR-007A5c-SV for pgsql-admin@lists.postgresql.org; Thu, 30 Oct 2025 10:03:44 +0000 Received: from mail-ej1-x631.google.com ([2a00:1450:4864:20::631]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1vEPVP-004Xhz-05 for pgsql-admin@lists.postgresql.org; Thu, 30 Oct 2025 10:03:43 +0000 Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-b3e9d633b78so218111666b.1 for ; Thu, 30 Oct 2025 03:03:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec.at; s=google; t=1761818621; x=1762423421; darn=lists.postgresql.org; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:from:to:cc:subject:date :message-id:reply-to; bh=iTkTTZDn2oeIafelH80JRUvvXwmXc8vSnMhlyOTc5lE=; b=fIXs3421EBWSUjMGtsNImuI3UgLrCvhEcMxF04WFJ+nGfl17LXo6ACPI5LWKJ6Hy2o agKWaeRjRCtPfONyYWGN56waci6aWt63ZpkZ7+U+OKF1Rr4zuSO56SmcOHszHHNr/Pap 31W0iNINk8hzJdpKbgh8zIWCJPAXvbSH/TN0ks1tepf2WWHrPQAx+cqv7XCdPxRVZcnH eB0m952ot96Bvh9IqMPJG95Cs3kq1uMZ7w5V4Pc2CzK0EqdzEbBPEFxGKmEFbxNPx54T g1RnzKXwGo5qFrq+RdLmrYrdcJZiBjnqJKfWPDy1q0Tytcu5ByiDg8Juo1CUyV8xYPt4 qBGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761818621; x=1762423421; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=iTkTTZDn2oeIafelH80JRUvvXwmXc8vSnMhlyOTc5lE=; b=wKdqwLut2OTVwj9FBs2Lol6sGUBUk599CbOkBRaXReBssGA0oSHN46Lnjv6mfFS6rM Sq0BPqt4iKcnXWZHw2Jc1EeoM5R9VZ/xJwkHbyjXbD/eK2bU4C9zV2YlLKSc67E5AR8I KdzW1rUAap8bKBC+b0W9g1egxMXGnULwzLUCwNXmNlfvsJJzNkPIo/La4GV/aDq4VWR/ BXgMCGWoWKgH1woT0O7K8s+MnbeYyhuRlMikil7Sv6JQNrjEx9pw75/IZ6S+dnKmynRL 79zNgtpLnkj9YzqSHc5TvB8BlaoViXNVR8Eqha0EQJgI5+QsX5ogHPjr+kP2WsEwDunX OG/A== X-Forwarded-Encrypted: i=1; AJvYcCV0MUT8sBhz5eErEJ/QeK4tU0hvcVJ0oi2cDQ59T3R47MK1ynajkPqB9OUaJpbckicwZjV59dWRdwjJ/A==@lists.postgresql.org X-Gm-Message-State: AOJu0Yxa2MKQTpEfWFH2q8/5sVAIGs0HHLZ3E9briZ+ZWGKxhgNmNvM1 fcRGTggskszYEKP09IaAU4LAjvPGjt0wr9LX+OwLirzg5oyOipdtrWfLMvmfB2ycIHw= X-Gm-Gg: ASbGnctiXIjWdztat0Y98UT4ahfaHT2AkRIwg7m3keB+qlc+CLbY5/GT6DGQLinXbxG DScXLhTRIDa8Xm+tbS7exN+7YKtXEuplmWTVrJFhi2DqPe3+o5LAIS6ZhS3oqgj1lZODN7rr/ex j9i4zy/G+UN7VjXi86OFFqTiLiJQUVx82v3ci0Hkvwy/KPTTFDg6kzY0dmTiCB8jjF9ez/ciE5N y6ukT+zx2eJsOSPDWgMxDEvlVwo7h7uO6yGMmwakCHaD1XJ+PLm75wXYHYw0BL9+p9j8Of8/WZE 0fFbUqivamwvJy/XWFlr21tvHE6kKoodMvBhyGZRgftgoMVBiFpmXC7e7gFdElbDa27KkiOGrdW 3LhDhIpyEs5gAVm29cM26kriXUmfQDqQAfAeUd7Ni+Yw34l5nOoP0rDL0T4onIqjDujwru87jRB sJD7tWiXnlYnbrdoUTJQVV4jU3TlC9 X-Google-Smtp-Source: AGHT+IE39NU6rkDZvsqWTpYCS6MtkxlxEwaR7tWPxQ4iSv6OnPVsLaueIV5KaBQINiVSVcXBtFhxMA== X-Received: by 2002:a17:907:72c4:b0:b6d:4d55:d1a8 with SMTP id a640c23a62f3a-b7051f6309fmr282262466b.8.1761818621488; Thu, 30 Oct 2025 03:03:41 -0700 (PDT) Received: from laurenz.albe-K4N0CV00F97414D ([2001:871:260:3448:be1a:44:7171:3d57]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-63ecf3686c5sm8318453a12.36.2025.10.30.03.03.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Oct 2025 03:03:41 -0700 (PDT) Message-ID: <1e134f4d86881e35dad2147654d49735044eb40a.camel@cybertec.at> Subject: Re: WAL replay is too slow on secondary server From: Laurenz Albe To: OMPRAKASH SAHU , pgsql-admin@lists.postgresql.org Date: Thu, 30 Oct 2025 11:03:40 +0100 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.56.2 (3.56.2-2.fc42) MIME-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Thu, 2025-10-30 at 10:06 +0530, OMPRAKASH SAHU wrote: > We have a postgresql cluster setup using patroni. > The DB is being used for heavy transactional application, now the problem= =C2=A0is that on replica server WAL replay is too slow. > We have increased the IOPS to 6k and Throughput to 600 on nvme EBS volume= of wal directory and 10k &800 on data directory. >=20 > but the WAL is being accumulated on the replica as usual and applying wal= is having no improvement. > changed=C2=A0the maintenance_io_concurrency on replica to 32. > CPU utilization max=3D20% , RAM utilization is also max 20. Is the disk saturated? > I would request your thoughts and suggestions if we can get rid of this s= lowness and get some speed. WAL replay during streaming replication is single-threaded. So you can only use a faster CPU or disk, depending on what is the bottlene= ck. Perhaps your shared buffers are too small, and you have cache contention. Yours, Laurenz Albe