Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oJxvr-0006m5-J2 for pgsql-performance@arkaria.postgresql.org; Fri, 05 Aug 2022 14:04:07 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1oJxvq-0006kd-GA for pgsql-performance@arkaria.postgresql.org; Fri, 05 Aug 2022 14:04:06 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oJxvq-0006hZ-1o for pgsql-performance@lists.postgresql.org; Fri, 05 Aug 2022 14:04:06 +0000 Received: from mail-qt1-x82c.google.com ([2607:f8b0:4864:20::82c]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1oJxvn-00026D-Fs for pgsql-performance@lists.postgresql.org; Fri, 05 Aug 2022 14:04:05 +0000 Received: by mail-qt1-x82c.google.com with SMTP id h22so2099568qta.3 for ; Fri, 05 Aug 2022 07:04:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20210112.gappssmtp.com; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc; bh=KPPOwbN6aICkfHdygEg+kJy3CESdM4Y82eIfmUQVPGA=; b=hfNz69UaCaosNgvCGTuwtpVWrQnLbF9KPgC/gmFdwlQNY+f5KkrABibJ0fjv5YSdTH oc60RzKXMhZhtjL3XTuCmmLU5MWOIMOc+qfCQrshPNamlxCEu4HDBqrDFqELOZdlfL6Z Oa6m2+i16DyQCB3TYbTpaXjTMk+rFYlwDdakUkEPzhM1udPYYfRVhw35CYS3/zFL21AS Nvvs8kyTb0gAvaDtkOXD5JBSaNMc49NQtKutOmdBuP7r4AaEdQbdziziS+Z+7frqtU1i gRgh5coC0m1Kn3Ws9MTMTtLlenCd7lxx8dp3wIQ705R7K2a9rVmQBhS20bmQKgUwQSTV rO6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc; bh=KPPOwbN6aICkfHdygEg+kJy3CESdM4Y82eIfmUQVPGA=; b=jUIVpKhizPhUEZ89VHNvGGjGJwpufQnxrGggYYOtGQuAHRtt2xCqXtlLgdFb2XPGV+ cGE6DjW4p+wd03PXfNkuRsZwWFEqC9ZIbbkJwY5ApZzk3+Z40oUWnjGPZpqm6WUA2Tys VWtwYqKyddvIu4h64A3grwjLIpdPI+nTkbIRK4ZM/Aj9NAhj7H84bkIWg2NiIX1mQF8Y Wk9wNUIH60W62ohZfZa1xqINwBz6IXn9qJU/M5mSpoIzUbCRXR2hbcp1UF9tAUpwIgyU hcQuAx/fLUQe4szegb2qv2MuEESMNKh95eonFE5wis9Mdz5yNUxKkeO4+w0lEduo2sRx RpTw== X-Gm-Message-State: ACgBeo3LLzpqqJXMM2kOh6EalDDjMHtk+GA2X5pzUl8I87KxzBYg8EI/ vMAd7gKQB9m/ztTfteOD9ll3ag== X-Google-Smtp-Source: AA6agR6xXMda73ZDhyMih6UyqfPgxyhcRX2NZQoiqcYNxqeHmgZ3pmL5YamXSAl6QfM+GPMDeBdYrg== X-Received: by 2002:ac8:5e0d:0:b0:31f:239c:99c6 with SMTP id h13-20020ac85e0d000000b0031f239c99c6mr5992791qtx.189.1659708241850; Fri, 05 Aug 2022 07:04:01 -0700 (PDT) Received: from pryzbyj.telsasoft (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id j10-20020a05620a410a00b006a65c58db99sm2997185qko.64.2022.08.05.07.04.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Aug 2022 07:04:01 -0700 (PDT) Received: by pryzbyj.telsasoft (Postfix, from userid 1000) id 82317800DE8; Fri, 5 Aug 2022 09:04:00 -0500 (CDT) Date: Fri, 5 Aug 2022 09:04:00 -0500 From: Justin Pryzby To: aditya desai Cc: pgsql-performance@lists.postgresql.org Subject: Re: pg_wal filling up while running huge updates Message-ID: <20220805140400.GQ19644@telsasoft.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Fri, Aug 05, 2022 at 06:00:02PM +0530, aditya desai wrote: > Hi, > We are doing an oracle to postgres migration(5 TB+ data). We are encoding > and decoding BLOB data after migration and for that we are running updates > on tables having BLOB/CLOB data. When we execute this pg_wal is filling up. Could you please include basic information in each new thread you create ? https://wiki.postgresql.org/wiki/Server_Configuration https://wiki.postgresql.org/wiki/Guide_to_reporting_problems -- Justin