Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jaLjo-0008MM-Pk for pgsql-bugs@arkaria.postgresql.org; Sun, 17 May 2020 16:02:04 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jaLjm-0005Px-BH for pgsql-bugs@arkaria.postgresql.org; Sun, 17 May 2020 16:02:02 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jaLjm-0005Pl-5M for pgsql-bugs@lists.postgresql.org; Sun, 17 May 2020 16:02:02 +0000 Received: from excelsior.roeckx.be ([2a05:7300:0:100::3]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jaLjf-0008MG-PH for pgsql-bugs@lists.postgresql.org; Sun, 17 May 2020 16:02:01 +0000 Received: from intrepid.roeckx.be (localhost [127.0.0.1]) by excelsior.roeckx.be (Postfix) with ESMTP id 8AA59A8A0108; Sun, 17 May 2020 16:01:50 +0000 (UTC) Received: by intrepid.roeckx.be (Postfix, from userid 1000) id 556911FE0CD8; Sun, 17 May 2020 18:01:50 +0200 (CEST) Date: Sun, 17 May 2020 18:01:50 +0200 From: Kurt Roeckx To: Tom Lane Cc: pgsql-bugs@lists.postgresql.org Subject: Re: BUG #16443: Too much memory usage on insert query Message-ID: <20200517160150.GJ2915@roeckx.be> References: <16443-030629aae2859708@postgresql.org> <1360.1589730053@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1360.1589730053@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Sun, May 17, 2020 at 11:40:53AM -0400, Tom Lane wrote: > PG Bug reporting form writes: > > I was executing this query: > > insert into ct_entry_chain (ct_entry_id, certificate_id) select id, > > unnest(certificate_chain_id) from ct_entry; > > How much data is that? The ct_entry table contains 305 GB data over about 2.1e9 rows. certificate_chain_id contains about 2.2 entries per row. > > The process was using at least 14 GB, of the 8 GB of RAM that's available. > > My first guess is that the space was being eaten by trigger list entries > to verify the foreign-key constraints on the target table. You might be > better advised to fill the new table first and then create its FK > constraints. (Building the indexes afterwards wouldn't be a bad idea, > either.) > > We are looking at better mechanisms for handling FK verification, but > that won't see the light of day before v14 at the earliest. I will try that, thanks. Kurt