X-Original-To: pgsql-patches-postgresql.org@postgresql.org Received: from localhost (wm.hub.org [200.46.204.128]) by postgresql.org (Postfix) with ESMTP id 5A0E49FB256 for ; Wed, 2 Aug 2006 12:56:41 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.128]) (amavisd-new, port 10024) with ESMTP id 43069-08 for ; Wed, 2 Aug 2006 15:56:29 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey- Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by postgresql.org (Postfix) with ESMTP id C70849FABE7 for ; Wed, 2 Aug 2006 12:56:28 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.6/8.13.6) with ESMTP id k72FuN7H017705; Wed, 2 Aug 2006 11:56:23 -0400 (EDT) To: Joe Conway cc: pgsql-patches@postgreSQL.org Subject: Re: Values list-of-targetlists patch for comments (was Re: In-reply-to: <44D0B9C1.2020608@joeconway.com> References: <44B69778.5050104@dunslane.net> <50461.84.63.113.55.1152832991.squirrel@pezone.net> <44B7ADFC.90205@dunslane.net> <44D03D9F.80203@joeconway.com> <15263.1154526181@sss.pgh.pa.us> <44D0AF87.2030304@joeconway.com> <16892.1154528708@sss.pgh.pa.us> <44D0B9C1.2020608@joeconway.com> Comments: In-reply-to Joe Conway message dated "Wed, 02 Aug 2006 07:42:09 -0700" Date: Wed, 02 Aug 2006 11:56:23 -0400 Message-ID: <17704.1154534183@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=3.816 tagged_above=0 required=5 tests=SPF_HELO_SOFTFAIL, SPF_SOFTFAIL X-Spam-Level: *** X-Archive-Number: 200608/57 X-Sequence-Number: 20573 Joe Conway writes: > Tom Lane wrote: >> I wonder whether there is any reasonable way to determine which data >> structures are responsible for how much space ... in my test I'm seeing > I was doing it by sprinkling MemoryContextStats() in various places. Yeah, I've just been doing that and some hand analysis too. What I get (on a 64-bit machine) is that essentially all the space goes into lists of A_Const lists: 320000000 lists of Const lists: 320000000 transformInsertRow extra lists: 144000000 I think we could safely list_free the input list in transformInsertRow as your patch suggests, which would buy back the 144M part. But I don't believe it's safe at all to free the raw_parser output --- the grammar sometimes makes multiple links to the same subtree, eg in BETWEEN. In any case the patch as proposed wouldn't catch all the detritus for any case more complicated than a simple integer constant. The way that the list memory usage works (again, 64-bit machine) is sizeof(List) = 24 sizeof(ListCell) = 16 sizeof(A_Const) = 32 Each of these nodes will have 16 bytes palloc overhead, and the List header will be rounded up to 32 bytes as well, so we have total space for a 3-element integer list of 32+16 + (16+16 + 32+16) * 3 Add in 16+16 for the associated ListCell of the top list-of-lists, and you come to 320 bytes per sublist. Const happens to also be 32 bytes so the transformed lists are the same size. It's interesting to reflect on the fact that this comes to 184 bytes of useful data and 136 bytes of palloc overhead per row ... not sure if we can do much about the overhead though. regards, tom lane