X-Original-To: pgsql-patches-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 1B189528EF for ; Tue, 9 Aug 2005 22:01:44 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 32841-03 for ; Wed, 10 Aug 2005 01:01:42 +0000 (GMT) Received: from trolak.mydnsbox2.com (ns1.mydnsbox2.com [207.44.142.118]) by svr1.postgresql.org (Postfix) with ESMTP id 5BE7C528E1 for ; Tue, 9 Aug 2005 22:01:41 -0300 (ADT) Received: from [192.168.1.103] (cpe-024-211-165-134.nc.res.rr.com [24.211.165.134]) (authenticated (0 bits)) by trolak.mydnsbox2.com (8.11.6/8.11.6) with ESMTP id j7A0GoK23122; Tue, 9 Aug 2005 19:16:50 -0500 Message-ID: <42F951F2.8010104@dunslane.net> Date: Tue, 09 Aug 2005 21:01:38 -0400 From: Andrew Dunstan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050719 Fedora/1.7.10-1.3.1 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Alon Goldshuv Cc: pgsql-patches@postgresql.org Subject: Re: COPY FROM performance improvements References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.048 tagged_above=0 required=5 tests=AWL X-Spam-Level: X-Archive-Number: 200508/76 X-Sequence-Number: 17023 Alon Goldshuv wrote: >I performed those measurement by executing *only the parsing logic* of the >COPY pipeline. All data conversion (functioncall3(string...)) and tuple >handling (form_heaptuple etc...) and insertion were manually disabled. So >the only code measured is reading from disk and parsing to the attribute >level. > > Arguably this might exaggerate the effect quite significantly. Users will want to know the real time effect on a complete COPY. Depending on how much the pasing is in the total time your 20% improvement in parsing might only be a small fraction of 20% improvement in COPY. Like you, I'm happy we have seen a 5 times improvement in parsing. Is it possible you can factor out something smallish from your patch that might make up the balance? cheers andrew