X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [64.117.224.130]) by svr1.postgresql.org (Postfix) with ESMTP id 778E4D1F473 for ; Thu, 14 Aug 2003 18:18:48 +0000 (GMT) Received: from svr1.postgresql.org ([64.117.224.193]) by localhost (neptune.hub.org [64.117.224.130]) (amavisd-new, port 10024) with ESMTP id 23441-05 for ; Thu, 14 Aug 2003 15:18:38 -0300 (ADT) Received: from mail-2.zoominternet.net (mail-2.zoominternet.net [63.67.120.22]) by svr1.postgresql.org (Postfix) with SMTP id 2B4B0D1F6AC for ; Thu, 14 Aug 2003 15:11:13 -0300 (ADT) Received: (qmail 15194 invoked from network); 14 Aug 2003 18:11:10 -0000 Received: from acs-24-154-209-142.zoominternet.net (HELO gmod.lajolla) ([24.154.209.142]) (envelope-sender ) by mail-2.zoominternet.net (qmail-ldap-1.03) with SMTP for ; 14 Aug 2003 18:11:10 -0000 Subject: Re: [SQL] EXTERNAL storage and substring on long strings From: Scott Cain To: Tom Lane Cc: Jan Wieck , PgSQL Performance ML , pgsql-sql@postgresql.org In-Reply-To: <6311.1060884021@sss.pgh.pa.us> References: <1059679600.1429.41.camel@localhost.localdomain> <3F297CBA.1080703@joeconway.com> <1059686498.7505.558.camel@haggis> <1060010735.1433.26.camel@localhost.localdomain> <3574.1060012423@sss.pgh.pa.us> <3F3BA49D.9070405@Yahoo.com> <6311.1060884021@sss.pgh.pa.us> Content-Type: text/plain Organization: Cold Spring Harbor Lab Message-Id: <1060884667.1430.6.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.3 Date: 14 Aug 2003 14:11:07 -0400 Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at postgresql.org X-Archive-Number: 200308/266 X-Sequence-Number: 3082 Agreed. When I actually Did It Right (tm), EXTERNAL storage gave similar (probably better) performance as my shredding method, without all the hoops to breakup and reassemble the string. Scott On Thu, 2003-08-14 at 14:00, Tom Lane wrote: > Jan Wieck writes: > > Doesn't look that unrealistic to me. A plain text based substring > > function will reassemble the whole beast first before cutting out the > > wanted part. His manually chunked version will read only those chunks > > needed. > > So does substr(), if applied to EXTERNAL (non-compressed) toasted text. > See John Gray's improvements a release or two back. > > regards, tom lane -- ------------------------------------------------------------------------ Scott Cain, Ph. D. cain@cshl.org GMOD Coordinator (http://www.gmod.org/) 216-392-3087 Cold Spring Harbor Laboratory