Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1UcA0r-00087M-7N for pgsql-students@arkaria.postgresql.org; Tue, 14 May 2013 07:51:09 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.72) (envelope-from ) id 1UcA0q-0007uN-3f for pgsql-students@arkaria.postgresql.org; Tue, 14 May 2013 07:51:08 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1UcA0p-0007uF-Nf; Tue, 14 May 2013 07:51:07 +0000 Received: from moutng.kundenserver.de ([212.227.17.9]) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1UcA0m-0000nr-AH; Tue, 14 May 2013 07:51:07 +0000 Received: from [192.168.43.209] ([41.202.233.189]) by mrelayeu.kundenserver.de (node=mrbap2) with ESMTP (Nemesis) id 0M8opw-1UoDE61gdQ-00CUw8; Tue, 14 May 2013 09:50:59 +0200 Message-ID: <5191ECDA.3080204@krosing.net> Date: Tue, 14 May 2013 10:50:50 +0300 From: Hannu Krosing User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130329 Thunderbird/17.0.5 MIME-Version: 1.0 To: Simon Riggs CC: PostgreSQL-development , pgsql-students@postgresql.org Subject: Re: [HACKERS] Slicing TOAST References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:kGPlPVxrxmwCTC5nmlGCynfLOoVD+TYxDV9ZiPve0IO Ha7AqIcW5Kc/NMeotyQ6tZzJ6VjQewzjbNr0YnyWq28JBKgAvw RUJUqep2JgXmqljFn7u/fenRIMo9XoVbt432iiXpABOcWEipE7 6z+cX1/WOCVOrA3qwFJPZbnHuA75vNG9jPkVXLbL7iByLivbzc DaSOHGO6uRPQ2OhQXU1hzGV/e3Jr+mWrQzQQYECTG9b3MhtkDm zkXiLBfSVgjWuX/ZXdOmryigJ/c+9wH5NY+9Ui7rn5d6sN8Awp fQa/FPl3Y0co+Lu/8jOpvIFTeAnlJTEEx7OVZ+i0YK9YTMJWt6 5vDAh4AW8eBUjQdWowO4= X-Pg-Spam-Score: -1.9 (-) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-students Precedence: bulk Sender: pgsql-students-owner@postgresql.org On 05/14/2013 10:05 AM, Simon Riggs wrote: > I'm proposing this now as a possible GSoC project: > > In 1-byte character encodings (i.e. not UTF-8), SUBSTR() is optimised > to allow seeking straight to the exact slice when retrieving a large > toasted value. This reduces I/O considerably when you have large > toasted values since it is an O(1) action rather than an O(N). > > This is possible because the slicing of toasted values is predictable > on 1 byte encodings. > > It would be useful to have a predictable function perform the slicing, > so we could use that knowledge later to optimise searches in a wider > range of situations. More specifically, since UTF-8 is so common, to > allow optimisations in that encoding of common data: text, XML, JSON. > > e.g. if we knew that an XML document has a required element called > TITLE and that occurs only once and always in the first slice, it > would be useful information to use in search functions. (Not sure, but > it may be possible to assign non-consecutive slice numbers to allow > variable data mid-way through a column value if needed). > > e.g. in UTF-8 free text we could put 500 characters in each slice, so > that even if that could be anywhere between 500 and 2000 bytes it > would still fit just fine. > > e.g. for arrays, if we put say 200 elements per slice, then accessing > particular elements would require only 1 slice retrieval. > > Doing this would *possibly* reduce packing density, but not certainly > so. But it would greatly improve access times to large structured > toast values. On the contrary, as it would enable us to pack the chunks fitting more on the page, especially for :) That is, first chunk into N bytes, then compress each chunk ----------------- Hannu -- Sent via pgsql-students mailing list (pgsql-students@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-students