From: Hannu Krosing <hannu@tm.ee>
To: Dann Corbit <DCorbit@connx.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Steve Crawford <scrawford@pinpointresearch.com>
Cc: pgsql-performance@postgreSQL.org
Cc: pgsql-hackers@postgreSQL.org
Subject: Re: [PERFORM] Terrible performance on wide selects
Date: 23 Jan 2003 12:28:21 +0200
Message-ID: <1043317701.2348.32.camel@localhost.localdomain> (raw)
In-Reply-To: <D90A5A6C612A39408103E6ECDD77B8294CD866@voyager.corporate.connx.com>
References: <D90A5A6C612A39408103E6ECDD77B8294CD866@voyager.corporate.connx.com>
Dann Corbit kirjutas N, 23.01.2003 kell 02:39:
> [snip]
> > For the disk case, why not have the start of the record
> > contain an array of offsets to the start of the data for each
> > column? It would only be necessary to have a list for
> > variable fields.
> >
> > So (for instance) if you have 12 variable fields, you would
> > store 12 integers at the start of the record.
>
> You have to store this information anyway (for variable length objects).
> By storing it at the front of the record you would lose nothing (except
> the logical coupling of an object with its length). But I would think
> that it would not consume any additional storage.
I don't think it will win much either (except for possible cache
locality with really huge page sizes), as the problem is _not_ scanning
over big strings finding their end marker, but instead is chasing long
chains of pointers.
There could be some merit in the idea of storing in the beginning of
tuple all pointers starting with first varlen field (16 bit int should
be enough)
so people can minimize the overhead by moving fixlen fields to the
beginning. once we have this setup, we no longer need the varlen fields
/stored/ together with field data.
this adds complexity of converting form (len,data) to ptr,...,data) when
constructing the tuple
as tuple (int,int,int,varchar,varchar)
which is currently stored as
(intdata1, intdata2, intdata3, (len4, vardata4), (len5,vardata5))
should be rewritten on storage to
(ptr4,ptr5),(intdata1, intdata2, intdata3, vardata4,vardata5)
but it seems to solve the O(N) problem quite nicely (and forces no
storage growth for tuples with fixlen fields in the beginning of tuple)
and we must also account for NULL fields in calculations .
--
Hannu Krosing <hannu@tm.ee>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: hannu@tm.ee, DCorbit@connx.com, tgl@sss.pgh.pa.us, scrawford@pinpointresearch.com, pgsql-hackers@postgreSQL.org
Subject: Re: [PERFORM] Terrible performance on wide selects
In-Reply-To: <1043317701.2348.32.camel@localhost.localdomain>
References: <D90A5A6C612A39408103E6ECDD77B8294CD866@voyager.corporate.connx.com> <1043317701.2348.32.camel@localhost.localdomain>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox