agora inbox for pgsql-sql@postgresql.org
help / color / mirror / Atom feedQuerying with arrays
6+ messages / 4 participants
[nested] [flat]
* Querying with arrays
@ 2014-11-27 13:09 Tim Dudgeon <tdudgeon.ml@gmail.com>
0 siblings, 1 reply; 6+ messages in thread
From: Tim Dudgeon @ 2014-11-27 13:09 UTC (permalink / raw)
To: pgsql-sql
I'm considering using arrays to handle managing "lists" of rows (I know
this may not be the best approach, but bear with me).
I create a table for my lists like this:**
create table lists (
id SERIAL PRIMARY KEY,
hits INTEGER[] NOT NULL
);
Then I can insert the results of a query into that table as a new list
of hits
INSERT INTO lists (hits)
SELECT array_agg(id)
FROM some_table
WHERE ...;
Now the problem part. How to best use that array of primary key values
to restore the data at a later stage. Conceptually I'm wanting this:
SELECT * from some_table
WHERE id <is in the list of ids in the array in the lists table>;
These both work by are really slow:
SELECT t1.*
FROM some_table t1
WHERE t1.id IN (SELECT unnest(hits) from lists WHERE id = 2);
SELECT t1.*
FROM some_table t1
JOIN lists l ON t1.id = any(l.hits)
WHERE l.id = 2;
Is there an efficient way to do this, or is this a dead end?
Thanks
Tim
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Querying with arrays
@ 2014-11-27 14:54 Tom Lane <tgl@sss.pgh.pa.us>
parent: Tim Dudgeon <tdudgeon.ml@gmail.com>
0 siblings, 2 replies; 6+ messages in thread
From: Tom Lane @ 2014-11-27 14:54 UTC (permalink / raw)
To: Tim Dudgeon <tdudgeon.ml@gmail.com>; +Cc: pgsql-sql
Tim Dudgeon <tdudgeon.ml@gmail.com> writes:
> I'm considering using arrays to handle managing "lists" of rows (I know
> this may not be the best approach, but bear with me).
> I create a table for my lists like this:**
> create table lists (
> id SERIAL PRIMARY KEY,
> hits INTEGER[] NOT NULL
> );
> Then I can insert the results of a query into that table as a new list
> of hits
> INSERT INTO lists (hits)
> SELECT array_agg(id)
> FROM some_table
> WHERE ...;
> Now the problem part. How to best use that array of primary key values
> to restore the data at a later stage. Conceptually I'm wanting this:
> SELECT * from some_table
> WHERE id <is in the list of ids in the array in the lists table>;
> These both work by are really slow:
> SELECT t1.*
> FROM some_table t1
> WHERE t1.id IN (SELECT unnest(hits) from lists WHERE id = 2);
> SELECT t1.*
> FROM some_table t1
> JOIN lists l ON t1.id = any(l.hits)
> WHERE l.id = 2;
> Is there an efficient way to do this, or is this a dead end?
You could create a GIN index on lists.hits and then do
SELECT t1.*
FROM some_table t1
JOIN lists l ON array[t1.id] <@ l.hits
WHERE l.id = 2;
How efficient that will be remains to be determined though;
if the l.id condition will eliminate a lot of matches it
could still be kind of slow.
(ISTR some talk of teaching the planner to convert =ANY(array)
conditions to this form automatically when there's a suitable
index, but for now you'd have to write it out like this.)
regards, tom lane
--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Querying with arrays
@ 2014-11-27 16:55 Tim Dudgeon <tdudgeon.ml@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 6+ messages in thread
From: Tim Dudgeon @ 2014-11-27 16:55 UTC (permalink / raw)
To: ; +Cc: pgsql-sql
On 27/11/2014 14:54, Tom Lane wrote:
> You could create a GIN index on lists.hits and then do
>
> SELECT t1.*
> FROM some_table t1
> JOIN lists l ON array[t1.id] <@ l.hits
> WHERE l.id = 2;
>
> How efficient that will be remains to be determined though;
> if the l.id condition will eliminate a lot of matches it
> could still be kind of slow.
Afraid that was *much* slower.
Tim
--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Querying with arrays
@ 2014-12-04 13:42 Tim Dudgeon <tdudgeon.ml@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 2 replies; 6+ messages in thread
From: Tim Dudgeon @ 2014-12-04 13:42 UTC (permalink / raw)
To: pgsql-sql
Looking into this further I don't seem able to get the index used.
I created this simple example:
create table lists (
id SERIAL PRIMARY KEY,
name VARCHAR(32) NOT NULL,
hits INTEGER[] NOT NULL
);
CREATE INDEX idx_lists_hits ON lists USING gin (hits);
INSERT INTO lists (name, hits) VALUES ('list1-10',
ARRAY[1,2,3,4,5,6,7,8,9,10]);
explain analyze SELECT id, name FROM lists
WHERE hits @> array[7];
The plan for the query is this:
"Seq Scan on lists (cost=0.00..16.88 rows=3 width=86) (actual
time=0.006..0.008 rows=1 loops=1)"
" Filter: (hits @> '{7}'::integer[])"
"Planning time: 0.058 ms"
"Execution time: 0.025 ms"
What am I doing wrong?
Tim
On 27/11/2014 11:54, Tom Lane wrote:
> Tim Dudgeon <tdudgeon.ml@gmail.com> writes:
>> I'm considering using arrays to handle managing "lists" of rows (I know
>> this may not be the best approach, but bear with me).
>> I create a table for my lists like this:**
>> create table lists (
>> id SERIAL PRIMARY KEY,
>> hits INTEGER[] NOT NULL
>> );
>> Then I can insert the results of a query into that table as a new list
>> of hits
>> INSERT INTO lists (hits)
>> SELECT array_agg(id)
>> FROM some_table
>> WHERE ...;
>> Now the problem part. How to best use that array of primary key values
>> to restore the data at a later stage. Conceptually I'm wanting this:
>> SELECT * from some_table
>> WHERE id <is in the list of ids in the array in the lists table>;
>> These both work by are really slow:
>> SELECT t1.*
>> FROM some_table t1
>> WHERE t1.id IN (SELECT unnest(hits) from lists WHERE id = 2);
>> SELECT t1.*
>> FROM some_table t1
>> JOIN lists l ON t1.id = any(l.hits)
>> WHERE l.id = 2;
>> Is there an efficient way to do this, or is this a dead end?
> You could create a GIN index on lists.hits and then do
>
> SELECT t1.*
> FROM some_table t1
> JOIN lists l ON array[t1.id] <@ l.hits
> WHERE l.id = 2;
>
> How efficient that will be remains to be determined though;
> if the l.id condition will eliminate a lot of matches it
> could still be kind of slow.
>
> (ISTR some talk of teaching the planner to convert =ANY(array)
> conditions to this form automatically when there's a suitable
> index, but for now you'd have to write it out like this.)
>
> regards, tom lane
--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Querying with arrays
@ 2014-12-04 13:54 Achilleas Mantzios <achill@matrix.gatewaynet.com>
parent: Tim Dudgeon <tdudgeon.ml@gmail.com>
1 sibling, 0 replies; 6+ messages in thread
From: Achilleas Mantzios @ 2014-12-04 13:54 UTC (permalink / raw)
To: pgsql-sql
On 04/12/2014 15:42, Tim Dudgeon wrote:
> Looking into this further I don't seem able to get the index used.
> I created this simple example:
>
> create table lists (
> id SERIAL PRIMARY KEY,
> name VARCHAR(32) NOT NULL,
> hits INTEGER[] NOT NULL
> );
>
> CREATE INDEX idx_lists_hits ON lists USING gin (hits);
>
> INSERT INTO lists (name, hits) VALUES ('list1-10', ARRAY[1,2,3,4,5,6,7,8,9,10]);
>
> explain analyze SELECT id, name FROM lists
> WHERE hits @> array[7];
>
>
> The plan for the query is this:
>
> "Seq Scan on lists (cost=0.00..16.88 rows=3 width=86) (actual time=0.006..0.008 rows=1 loops=1)"
> " Filter: (hits @> '{7}'::integer[])"
> "Planning time: 0.058 ms"
> "Execution time: 0.025 ms"
>
> What am I doing wrong?
>
Maybe your test table is tiny?
> Tim
>
>
>
>
> On 27/11/2014 11:54, Tom Lane wrote:
>> Tim Dudgeon <tdudgeon.ml@gmail.com> writes:
>>> I'm considering using arrays to handle managing "lists" of rows (I know
>>> this may not be the best approach, but bear with me).
>>> I create a table for my lists like this:**
>>> create table lists (
>>> id SERIAL PRIMARY KEY,
>>> hits INTEGER[] NOT NULL
>>> );
>>> Then I can insert the results of a query into that table as a new list
>>> of hits
>>> INSERT INTO lists (hits)
>>> SELECT array_agg(id)
>>> FROM some_table
>>> WHERE ...;
>>> Now the problem part. How to best use that array of primary key values
>>> to restore the data at a later stage. Conceptually I'm wanting this:
>>> SELECT * from some_table
>>> WHERE id <is in the list of ids in the array in the lists table>;
>>> These both work by are really slow:
>>> SELECT t1.*
>>> FROM some_table t1
>>> WHERE t1.id IN (SELECT unnest(hits) from lists WHERE id = 2);
>>> SELECT t1.*
>>> FROM some_table t1
>>> JOIN lists l ON t1.id = any(l.hits)
>>> WHERE l.id = 2;
>>> Is there an efficient way to do this, or is this a dead end?
>> You could create a GIN index on lists.hits and then do
>>
>> SELECT t1.*
>> FROM some_table t1
>> JOIN lists l ON array[t1.id] <@ l.hits
>> WHERE l.id = 2;
>>
>> How efficient that will be remains to be determined though;
>> if the l.id condition will eliminate a lot of matches it
>> could still be kind of slow.
>>
>> (ISTR some talk of teaching the planner to convert =ANY(array)
>> conditions to this form automatically when there's a suitable
>> index, but for now you'd have to write it out like this.)
>>
>> regards, tom lane
>
>
>
--
Achilleas Mantzios
Head of IT DEV
IT DEPT
Dynacom Tankers Mgmt
--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Querying with arrays
@ 2014-12-04 19:56 Gerardo Herzig <gherzig@fmed.uba.ar>
parent: Tim Dudgeon <tdudgeon.ml@gmail.com>
1 sibling, 0 replies; 6+ messages in thread
From: Gerardo Herzig @ 2014-12-04 19:56 UTC (permalink / raw)
To: Tim Dudgeon <tdudgeon.ml@gmail.com>; +Cc: pgsql-sql
>
> The plan for the query is this:
>
> "Seq Scan on lists (cost=0.00..16.88 rows=3 width=86) (actual
> time=0.006..0.008 rows=1 loops=1)"
> " Filter: (hits @> '{7}'::integer[])"
> "Planning time: 0.058 ms"
> "Execution time: 0.025 ms"
>
> What am I doing wrong?
>
> Tim
>
With so few rows to read, it is actually slower to do a index scan followed by the table scan (in order to actually read the data).
Thats why it is using a seq scan.
Gerardo
--
Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2014-12-04 19:56 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-11-27 13:09 Querying with arrays Tim Dudgeon <tdudgeon.ml@gmail.com>
2014-11-27 14:54 ` Tom Lane <tgl@sss.pgh.pa.us>
2014-11-27 16:55 ` Tim Dudgeon <tdudgeon.ml@gmail.com>
2014-12-04 13:42 ` Tim Dudgeon <tdudgeon.ml@gmail.com>
2014-12-04 13:54 ` Achilleas Mantzios <achill@matrix.gatewaynet.com>
2014-12-04 19:56 ` Gerardo Herzig <gherzig@fmed.uba.ar>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox