pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Very long time to execute and Update, suggestions?
3+ messages / 2 participants
[nested] [flat]

* Very long time to execute and Update, suggestions?
@ 2005-03-31 19:04 Madison Kelly <linux@alteeve.com>
  2005-03-31 19:51 ` Re: Very long time to execute and Update, suggestions? Madison Kelly <linux@alteeve.com>
  2005-03-31 21:28 ` Re: Very long time to execute and Update, suggestions? PFC <lists@boutiquenumerique.com>
  0 siblings, 2 replies; 3+ messages in thread

From: Madison Kelly @ 2005-03-31 19:04 UTC (permalink / raw)
  To: pgsql-performance

Hi all,

   I have a table with a little over 200,000 columns in it that I need 
to update using a regular expression. I admit that though I am not a 
beginner and postgres, I am also far from an expert. :p

   I tried to create an Index that would optimize the UPDATE but I may 
have made an error in how I built it. Here is the table structure, the 
index I tried to create and an 'EXPLAIN ANALYZE' of the UPDATE (though I 
  am still just now learning how to use 'EXPLAIN').

tle-bu=> \d file_info_3
                             Table "public.file_info_3"
      Column      |         Type         |                Modifiers
-----------------+----------------------+-----------------------------------------
  file_group_name | text                 | not null
  file_group_uid  | bigint               | not null
  file_mod_time   | bigint               | not null
  file_name       | text                 | not null
  file_parent_dir | text                 | not null
  file_perm       | text                 | not null
  file_size       | bigint               | not null
  file_type       | character varying(2) | not null default 
'f'::character varying
  file_user_name  | text                 | not null
  file_user_uid   | bigint               | not null
  file_backup     | boolean              | not null default true
  file_display    | boolean              | not null default false
  file_restore    | boolean              | not null default false
Indexes:
     "file_info_3_display_idx" btree (file_type, file_parent_dir, file_name)

   Here is the EXPLAIN:

tle-bu=> EXPLAIN ANALYZE UPDATE file_info_3 SET file_backup='f' WHERE 
file_parent_dir~'^/home' OR (file_parent_dir='/' AND file_name='home');
                                                       QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------
  Seq Scan on file_info_3  (cost=0.00..7770.00 rows=1006 width=206) 
(actual time=1050.813..5648.462 rows=67835 loops=1)
    Filter: ((file_parent_dir ~ '^/home'::text) OR ((file_parent_dir = 
'/'::text) AND (file_name = 'home'::text)))
  Total runtime: 68498.898 ms
(3 rows)

   I thought that it would have used the index because 'file_parent_dir' 
and 'file_name' are in the index but is I am reading the "EXPLAIN" 
output right it isn't but is instead doing a sequencial scan. If that is 
the case, how would I best built the index? Should I have just used the 
'file_parent_dir' and 'file_name'?

   Thanks all!!

Madison

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Madison Kelly (Digimer)
TLE-BU, The Linux Experience; Back Up
http://tle-bu.thelinuxexperience.com
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: Very long time to execute and Update, suggestions?
  2005-03-31 19:04 Very long time to execute and Update, suggestions? Madison Kelly <linux@alteeve.com>
@ 2005-03-31 19:51 ` Madison Kelly <linux@alteeve.com>
  1 sibling, 0 replies; 3+ messages in thread

From: Madison Kelly @ 2005-03-31 19:51 UTC (permalink / raw)
  To: Philip Hallstrom <postgresql@philip.pjkh.com>; +Cc: pgsql-performance

Philip Hallstrom wrote:
> I'm not sure about this which is why I'm replying off list, but your 
> index is on file_type, file_parent_dir, and file_name and you're query 
> is on file_parent_dir and file_name.
> 
> I seem to remember reading that that the index will only get used if the 
> columns in the where clause "match up" "in order".
> 
> That is um... if you have an index on columns a and b and a where clause 
> of "b = 1" it woin't use the index since the index "looks like"
> 
> a, b
> a, b
> a, b
> etc...
> 
> Does that make any sense?  Not sure if that's right or not, but easy 
> enough to remove the "file_type" from your index and try it.
> 
> post back to the list if that's it.
> 
> -philip

Thanks for the reply!

   I have played around a little more and have created a few different 
test Indexes and it looks like it is the regex that is causing it to do 
the sequential scan. If I remove the regex and create a 
'file_parent_dir', 'file_name' index it will use it. If I create an 
Index just for 'file_parent_dir' and change my UPDATE to just look for 
the regex '... WHERE file_parent_dir~'^/<dir>'...' it will still do the 
sequential scan anyway.

   So I need to either find an Index that will work with regexes or 
re-write my code to update each subdirectory separately and use simpler 
UPDATE statement for each.

   Thanks again!

Madison

PS - I cc'ed the list to follow up on what I found out so far. (Hi list!)


-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Madison Kelly (Digimer)
TLE-BU, The Linux Experience; Back Up
http://tle-bu.thelinuxexperience.com
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: Very long time to execute and Update, suggestions?
  2005-03-31 19:04 Very long time to execute and Update, suggestions? Madison Kelly <linux@alteeve.com>
@ 2005-03-31 21:28 ` PFC <lists@boutiquenumerique.com>
  1 sibling, 0 replies; 3+ messages in thread

From: PFC @ 2005-03-31 21:28 UTC (permalink / raw)
  To: Madison Kelly <linux@alteeve.com>; Philip Hallstrom <postgresql@philip.pjkh.com>; +Cc: pgsql-performance


>    So I need to either find an Index that will work with regexes or  
> re-write my code to update each subdirectory separately and use simpler  
> UPDATE statement for each.

	Why don't you use a LTREE type to model your directory tree ? It's been  
designed specifically for this purpose and has indexed regular expression  
search.

http://www.sai.msu.su/~megera/postgres/gist/ltree/
http://www.sai.msu.su/~megera/postgres/gist/



^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2005-03-31 21:28 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-03-31 19:04 Very long time to execute and Update, suggestions? Madison Kelly <linux@alteeve.com>
2005-03-31 19:51 ` Madison Kelly <linux@alteeve.com>
2005-03-31 21:28 ` PFC <lists@boutiquenumerique.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox