pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
How to speed up tsearch2 indexing
4+ messages / 2 participants
[nested] [flat]

* How to speed up tsearch2 indexing
@ 2005-03-09 17:25 Werner Bohl <WernerBohl@infutor.com>
  2005-03-09 17:41 ` Re: How to speed up tsearch2 indexing Oleg Bartunov <oleg@sai.msu.su>
  0 siblings, 1 reply; 4+ messages in thread

From: Werner Bohl @ 2005-03-09 17:25 UTC (permalink / raw)
  To: pgsql-performance

All,

I have a table with ~ 3 million records. I'm indexing a field holding
names, no more than 200 bytes each. Indexing the resulting tsvector
takes forever. It's been running now for more than 40 hours on a Linux
with PG 8.01, a single Xeon & 4GB RAM. My  work_mem postgresql.conf
parameter is at 240960 and maintenance_work_mem at 96384, although the
index task is using at most 12MB. Task is 99% cpu bound. Is there any
way I may speed up the indexing?


TIA,

	
-- 
Werner Bohl <WernerBohl@infutor.com>
IDS de Costa Rica S.A.




^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: How to speed up tsearch2 indexing
  2005-03-09 17:25 How to speed up tsearch2 indexing Werner Bohl <WernerBohl@infutor.com>
@ 2005-03-09 17:41 ` Oleg Bartunov <oleg@sai.msu.su>
  2005-03-09 20:21   ` Re: How to speed up tsearch2 indexing Werner Bohl <WernerBohl@infutor.com>
  0 siblings, 1 reply; 4+ messages in thread

From: Oleg Bartunov @ 2005-03-09 17:41 UTC (permalink / raw)
  To: Werner Bohl <WernerBohl@infutor.com>; +Cc: pgsql-performance

On Wed, 9 Mar 2005, Werner Bohl wrote:

> All,
>
> I have a table with ~ 3 million records. I'm indexing a field holding
> names, no more than 200 bytes each. Indexing the resulting tsvector
> takes forever. It's been running now for more than 40 hours on a Linux
> with PG 8.01, a single Xeon & 4GB RAM. My  work_mem postgresql.conf
> parameter is at 240960 and maintenance_work_mem at 96384, although the
> index task is using at most 12MB. Task is 99% cpu bound. Is there any
> way I may speed up the indexing?

What's your tsearch2 configuration ? Do you use dictionaries ?
I wrote a brief explanation of tsearch2 internals
http://www.sai.msu.su/~megera/oddmuse/index.cgi/Tsearch_V2_internals

Hope, it could help you.

>
>
> TIA,
>
>
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: How to speed up tsearch2 indexing
  2005-03-09 17:25 How to speed up tsearch2 indexing Werner Bohl <WernerBohl@infutor.com>
  2005-03-09 17:41 ` Re: How to speed up tsearch2 indexing Oleg Bartunov <oleg@sai.msu.su>
@ 2005-03-09 20:21   ` Werner Bohl <WernerBohl@infutor.com>
  2005-03-09 20:27     ` Re: How to speed up tsearch2 indexing Oleg Bartunov <oleg@sai.msu.su>
  0 siblings, 1 reply; 4+ messages in thread

From: Werner Bohl @ 2005-03-09 20:21 UTC (permalink / raw)
  To: Oleg Bartunov <oleg@sai.msu.su>; +Cc: pgsql-performance

On Wed, 2005-03-09 at 20:41 +0300, Oleg Bartunov wrote:

> What's your tsearch2 configuration ? Do you use dictionaries ?
> I wrote a brief explanation of tsearch2 internals
> http://www.sai.msu.su/~megera/oddmuse/index.cgi/Tsearch_V2_internals
> 
Tsearch2 is using default english configuration. No dictionaries, just
put some more stop words (10) in english.stop.

> Hope, it could help you.
> 
> >
> >
> > TIA,
> >
> >
> >
> 
>  	Regards,
>  		Oleg
> _____________________________________________________________
> Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
> Sternberg Astronomical Institute, Moscow University (Russia)
> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
> phone: +007(095)939-16-83, +007(095)939-23-83
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
> 
>                http://archives.postgresql.org




^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: How to speed up tsearch2 indexing
  2005-03-09 17:25 How to speed up tsearch2 indexing Werner Bohl <WernerBohl@infutor.com>
  2005-03-09 17:41 ` Re: How to speed up tsearch2 indexing Oleg Bartunov <oleg@sai.msu.su>
  2005-03-09 20:21   ` Re: How to speed up tsearch2 indexing Werner Bohl <WernerBohl@infutor.com>
@ 2005-03-09 20:27     ` Oleg Bartunov <oleg@sai.msu.su>
  0 siblings, 0 replies; 4+ messages in thread

From: Oleg Bartunov @ 2005-03-09 20:27 UTC (permalink / raw)
  To: Werner Bohl <WernerBohl@infutor.com>; +Cc: pgsql-performance

On Wed, 9 Mar 2005, Werner Bohl wrote:

> On Wed, 2005-03-09 at 20:41 +0300, Oleg Bartunov wrote:
>
>> What's your tsearch2 configuration ? Do you use dictionaries ?
>> I wrote a brief explanation of tsearch2 internals
>> http://www.sai.msu.su/~megera/oddmuse/index.cgi/Tsearch_V2_internals
>>
> Tsearch2 is using default english configuration. No dictionaries, just
> put some more stop words (10) in english.stop.

it's not good, because you, probably, have a lot of unique words.
Do you have some statistics, see stat() function ?


>
>> Hope, it could help you.
>>
>>>
>>>
>>> TIA,
>>>
>>>
>>>
>>
>>  	Regards,
>>  		Oleg
>> _____________________________________________________________
>> Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
>> Sternberg Astronomical Institute, Moscow University (Russia)
>> Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
>> phone: +007(095)939-16-83, +007(095)939-23-83
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 6: Have you searched our list archives?
>>
>>                http://archives.postgresql.org
>

 	Regards,
 		Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83



^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2005-03-09 20:27 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-03-09 17:25 How to speed up tsearch2 indexing Werner Bohl <WernerBohl@infutor.com>
2005-03-09 17:41 ` Oleg Bartunov <oleg@sai.msu.su>
2005-03-09 20:21   ` Werner Bohl <WernerBohl@infutor.com>
2005-03-09 20:27     ` Oleg Bartunov <oleg@sai.msu.su>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox