pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tim <elatllat@gmail.com>
To: Mark Johnson <mark@remingtondatabasesolutions.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: tsvector limitations
Date: Mon, 13 Jun 2011 21:58:54 -0400
Message-ID: <BANLkTiniXCCAdwD0qDXb3mqLSSQrzqKSgQ@mail.gmail.com> (raw)
In-Reply-To: <W15534311280741308015817@webmail13>
References: <W15534311280741308015817@webmail13>

Mark,

That link is a mirror of this mailing list; it's not from 5 months ago.
If you are in the year 2012 please respond with lottery numbers and the
like.



On Mon, Jun 13, 2011 at 9:43 PM, Mark Johnson <
mark@remingtondatabasesolutions.com> wrote:

>
>
> I found another post where you asked the same questions 5 months ago.  Have
> you tested in that time?
> http://www.spinics.net/lists/pgsql-admin/msg19438.html
>
>
> A text search vector is an array of distinct lexemes (less any stopwords)
> and their positions.  Taking your example we get ...
>
> select to_tsvector('the lord of the rings.txt') "answer";
>       answer
> -------------------
> 'lord':2, 'rings.txt':5
>
> You can put the length() function around it to just get the number of
> lexemes.  This is the size in terms of number of distinct lexemes, not size
> in terms of space utilization.
>
> select length(to_tsvector('the lord of the rings.txt')) "answer";
>   answer
> --------
>         2
>
> You might find the tsvector data consumes 2x the space required by the
> input text.  It will depend on your configuration and your input data.  Test
> it and let us know what you find.
>
> -Mark
>
> -----Original Message-----
> *From:* Tim [mailto:elatllat@gmail.com]
> *Sent:* Monday, June 13, 2011 03:19 PM
> *To:* pgsql-admin@postgresql.org
> *Subject:* [ADMIN] tsvector limitations
>
> Dear list,
>
> How big of a file would one need to fill the 1MB limit of a tsvector?
> Reading
> http://www.postgresql.org/docs/9.0/static/textsearch-limitations.htmlseems to hint that filling a tsvector is improbable.
>
> Is there an easy way of query the bytes of a tsvector?
> something like length(tsvector) but bytes(tsvector).
>
> If there no easy method to query the bytes of a tsvector
> I realize the answer is highly dependent on the contents of the file, so
> here are 2 random examples:
> How many bytes of a tsvector would a 32MB ascii english unique word list
> make?
> How many bytes of a tsvector would something like "The Lord of the
> Rings.txt" make?
>
> If this limitation is ever hit is there a common practice for using more
> than one tsvector?
> Using a separate "one to many" table seems like an obvious solution piece,
> but I would not know how to detect or calculate how much text to give each
> tsvector.
> Assuming tsvectors can't be linked maybe they would need some overlap.
>
>
> Thanks in advance.
>
>

view thread (21+ messages)  latest in thread

Message-ID: <BANLkTiniXCCAdwD0qDXb3mqLSSQrzqKSgQ@mail.gmail.com>
Permalink:  ../BANLkTiniXCCAdwD0qDXb3mqLSSQrzqKSgQ@mail.gmail.com/
Also on:    postgresql.org/message-id/BANLkTiniXCCAdwD0qDXb3mqLSSQrzqKSgQ@mail.gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: elatllat@gmail.com, mark@remingtondatabasesolutions.com
  Subject: Re: tsvector limitations
  In-Reply-To: <BANLkTiniXCCAdwD0qDXb3mqLSSQrzqKSgQ@mail.gmail.com>
  References: <W15534311280741308015817@webmail13> <BANLkTiniXCCAdwD0qDXb3mqLSSQrzqKSgQ@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox