Message-Id: <4DF7399B020000250003E5AB@gw.wicourts.gov>
Date: Tue, 14 Jun 2011 10:36:11 -0500
From: "Kevin Grittner" <Kevin.Grittner@wicourts.gov>
To: "Tim" <elatllat@gmail.com>
Cc: <pgsql-admin@postgresql.org>
Subject: Re: tsvector limitations
References: <BANLkTi=hAv876hXEru6FO8PNGWFNYB21ZA@mail.gmail.com>
	<4DF7275C020000250003E59F@gw.wicourts.gov>
	<BANLkTikz+MY1DoO5UcxzJbG+VNgGKn0HFg@mail.gmail.com>
In-Reply-To: <BANLkTikz+MY1DoO5UcxzJbG+VNgGKn0HFg@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Tim <elatllat@gmail.com> wrote:
 
> I would be surprised if there is no general "how big is this
> object" method in PostgreSQL.
 
You could cast to text and use octet_length().
 
> If it's "bad design" to store large text documents (pdf,docx,etc)
> as a BLOBs or on a filesystem and make them searchable with
> tsvectors can you suggest a good design?
 
Well, I suggested that storing a series of novels as a single entry
seemed bad design to me.  Perhaps one entry per novel or even finer
granularity would make more sense in most applications, but there
could be exceptions.  Likewise, a list of distinct words is of
dubious value in most applications' text searches.  We extract text
from court documents and store a tsvector for each document; we
don't aggregate all court documents for a year and create a
tsvector for that -- that would not be useful for us.
 
> If making your own search implementation is "better" what is the
> point of tsvectors?
 
I remember you asking about doing that, but I don't think anyone
else has advocated it.
 
> Maybe I'm missing something here?
 
If you were to ask for real-world numbers you'd probably get farther
than demanding that people volunteer their time to perform tests
that you define but don't seem willing to run.  Or if you describe
your use case in more detail, with questions about alternative
approaches, you're likely to get useful advice.
 
-Kevin