Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id 0FA85B5DBCC for ; Wed, 15 Jun 2011 02:56:36 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 34497-02 for ; Wed, 15 Jun 2011 05:56:29 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from nm4-vm0.bullet.mail.sp2.yahoo.com (nm4-vm0.bullet.mail.sp2.yahoo.com [98.139.91.190]) by mail.postgresql.org (Postfix) with SMTP id E0763B5DBC4 for ; Wed, 15 Jun 2011 02:56:28 -0300 (ADT) Received: from [98.139.91.66] by nm4.bullet.mail.sp2.yahoo.com with NNFMP; 15 Jun 2011 05:56:27 -0000 Received: from [98.139.91.40] by tm6.bullet.mail.sp2.yahoo.com with NNFMP; 15 Jun 2011 05:56:27 -0000 Received: from [127.0.0.1] by omp1040.mail.sp2.yahoo.com with NNFMP; 15 Jun 2011 05:56:27 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 706575.46173.bm@omp1040.mail.sp2.yahoo.com Received: (qmail 41990 invoked by uid 60001); 15 Jun 2011 05:56:27 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1308117387; bh=De07VWSr7LlMl+zEXsY50XrDMAMKc0q2jZwIA4Qt1tY=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=ewS6cm55Qg+drg503Hr4VYeq5JS/LK7WKUcsp7sl/vegflsOqRgNpI6jI+Xm+Z3D7DVZ+ycDXKu5AwT6JabXzMWgZWl8XyBtmodWboe37atO3CtLPPpDd+62xeA7GZQiEppQKsnuWBKxCoOSRy5QfEYiJWs61Tl4Td0s8A4AR+0= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=btDFXVGfzLh+9qBlxtlsfpgQLoZLUJBS5wl5+u1gsgh0wvGElreFQkA0pregjcX/C3Uly5nPqVe6Br0NPdJvj1Zj8oAjPqlMcS2lK/kp3Oq3u69egNXslnx6eCSDFMkY1FyePZ3Mr3F9Cc4N6AHXEGqf8+QPPM4ZCbCP+bZ/ykk=; Message-ID: <453674.41390.qm@web46113.mail.sp1.yahoo.com> X-YMail-OSG: UA0A0MwVM1nfDoyaC1pgP6W7q_99OWiRf88Yodpoq69HuaM fjlDTTJMY3BRdQr0T3eLlLNVh0eNqGOIzt0x6nO2yY6iD77zhk902CcPFdcz 7mCkKjpWdlNnUjkMX.Umu58XvubqOi7jzFyzTaqaQrt32LvSgb82QHZsdYN0 DOFgOHTbwBFyeGZxCu9sDTc2XgqTEdSu4Kag0nMRapSUpj8qGIHUYrQhTEWY 8n5UpBs8ZI3FTUTdoipXzUFyBs8P6_ttq3Ni4wC1ke4yeNB6rvDojFTdBzU0 N_U76uXDnE_DWfMtIDJ0NZQPq3tXsQbiStiTjjM1lmZ9lmxaMVbZwkPUmYcL KV7DncajwWvg2XNvqL8.PM.zGW4ZD723Cp2KvvHDGU2rv1Q-- Received: from [98.210.146.38] by web46113.mail.sp1.yahoo.com via HTTP; Tue, 14 Jun 2011 22:56:26 PDT X-Mailer: YahooMailRC/570 YahooMailWebService/0.8.111.304355 References: <4DF7275C020000250003E59F@gw.wicourts.gov> <4DF7399B020000250003E5AB@gw.wicourts.gov> <4DF79808020000250003E649@gw.wicourts.gov> Date: Tue, 14 Jun 2011 22:56:26 -0700 (PDT) From: Greg Williamson Subject: Re: tsvector limitations To: Kevin Grittner , Tim , pgsql-admin@postgresql.org In-Reply-To: <4DF79808020000250003E649@gw.wicourts.gov> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-0.355 tagged_above=-10 required=5 tests=BAYES_00=-1.9, FREEMAIL_ENVFROM_END_DIGIT=1.553, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RFC_ABUSE_POST=0.001, T_RP_MATCHES_RCVD=-0.01 X-Spam-Level: X-Archive-Number: 201106/78 X-Sequence-Number: 37704 Kevin Grittner wrote: > Tim wrote: > <...> > Your test (whatever data it is that you used) don't seem typical of > English text. The entire PostgreSQL documentation in HTML form, > when all the html files are concatenated is 11424165 bytes (11MB), > and the tsvector of that is 364410 (356KB). I don't suppose you > know of some publicly available file on the web that I could use to > reproduce your problem? Try trolling texts at the Internet Archive (archive.org) -- lots of stuff that has been rendered into ASCII ... Government documents and the like from all periods; novels and the like that are no longer under copyright, so lots of long classics. for example ... 765K HTH, Greg Williamson -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin