Date: Wed, 16 Mar 2016 15:01:23 +0100 (CET)
From: Andreas Joseph Krogh <andreas@visena.com>
To: pgsql-performance@postgresql.org
Message-ID: <VisenaEmail.56.28aa743b723b9d9f.1537fb644d3@tc7-visena>
In-Reply-To: <VisenaEmail.55.5cac9849d07151e7.1537fb2771f@tc7-visena>
Subject: Re: Searching GIN-index (FTS) and sort by
 timestamp-column
MIME-Version: 1.0
Content-Type: multipart/related;
	boundary="----=_Part_411_1980921947.1458136883717"
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

------=_Part_410_16936214.1458136883717
Content-Type: multipart/related; 
	boundary="----=_Part_411_1980921947.1458136883717"

------=_Part_411_1980921947.1458136883717
Content-Type: multipart/alternative;
	boundary="----=_Part_412_1787936865.1458136883732"

------=_Part_412_1787936865.1458136883732
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

P=C3=A5 onsdag 16. mars 2016 kl. 14:53:04, skrev Andreas Joseph Krogh <
andreas@visena.com <mailto:andreas@visena.com>>:
P=C3=A5 onsdag 16. mars 2016 kl. 14:37:27, skrev Tom Lane <tgl@sss.pgh.pa.u=
s=20
<mailto:tgl@sss.pgh.pa.us>>:
Andreas Joseph Krogh <andreas@visena.com> writes:
 > 1. Why isnt' folder_id part of the index-cond?

 Because a GIN index is useless for sorting.

 > 2. Is there a way to make it use the (same) index to sort by
 > received_timestamp?

 No.

 > 3. Using a GIN-index, is there a way to use the index at all for sorting=
?

 No.

 > 4. It doesn't seem like ts_rank uses the index for sorting either.

 Same reason.

 regards, tom lane
=C2=A0
So it's basically impossible to use FTS/GIN with sorting on large datasets?
Are there any plans to improve this situation?
=C2=A0
Thanks.
=C2=A0
This paper talks about ORDER BY optimizations for FTS (starting at slide 6 =
and=20
7):
http://www.sai.msu.su/~megera/postgres/talks/Next%20generation%20of%20GIN.p=
df
=C2=A0
This indicates some work is being done in this area.
=C2=A0
Oleg, if you're listening, do you guys have any exiting news regarding this=
?
=C2=A0
-- Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
andreas@visena.com <mailto:andreas@visena.com>
www.visena.com <https://www.visena.com>
 <https://www.visena.com>


=C2=A0

------=_Part_412_1787936865.1458136883732
Content-Type: text/html;charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div>P=C3=A5 onsdag 16. mars 2016 kl. 14:53:04, skrev Andreas Joseph Krogh =
&lt;<a href=3D"mailto:andreas@visena.com">andreas@visena.com</a>&gt;:</div>

<blockquote style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt=
 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>P=C3=A5 onsdag 16. mars 2016 kl. 14:37:27, skrev Tom Lane &lt;<a href=
=3D"mailto:tgl@sss.pgh.pa.us">tgl@sss.pgh.pa.us</a>&gt;:</div>

<blockquote style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt=
 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style=3D"display:inline; font-family: monospace; font-size: 12px;">And=
reas Joseph Krogh &lt;andreas@visena.com&gt; writes:<br>
&gt; 1. Why isnt' folder_id part of the index-cond?<br>
<br>
Because a GIN index is useless for sorting.<br>
<br>
&gt; 2. Is there a way to make it use the (same) index to sort by<br>
&gt; received_timestamp?<br>
<br>
No.<br>
<br>
&gt; 3. Using a GIN-index, is there a way to use the index at all for sorti=
ng?<br>
<br>
No.<br>
<br>
&gt; 4. It doesn't seem like ts_rank uses the index for sorting either.<br>
<br>
Same reason.<br>
<br>
regards, tom lane</div>
</blockquote>

<div>=C2=A0</div>

<div>So it's basically impossible to use FTS/GIN with sorting on large data=
sets?</div>

<div>Are there any plans to improve this situation?</div>

<div>=C2=A0</div>

<div>Thanks.</div>
</blockquote>

<div>=C2=A0</div>

<div>This paper talks about ORDER BY optimizations for FTS (starting at sli=
de 6 and 7):</div>

<div>http://www.sai.msu.su/~megera/postgres/talks/Next%20generation%20of%20=
GIN.pdf</div>

<div>=C2=A0</div>

<div>This indicates some work is being done in this area.</div>

<div>=C2=A0</div>

<div>Oleg, if you're listening, do you guys have any exiting news regarding=
 this?</div>

<div>=C2=A0</div>

<div class=3D"origo-email-signature">
<div>--
<div><span style=3D"font-family:arial,helvetica,sans-serif;"><strong>Andrea=
s Joseph Krogh</strong></span></div>

<div><span style=3D"font-family:arial,helvetica,sans-serif;">CTO / Partner<=
/span> - Visena AS</div>

<div><span style=3D"font-family:arial,helvetica,sans-serif;">Mobile: +47 90=
9 56 963</span></div>

<div><a href=3D"mailto:andreas@visena.com">andreas@visena.com</a></div>

<div><a href=3D"https://www.visena.com">www.visena.com</a></div>

<div style=3D"padding-top: 5px"><a href=3D"https://www.visena.com"><img alt=
=3D"" src=3D"cid:part_F1448825556106OPWV7H"></a></div>
</div>
</div>

<div>=C2=A0</div>
------=_Part_412_1787936865.1458136883732--

------=_Part_411_1980921947.1458136883717
Content-Type: image/png
Content-Transfer-Encoding: base64
Content-Disposition: inline
Content-ID: <part_F1448825556106OPWV7H>

iVBORw0KGgoAAAANSUhEUgAAAIUAAAAYCAYAAADUIj6hAAAABHNCSVQICAgIfAhkiAAABzBJREFU
aEPtmNFxHDcMhmVP3i1VECpvnjzkVIHWFfhcgVcVRKrAUgWRK/C6Al8H3lTgy0PGbzFdQc4VJP/H
ADs43q6kROeJNbOYgQACIAgCWJKng4MZ5gxUGXh0U0b+SIuF9G+EWXj2Q15vbrE/NPtk9uub7Gfd
t5mBx1NhqSEo8HshjbEUtlO2yIM9tsx5dZP9rPt2MzDZFAqZE4LGcFjdsg3saQaHt7fY70X949On
aS+OZidDBkavD33157L4xaw2os+Mp/DAC10l2XhOCeStj0W5ajrJL8WfCi80Xgf9vVlrhg9yRONe
/f7xI2vNsIcM7JwU9o6oGyJrLT8JOA1aX9sKP4wl94bAniukEbo/n7YPmuTETzIab4Y9ZWCrKexd
8M58lxPCvvD6aihfvexbEQoPYO8NQRO0JnddGO6FJYaVsBde7cXj7KRk4LsqDxQzCYeGsJNgaXZe
+JXkyGgWINq3Gp+bHNILz+wEYs5qH1eJrouNrpDXtk4O6x1IvtD40GWyJYYdkF0jIbYZlN16xygI
gl9smbMDsmFdfAKb23xGB8H/mv3tOJfArs1kukk79DEPdQ5u0g1vCvvqKXIW8mZYW+F3Tg4r8HvZ
kQCCLydK8EFMQCe5N8RgL9mRG0SqQFuNvdE6beSs0qPDBuCdg0+gvClso8SbTO4ki7mQzQqBrcMH
QPwReg2wW8umEe/+L8T/LEzBGF9nXjzZ46s+ITHvhegW8LL39xm6App7KYL/GE+vcYnFbJiP/0aY
hdiCnRC7jaj7eiK2ETInC5PRF6LYkSN0+HabF75WuT6syCyI0YkVGGMvUC33AmfZTDUEj8u6IVhu
EhRUJ2U2g6UlugyNb03Hl9obHwlxJbcRzcYjO4QPjVfGFTQav4vrmp7cpMp2qbHnBxWJbisbho2Q
XI6C1sIHDUFhH4Hij4UbIev66eANeiwbkA+LBsO36zAHzoXU7AhbqLAXEiOYTXcSdO8VS9L4wN8U
BIYhBd6oSUgYMijOkecRuTdQa/YiBXhbXFcnCnI2SrfeBG9NydrLYMhGHa5qB9pQIxlzAE6Okjzx
JO5afGe6kmhBicWKQNKuTZ5EW+MjQY8v/9rQ0bhJSJyNGUe/JH1t8h1iMbdSPAvxHYin6YmN9YBX
QmTYZZNh14vHhhhal5vtcIrJjmvszPRJdExH3MWHvykoBEc9CoCGWJisOLOGoCORs1FvIMbYA8z3
kwM59oemy6LlWrLxFLmWgiQAfEGd8S+NssbK+EhyGLxUkhhiy717waBqHOJYSEacwBejkFNhjJNj
v/gANOdQxPfciP/JVBCK2cOIcg3RRJ+CPrLPNVhhN6F38VKMF3XLlIJrjU5C8gMFxvKDvOcPc4rV
NjCn7KM0BV+161V8viSCuJZ8SITGJGEh7IUUlxOFMYUH1kJOCN4Wh+I5pqCuK01k40kSNtnKiKIl
UfxAgW5sU5JlS05rtt5YFJF1SWoqHv6BRgQcA4/bdb9WRjmMk3jyUMAbIoyJq9e4cVmgzKt9j5gN
J/aYDhk+hhjExwaPcz5PObA5Zd9bvz5UzCTZuZDidu5AchpiKSwPR+TV1bCWqBQ9nCjJ5neivC82
Nr4LeS2j1gw5LUqwBuhGQQU5UwHeSslXkwIynz2U2A2yKDgG7OffwLA3TpGRpk0Tzpj3ZEJXi/GR
a6GN0e0NtppChePdcAz1FTS+FN8Kpxoiykk+J8fC5tenjbu9kSqpHLu9jBrhUuhNwVGbpyZrzrl0
HPVD8SXj5EOOD/eDC64VjvYBZLuUbIVAfBN1t/C/SU+cAGtdGo+fVnzycUX5wmn6eCKPmWYJG2E/
ppTsuRCbvUD9fwquksG5GqLVKhzDQ3GrEyLKSXhsiK3T5j9EyxffCFOYi2wUlPyFFDQAhehESDjQ
GoX0ho0oj8RPou6TxHJdcT0NTcWkO0AnG7+uXsnHqcas/72wvWF+mSf7N/Watp9kTXoluzeS0fB9
9CcZ/hvhSZTfh99pispZOXL9KrGrARkNUMu9ITbScZWs7xOYNt9pwxSZtYDsX/GE3xTkrXgwAsXm
fud08FiTeC+m2/o7ppo+PTS/NBK5ARpDG5YHr++DpmVfreYdiX8mnp+DzPEG9WbqJON0JBenZofs
sxBA1gj5NXGvfJu/Qh7HwQh/VDUEyUxCit5hH94QfKkEdu+GwK8BX0hvCF+D67xhjmXQCXMwJCZ+
olK0A1EKRCE4snsh42w8Mv/Zhxw9iD7Cjo7CyQC/KyF6oBfShL4PYgG+CHsYKyZxvxZSZBAgjhIz
YDz+AbfD37GtbaoSKzgGt+lKfI/GZtay6vG4VXTpPsh+IcQhOk9I7WYeP5AM3HZ9+DaSGIp9HIuu
huC4pCGGx+YD2fcc5tfIAA0h/Et4/jX8zz4fWAasIf4UbR9Y6HO4d8jAnd4U0Y8ageuCB+c+H5R3
CHU2mTMwZ+B/y8DfSMBLLOYXVuEAAAAASUVORK5CYII=
------=_Part_411_1980921947.1458136883717--

------=_Part_410_16936214.1458136883717--