From: Andrew Dunstan <andrew@dunslane.net>
To: Guillaume Lelarge <guillaume@lelarge.info>
Cc: Zheng Yang <zhengyang4k@gmail.com>
Cc: Shiv <rama.theone@gmail.com>
Cc: Selena Deckelmann <selena@chesnok.com>
Cc: Alvaro Herrera <alvherre@commandprompt.com>
Cc: pgsql-students <pgsql-students@postgresql.org>
Subject: Re: foreign data wrappers
Date: Tue, 29 Mar 2011 12:51:53 -0400
Message-ID: <4D920E29.2030706@dunslane.net> (raw)
In-Reply-To: <4D920A3D.8010004@lelarge.info>
References: <AANLkTi=FhaO5mX_YdqqsF1Yh0YNEHW8H1f0AUdpE0Fmo@mail.gmail.com>
<690867AB-C50B-4354-91D0-8A0811639C4C@gmail.com>
<AANLkTimKc4Jb4ydk=GO3shdv0gTPyad+y9z_kXXVKZoq@mail.gmail.com>
<4D879CD0.5030803@dunslane.net>
<AANLkTi=0KD_ef9XDJde52nb0n5YgysMHTjWwkKZurDR-@mail.gmail.com>
<3EF9E4CC-CF42-4B45-86FF-4CB9D46D7442@gmail.com>
<4D8AF8C2.3000007@lelarge.info>
<4ACC318A-1B6F-49A5-9B02-49EEAF7B97C7@gmail.com>
<1300985156-sup-3058@alvh.no-ip.org>
<47676.208.91.122.102.1300985713.squirrel@dunslane.net>
<AANLkTinCjkp2rxwivGVXZj5=bYpa4Bx=X0w6BWR=Vm0Q@mail.gmail.com>
<AANLkTi=NuFBQBhtEBuaPR9ZzhnqPCaVtpsuJ-eZ21GpF@mail.gmail.com>
<603812A0-F464-44AE-8A90-DF9BD1DC2DB9@gmail.com>
<4D8E0125.9060905@dunslane.net>
<8DB4D3E9-DA21-45FD-A2A2-5C6D479AA565@gmail.com>
<4D91FF65.2090401@lelarge.info>
<4D92098D.6040409@dunslane.net>
<4D920A3D.8010004@lelarge.info>
On 03/29/2011 12:35 PM, Guillaume Lelarge wrote:
> Le 29/03/2011 18:32, Andrew Dunstan a écrit :
>>
>> On 03/29/2011 11:48 AM, Guillaume Lelarge wrote:
>>> Le 29/03/2011 13:28, Zheng Yang a écrit :
>>>> Hi guys,
>>>>
>>>>>> I've briefly gone through the slides. Regarding the 6 callbacks, is
>>>>>> that correct to say that a full table scan will always be performed
>>>>>> irregardless of the sql statement,
>>>>>> the FDW is blind to the sql query performed, right?
>>>>> Yes, fairly much. If the feed is large you need some way to pass a
>>>>> limit to the foreign side, possibly via table options. I'm fairly
>>>>> sure you won't be able to get it via the SELECT statement.
>>>>>
>>>> Regarding the previous flickr example, I'm wondering how this 'free
>>>> text search' function can be done if the FDW is blind to the SELECT
>>>> statement.
>>>>
>>>> For instance, the following query is to retrieve a photo relevant to
>>>> 'panda':
>>>>
>>>> SELECT photo FROM flickr_table WHERE search LIKE '%panda%';
>>>>
>>>> In this case, the FDW can only open a connection to flickr web
>>>> service and return the next 'row' .
>>>> The problem is that there are a huge number of photos in flickr
>>>> server and retrieving them sequentially is not realistic.
>>>> Any ideas on how this can be done?
>>>>
>>> It probably means that flickr is not a good example of a nice fdw.
>>
>> Neither of you are being very creative. As I mentioned above, you need
>> to embed this sort of stuff in table options.
>>
>> so you would have something like:
>>
>> create foreign table panda_flickr (photo bytea, ...)
>> server flickr_server
>> options (searchterm 'panda', maxrows '50');
>> select photo from panda_flickr;
>>
> This would work but means you need to create a new foreign table to
> search something else.
>
> So, yeah, it works, but it's not convenient.
The other possibility is that you can dig down into the ForiegnScanState
object. The FDW routines are passed a ForeignScanState object which
contains a ScanState object which in turn contains a PlanState object
which has a list of quals. You probably need to dig quite a bit further
but that's a start.
cheers
andrew
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-general@postgresql.org
Cc: andrew@dunslane.net, guillaume@lelarge.info, zhengyang4k@gmail.com, rama.theone@gmail.com, selena@chesnok.com, alvherre@commandprompt.com, pgsql-students@postgresql.org
Subject: Re: foreign data wrappers
In-Reply-To: <4D920E29.2030706@dunslane.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox