Message-ID: <4D92098D.6040409@dunslane.net>
Date: Tue, 29 Mar 2011 12:32:13 -0400
From: Andrew Dunstan <andrew@dunslane.net>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US;
	rv:1.9.2.13) Gecko/20101209 Fedora/3.1.7-0.35.b3pre.fc13
	Lightning/1.0b3pre Thunderbird/3.1.7
MIME-Version: 1.0
To: Guillaume Lelarge <guillaume@lelarge.info>
CC: Zheng Yang <zhengyang4k@gmail.com>, Shiv <rama.theone@gmail.com>,
	Selena Deckelmann <selena@chesnok.com>,
	Alvaro Herrera <alvherre@commandprompt.com>,
	pgsql-students <pgsql-students@postgresql.org>
Subject: Re: foreign data wrappers
References: <AANLkTi=FhaO5mX_YdqqsF1Yh0YNEHW8H1f0AUdpE0Fmo@mail.gmail.com>
	<690867AB-C50B-4354-91D0-8A0811639C4C@gmail.com>
	<AANLkTimKc4Jb4ydk=GO3shdv0gTPyad+y9z_kXXVKZoq@mail.gmail.com>
	<4D879CD0.5030803@dunslane.net>
	<AANLkTi=0KD_ef9XDJde52nb0n5YgysMHTjWwkKZurDR-@mail.gmail.com>
	<3EF9E4CC-CF42-4B45-86FF-4CB9D46D7442@gmail.com>
	<4D8AF8C2.3000007@lelarge.info>
	<4ACC318A-1B6F-49A5-9B02-49EEAF7B97C7@gmail.com>
	<1300985156-sup-3058@alvh.no-ip.org>
	<47676.208.91.122.102.1300985713.squirrel@dunslane.net>
	<AANLkTinCjkp2rxwivGVXZj5=bYpa4Bx=X0w6BWR=Vm0Q@mail.gmail.com>
	<AANLkTi=NuFBQBhtEBuaPR9ZzhnqPCaVtpsuJ-eZ21GpF@mail.gmail.com>
	<603812A0-F464-44AE-8A90-DF9BD1DC2DB9@gmail.com>
	<4D8E0125.9060905@dunslane.net>
	<8DB4D3E9-DA21-45FD-A2A2-5C6D479AA565@gmail.com>
	<4D91FF65.2090401@lelarge.info>
In-Reply-To: <4D91FF65.2090401@lelarge.info>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit


On 03/29/2011 11:48 AM, Guillaume Lelarge wrote:
> Le 29/03/2011 13:28, Zheng Yang a écrit :
>> Hi guys,
>>
>>>> I've briefly gone through the slides. Regarding the 6 callbacks, is that correct to say that a full table scan will always be performed irregardless of the sql statement,
>>>> the FDW is blind to the sql query performed, right?
>>> Yes, fairly much. If the feed is large you need some way to pass a limit to the foreign side, possibly via table options. I'm fairly sure you won't be able to get it via the SELECT statement.
>>>
>>
>> Regarding the previous flickr example, I'm wondering how this 'free text search' function can be done if the FDW is blind to the SELECT statement.
>>
>> For instance, the following query is to retrieve a photo relevant to 'panda':
>>
>> 	SELECT photo FROM flickr_table WHERE search LIKE '%panda%';
>>
>> In this case, the FDW can only open a connection to flickr web service and return the next 'row' .
>> The problem is that there are a huge number of photos in flickr server and retrieving them sequentially is not realistic.
>> Any ideas on how this can be done?
>>
> It probably means that flickr is not a good example of a nice fdw.


Neither of you are being very creative. As I mentioned above, you need 
to embed this sort of stuff in table options.

so you would have something like:

    create foreign table panda_flickr (photo bytea, ...)
         server flickr_server
         options (searchterm 'panda', maxrows '50');
    select photo from panda_flickr;


cheers

andrew