Date: Fri, 2 Nov 2018 18:29:12 +0100
From: David Fetter <david@fetter.org>
To: Daniel Verite <daniel@manitou-mail.org>
Cc: Pavel Stehule <pavel.stehule@gmail.com>,
	"corey.huinker" <corey.huinker@gmail.com>, nasbyj@amazon.com,
	Surafel Temsgen <surafel3000@gmail.com>,
	Christoph Moench-Tegeder <cmt@burggraben.net>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: COPY FROM WHEN condition
Message-ID: <20181102172912.GK12677@fetter.org>
References: 
 <CAFj8pRCgNX9TUBD2SeoErMHs7bxb6xmxTpTFk1L_SqbEnxfinQ@mail.gmail.com>
 <b66941d6-fb91-47a0-9e9e-a06075ffb58d@manitou-mail.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <b66941d6-fb91-47a0-9e9e-a06075ffb58d@manitou-mail.org>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk

On Fri, Nov 02, 2018 at 12:58:12PM +0100, Daniel Verite wrote:
> 	Pavel Stehule wrote:
> 
> > > SELECT x.a, sum(x.b)
> > > FROM ( COPY INLINE '/path/to/foo.txt' FORMAT CSV ) as x( a integer, b
> > > numeric, c text, d date, e json) )
> > > WHERE x.d >= '2018-11-01'
> > >
> > >
> > Without some special feature this example is not extra useful. It is based
> > on copy on server that can use only super user with full rights.
> 
> And if superuser, one might use the file data wrapper [1]  to get
> the same results without the need for the explicit COPY in the query.
> 
> BTW, this brings into question whether the [WHEN condition] clause
> should be included in the options of file_fdw, as it supports pretty
> much all COPY options.
> 
> Also, psql's \copy should gain the option too?

tl;dr: \copy support is a very large can of worms.

psql's \copy is something which should probably be handled separately
from COPY, as it's both a way to access the filesystem without
superuser permission and an interface to the COPY part of the
protocol.  It seems like poor design to add grammar to support a
single client, so we'd need to think about this in terms of what we
want to support on the client side independent of specific clients. It
also seems like a violation of separation of concerns to couple FEBE
to grammar, so there'd need to be some way to do those things
separately, too.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate