Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kmQ7M-0002RV-40 for pgsql-hackers@arkaria.postgresql.org; Mon, 07 Dec 2020 23:40:32 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kmQ7L-0005uK-2k for pgsql-hackers@arkaria.postgresql.org; Mon, 07 Dec 2020 23:40:31 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kmQ7K-0005uD-Hx for pgsql-hackers@lists.postgresql.org; Mon, 07 Dec 2020 23:40:30 +0000 Received: from mail-wr1-x42b.google.com ([2a00:1450:4864:20::42b]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1kmQ7H-0000Q4-Pu for pgsql-hackers@postgresql.org; Mon, 07 Dec 2020 23:40:29 +0000 Received: by mail-wr1-x42b.google.com with SMTP id t4so14428052wrr.12 for ; Mon, 07 Dec 2020 15:40:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lenstra.fr; s=google; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=Li7MbRHmAQcFjQ9kl7qDF5BGJrhNanmYM30NTvDtQlw=; b=Ej8IGyzlkzUApywhPT6g3Fg2UZzyV/Gno6DXgZBHzqOLYf0d4Y4XoVRsEqSc3GxMAq R8CKAsYuGxVSXqAjNubELU2N/OJcZU6G6x5zN42kORE30Yi6ryZLWFQ3unZZpp3EA1V4 mnXO/2KlRY26eD5ZrKUUBzsi6MgKACzWOReLTwn5Dl1l2+/HtYsBhxBSn/YzXjUjO+hx C6xyG+9mV72/HIc0u+FxlPjGtbsPBmz0pqHY1rNHLqeX8VEAmHYnNoferjd7NEQeLehz doafa+M3EiGlvL+kNaWvB023yzV/LCKiOfZqkseFwgRHzrHKwRNHZLnHnsQflgkm44fZ /OiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=Li7MbRHmAQcFjQ9kl7qDF5BGJrhNanmYM30NTvDtQlw=; b=XwYq8ZdkAsg51pnxiKavEcT7fBer3oWn1UYPBu7awUq6mhWKnXWY5wiTN+N8SGjVAD bfzbSHpAdX9Yrpd3vNq6tbWEgIfcfrTcvgYo4hMyNFlfzYAao9mDW7jOExFO5fnXuUtZ LLQpqsJuEjAoz2keyNJC+DB13MtjRcO1P2B8D7Y20ua4k0t5o0CJlI4qcY5qDMqMg+QI 3mWdBMUzC3BDaFSBGAZLZwmKnGL33K+tCLNC+mgSHVUgIiMYCU9NVs6R2wG1KkstnHr4 g5DQbrZxo1YifC+m/LjLIEf4RKYjzD+tonGpDdtoSnZu7B8q35gxpmdn5GFw/f316+0w lm6g== X-Gm-Message-State: AOAM532OFHN3ql5bSZVmA/5e1ZOSKbcWQ6/wP2dz25/1/v4oMDJuCMJR qSeUHwP0WWd+9R1fthRaBtYryQ== X-Google-Smtp-Source: ABdhPJxpGe3A+0vas5VPj+OSoX11+UbtKZD6c0MthxwnelJBfmItRRvMsJJraU+guZ3bO/SPs1SAZg== X-Received: by 2002:adf:e512:: with SMTP id j18mr7080060wrm.52.1607384425909; Mon, 07 Dec 2020 15:40:25 -0800 (PST) Received: from [192.168.1.66] (176-189-146-38.abo.bbox.fr. [176.189.146.38]) by smtp.gmail.com with ESMTPSA id 138sm901790wma.41.2020.12.07.15.40.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 15:40:24 -0800 (PST) From: =?utf-8?Q?R=C3=A9mi_Lapeyre?= Message-Id: <9B3E4BEB-F6E2-4D4D-AFB8-58FB3957EB35@lenstra.fr> Content-Type: multipart/mixed; boundary="Apple-Mail=_EE29381D-BD82-4479-96AF-5AABBB7EEC9C" Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.20.0.2.21\)) Subject: Re: Add header support to text format and matching feature Date: Tue, 8 Dec 2020 00:40:24 +0100 In-Reply-To: <018819f7-6cbe-4f20-903f-de1b9e434119@manitou-mail.org> Cc: Michael Paquier , vignesh C , PostgreSQL-development To: Daniel Verite References: <018819f7-6cbe-4f20-903f-de1b9e434119@manitou-mail.org> X-Mailer: Apple Mail (2.3654.20.0.2.21) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --Apple-Mail=_EE29381D-BD82-4479-96AF-5AABBB7EEC9C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, here=E2=80=99s a rebased version of the patch. Best regards, R=C3=A9mi --Apple-Mail=_EE29381D-BD82-4479-96AF-5AABBB7EEC9C Content-Disposition: attachment; filename=v7-0001-Add-header-support-to-COPY-TO-text-format.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v7-0001-Add-header-support-to-COPY-TO-text-format.patch" Content-Transfer-Encoding: quoted-printable =46rom=20b919ac8aa8fffdf8b50728817fa9a3210887a1dd=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20=3D?UTF-8?q?R=3DC3=3DA9mi=3D20Lapeyre?=3D=20= =0ADate:=20Fri,=2017=20Jul=202020=2001:50:06=20= +0200=0ASubject:=20[PATCH=20v7=201/2]=20Add=20header=20support=20to=20= "COPY=20TO"=20text=20format=0AMIME-Version:=201.0=0AContent-Type:=20= multipart/mixed;=20boundary=3D"------------2.29.2"=0A=0AThis=20is=20a=20= multi-part=20message=20in=20MIME=20format.=0A--------------2.29.2=0A= Content-Type:=20text/plain;=20charset=3DUTF-8;=20format=3Dfixed=0A= Content-Transfer-Encoding:=208bit=0A=0A=0ACSV=20format=20supports=20the=20= HEADER=20option=20to=20output=20a=20header=20in=20the=20output,=0Ait=20= is=20convenient=20when=20other=20programs=20need=20to=20consume=20the=20= output.=20This=0Apatch=20adds=20the=20same=20option=20to=20the=20default=20= text=20format.=0A=0ADiscussion:=20= https://www.postgresql.org/message-id/flat/CAF1-J-0PtCWMeLtswwGV2M70U26n4g= 33gpe1rcKQqe6wVQDrFA@mail.gmail.com=0A---=0A=20= contrib/file_fdw/input/file_fdw.source=20=20|=20=201=20-=0A=20= contrib/file_fdw/output/file_fdw.source=20|=20=204=20+---=0A=20= doc/src/sgml/ref/copy.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20=20= 3=20++-=0A=20src/backend/commands/copy.c=20=20=20=20=20=20=20=20=20=20=20= =20=20|=20=204=20++--=0A=20src/backend/commands/copyto.c=20=20=20=20=20=20= =20=20=20=20=20|=20=205=20++++-=0A=20src/include/commands/copy.h=20=20=20= =20=20=20=20=20=20=20=20=20=20|=20=202=20+-=0A=20= src/test/regress/input/copy.source=20=20=20=20=20=20|=2012=20= ++++++++++++=0A=20src/test/regress/output/copy.source=20=20=20=20=20|=20=20= 8=20++++++++=0A=208=20files=20changed,=2030=20insertions(+),=209=20= deletions(-)=0A=0A=0A--------------2.29.2=0AContent-Type:=20= text/x-patch;=20= name=3D"v7-0001-Add-header-support-to-COPY-TO-text-format.patch"=0A= Content-Transfer-Encoding:=208bit=0AContent-Disposition:=20attachment;=20= filename=3D"v7-0001-Add-header-support-to-COPY-TO-text-format.patch"=0A=0A= diff=20--git=20a/contrib/file_fdw/input/file_fdw.source=20= b/contrib/file_fdw/input/file_fdw.source=0Aindex=20= 45b728eeb3..83edb71077=20100644=0A---=20= a/contrib/file_fdw/input/file_fdw.source=0A+++=20= b/contrib/file_fdw/input/file_fdw.source=0A@@=20-37,7=20+37,6=20@@=20= CREATE=20USER=20MAPPING=20FOR=20regress_no_priv_user=20SERVER=20= file_server;=0A=20=0A=20--=20validator=20tests=0A=20CREATE=20FOREIGN=20= TABLE=20tbl=20()=20SERVER=20file_server=20OPTIONS=20(format=20'xml');=20=20= --=20ERROR=0A-CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20file_server=20= OPTIONS=20(format=20'text',=20header=20'true');=20=20=20=20=20=20--=20= ERROR=0A=20CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20file_server=20= OPTIONS=20(format=20'text',=20quote=20':');=20=20=20=20=20=20=20=20=20=20= --=20ERROR=0A=20CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20= file_server=20OPTIONS=20(format=20'text',=20escape=20':');=20=20=20=20=20= =20=20=20=20--=20ERROR=0A=20CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20= file_server=20OPTIONS=20(format=20'binary',=20header=20'true');=20=20=20=20= --=20ERROR=0Adiff=20--git=20a/contrib/file_fdw/output/file_fdw.source=20= b/contrib/file_fdw/output/file_fdw.source=0Aindex=20= 52b4d5f1df..547b81fd16=20100644=0A---=20= a/contrib/file_fdw/output/file_fdw.source=0A+++=20= b/contrib/file_fdw/output/file_fdw.source=0A@@=20-33,14=20+33,12=20@@=20= CREATE=20USER=20MAPPING=20FOR=20regress_no_priv_user=20SERVER=20= file_server;=0A=20--=20validator=20tests=0A=20CREATE=20FOREIGN=20TABLE=20= tbl=20()=20SERVER=20file_server=20OPTIONS=20(format=20'xml');=20=20--=20= ERROR=0A=20ERROR:=20=20COPY=20format=20"xml"=20not=20recognized=0A= -CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20file_server=20OPTIONS=20= (format=20'text',=20header=20'true');=20=20=20=20=20=20--=20ERROR=0A= -ERROR:=20=20COPY=20HEADER=20available=20only=20in=20CSV=20mode=0A=20= CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20file_server=20OPTIONS=20= (format=20'text',=20quote=20':');=20=20=20=20=20=20=20=20=20=20--=20= ERROR=0A=20ERROR:=20=20COPY=20quote=20available=20only=20in=20CSV=20mode=0A= =20CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20file_server=20OPTIONS=20= (format=20'text',=20escape=20':');=20=20=20=20=20=20=20=20=20--=20ERROR=0A= =20ERROR:=20=20COPY=20escape=20available=20only=20in=20CSV=20mode=0A=20= CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20file_server=20OPTIONS=20= (format=20'binary',=20header=20'true');=20=20=20=20--=20ERROR=0A-ERROR:=20= =20COPY=20HEADER=20available=20only=20in=20CSV=20mode=0A+ERROR:=20=20= COPY=20HEADER=20available=20only=20in=20CSV=20and=20text=20mode=0A=20= CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20file_server=20OPTIONS=20= (format=20'binary',=20quote=20':');=20=20=20=20=20=20=20=20--=20ERROR=0A=20= ERROR:=20=20COPY=20quote=20available=20only=20in=20CSV=20mode=0A=20= CREATE=20FOREIGN=20TABLE=20tbl=20()=20SERVER=20file_server=20OPTIONS=20= (format=20'binary',=20escape=20':');=20=20=20=20=20=20=20--=20ERROR=0A= diff=20--git=20a/doc/src/sgml/ref/copy.sgml=20= b/doc/src/sgml/ref/copy.sgml=0Aindex=20369342b74d..fcab594f09=20100644=0A= ---=20a/doc/src/sgml/ref/copy.sgml=0A+++=20b/doc/src/sgml/ref/copy.sgml=0A= @@=20-271,7=20+271,8=20@@=20COPY=20{=20table_name=20[=20(=20CSV=20= format.=0A+=20=20=20=20=20=20This=20option=20is=20allowed=20only=20when=20= using=20CSV=20or=0A+=20=20=20=20=20=20= text=20format.=0A=20=20=20=20=20=20=0A=20=20=20= =20=20=0A=20=20=20=20=0Adiff=20--git=20= a/src/backend/commands/copy.c=20b/src/backend/commands/copy.c=0Aindex=20= b6143b8bf2..36fa4c0c74=20100644=0A---=20a/src/backend/commands/copy.c=0A= +++=20b/src/backend/commands/copy.c=0A@@=20-591,10=20+591,10=20@@=20= ProcessCopyOptions(ParseState=20*pstate,=0A=20=09=09=09=09=20= errmsg("COPY=20delimiter=20cannot=20be=20\"%s\"",=20opts_out->delim)));=0A= =20=0A=20=09/*=20Check=20header=20*/=0A-=09if=20(!opts_out->csv_mode=20= &&=20opts_out->header_line)=0A+=09if=20(opts_out->binary=20&&=20= opts_out->header_line)=0A=20=09=09ereport(ERROR,=0A=20=09=09=09=09= (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),=0A-=09=09=09=09=20errmsg("COPY=20= HEADER=20available=20only=20in=20CSV=20mode")));=0A+=09=09=09=09=20= errmsg("COPY=20HEADER=20available=20only=20in=20CSV=20and=20text=20= mode")));=0A=20=0A=20=09/*=20Check=20quote=20*/=0A=20=09if=20= (!opts_out->csv_mode=20&&=20opts_out->quote=20!=3D=20NULL)=0Adiff=20= --git=20a/src/backend/commands/copyto.c=20= b/src/backend/commands/copyto.c=0Aindex=20c7e5f04446..568b2ca46c=20= 100644=0A---=20a/src/backend/commands/copyto.c=0A+++=20= b/src/backend/commands/copyto.c=0A@@=20-912,8=20+912,11=20@@=20= CopyTo(CopyToState=20cstate)=0A=20=0A=20=09=09=09=09colname=20=3D=20= NameStr(TupleDescAttr(tupDesc,=20attnum=20-=201)->attname);=0A=20=0A-=09=09= =09=09CopyAttributeOutCSV(cstate,=20colname,=20false,=0A+=09=09=09=09if=20= (cstate->opts.csv_mode)=0A+=09=09=09=09=09CopyAttributeOutCSV(cstate,=20= colname,=20false,=0A=20=09=09=09=09=09=09=09=09=09= list_length(cstate->attnumlist)=20=3D=3D=201);=0A+=09=09=09=09else=0A+=09= =09=09=09=09CopyAttributeOutText(cstate,=20colname);=0A=20=09=09=09}=0A=20= =0A=20=09=09=09CopySendEndOfRow(cstate);=0Adiff=20--git=20= a/src/include/commands/copy.h=20b/src/include/commands/copy.h=0Aindex=20= 127a3c61e2..d115f72559=20100644=0A---=20a/src/include/commands/copy.h=0A= +++=20b/src/include/commands/copy.h=0A@@=20-32,7=20+32,7=20@@=20typedef=20= struct=20CopyFormatOptions=0A=20=09bool=09=09binary;=09=09=09/*=20binary=20= format?=20*/=0A=20=09bool=09=09freeze;=09=09=09/*=20freeze=20rows=20on=20= loading?=20*/=0A=20=09bool=09=09csv_mode;=09=09/*=20Comma=20Separated=20= Value=20format?=20*/=0A-=09bool=09=09header_line;=09/*=20CSV=20header=20= line?=20*/=0A+=09bool=09=09header_line;=09/*=20CSV=20or=20text=20header=20= line?=20*/=0A=20=09char=09=20=20=20*null_print;=09=09/*=20NULL=20marker=20= string=20(server=20encoding!)=20*/=0A=20=09int=09=09=09null_print_len;=20= /*=20length=20of=20same=20*/=0A=20=09char=09=20=20=20*null_print_client;=09= /*=20same=20converted=20to=20file=20encoding=20*/=0Adiff=20--git=20= a/src/test/regress/input/copy.source=20= b/src/test/regress/input/copy.source=0Aindex=20a1d529ad36..2368649111=20= 100644=0A---=20a/src/test/regress/input/copy.source=0A+++=20= b/src/test/regress/input/copy.source=0A@@=20-134,6=20+134,18=20@@=20this=20= is=20just=20a=20line=20full=20of=20junk=20that=20would=20error=20out=20= if=20parsed=0A=20=0A=20copy=20copytest3=20to=20stdout=20csv=20header;=0A=20= =0A+create=20temp=20table=20copytest4=20(=0A+=09c1=20int,=0A+=09"col=20= with=20tabulation:=20=09"=20text);=0A+=0A+copy=20copytest4=20from=20= stdin=20(header);=0A+this=20is=20just=20a=20line=20full=20of=20junk=20= that=20would=20error=20out=20if=20parsed=0A+1=09a=0A+2=09b=0A+\.=0A+=0A= +copy=20copytest4=20to=20stdout=20(header);=0A+=0A=20--=20test=20copy=20= from=20with=20a=20partitioned=20table=0A=20create=20table=20= parted_copytest=20(=0A=20=09a=20int,=0Adiff=20--git=20= a/src/test/regress/output/copy.source=20= b/src/test/regress/output/copy.source=0Aindex=20938d3551da..c1f7f99747=20= 100644=0A---=20a/src/test/regress/output/copy.source=0A+++=20= b/src/test/regress/output/copy.source=0A@@=20-95,6=20+95,14=20@@=20copy=20= copytest3=20to=20stdout=20csv=20header;=0A=20c1,"col=20with=20,=20= comma","col=20with=20""=20quote"=0A=201,a,1=0A=202,b,2=0A+create=20temp=20= table=20copytest4=20(=0A+=09c1=20int,=0A+=09"col=20with=20tabulation:=20=09= "=20text);=0A+copy=20copytest4=20from=20stdin=20(header);=0A+copy=20= copytest4=20to=20stdout=20(header);=0A+c1=09col=20with=20tabulation:=20= \t=0A+1=09a=0A+2=09b=0A=20--=20test=20copy=20from=20with=20a=20= partitioned=20table=0A=20create=20table=20parted_copytest=20(=0A=20=09a=20= int,=0A=0A--------------2.29.2--=0A=0A=0A= --Apple-Mail=_EE29381D-BD82-4479-96AF-5AABBB7EEC9C Content-Disposition: attachment; filename=v7-0002-Add-header-matching-mode-to-COPY-FROM.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="v7-0002-Add-header-matching-mode-to-COPY-FROM.patch" Content-Transfer-Encoding: quoted-printable =46rom=20ae8618091ba27a5e8200fe9207239cae3c4679ae=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20=3D?UTF-8?q?R=3DC3=3DA9mi=3D20Lapeyre?=3D=20= =0ADate:=20Tue,=2013=20Oct=202020=2014:45:56=20= +0200=0ASubject:=20[PATCH=20v7=202/2]=20Add=20header=20matching=20mode=20= to=20"COPY=20FROM"=0AMIME-Version:=201.0=0AContent-Type:=20= multipart/mixed;=20boundary=3D"------------2.29.2"=0A=0AThis=20is=20a=20= multi-part=20message=20in=20MIME=20format.=0A--------------2.29.2=0A= Content-Type:=20text/plain;=20charset=3DUTF-8;=20format=3Dfixed=0A= Content-Transfer-Encoding:=208bit=0A=0A=0ACOPY=20FROM=20supports=20the=20= HEADER=20option=20to=20silently=20discard=20the=20header=20from=0Aa=20= CSV=20or=20text=20file.=20It=20is=20possible=20to=20load=20by=20mistake=20= a=20file=20that=0Amatches=20the=20expected=20format,=20for=20example=20= if=20two=20text=20columns=20have=20been=0Aswapped,=20resulting=20in=20= garbage=20in=20the=20database.=0A=0AThis=20option=20adds=20the=20= possibility=20to=20actually=20check=20the=20header=20to=20make=0Asure=20= it=20matches=20what=20is=20expected=20and=20exit=20immediatly=20if=20it=20= does=20not.=0A=0ADiscussion:=20= https://www.postgresql.org/message-id/flat/CAF1-J-0PtCWMeLtswwGV2M70U26n4g= 33gpe1rcKQqe6wVQDrFA@mail.gmail.com=0A---=0A=20= contrib/file_fdw/input/file_fdw.source=20=20|=20=206=20+++=0A=20= contrib/file_fdw/output/file_fdw.source=20|=20=209=20+++-=0A=20= doc/src/sgml/ref/copy.sgml=20=20=20=20=20=20=20=20=20=20=20=20=20=20|=20= 12=20+++--=0A=20src/backend/commands/copy.c=20=20=20=20=20=20=20=20=20=20= =20=20=20|=2065=20+++++++++++++++++++++++--=0A=20= src/backend/commands/copyfromparse.c=20=20=20=20|=2050=20= +++++++++++++++++--=0A=20src/backend/commands/copyto.c=20=20=20=20=20=20=20= =20=20=20=20|=20=202=20+-=0A=20src/include/commands/copy.h=20=20=20=20=20= =20=20=20=20=20=20=20=20|=2012=20++++-=0A=20= src/test/regress/input/copy.source=20=20=20=20=20=20|=2025=20++++++++++=0A= =20src/test/regress/output/copy.source=20=20=20=20=20|=2017=20+++++++=0A=20= 9=20files=20changed,=20184=20insertions(+),=2014=20deletions(-)=0A=0A=0A= --------------2.29.2=0AContent-Type:=20text/x-patch;=20= name=3D"v7-0002-Add-header-matching-mode-to-COPY-FROM.patch"=0A= Content-Transfer-Encoding:=208bit=0AContent-Disposition:=20attachment;=20= filename=3D"v7-0002-Add-header-matching-mode-to-COPY-FROM.patch"=0A=0A= diff=20--git=20a/contrib/file_fdw/input/file_fdw.source=20= b/contrib/file_fdw/input/file_fdw.source=0Aindex=20= 83edb71077..7a3983c785=20100644=0A---=20= a/contrib/file_fdw/input/file_fdw.source=0A+++=20= b/contrib/file_fdw/input/file_fdw.source=0A@@=20-79,6=20+79,12=20@@=20= CREATE=20FOREIGN=20TABLE=20agg_bad=20(=0A=20OPTIONS=20(format=20'csv',=20= filename=20'@abs_srcdir@/data/agg.bad',=20header=20'true',=20delimiter=20= ';',=20quote=20'@',=20escape=20'"',=20null=20'');=0A=20ALTER=20FOREIGN=20= TABLE=20agg_bad=20ADD=20CHECK=20(a=20>=3D=200);=0A=20=0A+--=20test=20= header=20matching=0A+CREATE=20FOREIGN=20TABLE=20header_match=20("1"=20= int,=20foo=20text)=20SERVER=20file_server=0A+OPTIONS=20(format=20'csv',=20= filename=20'@abs_srcdir@/data/list1.csv',=20delimiter=20',',=20header=20= 'match');=0A+CREATE=20FOREIGN=20TABLE=20header_dont_match=20(a=20int,=20= foo=20text)=20SERVER=20file_server=0A+OPTIONS=20(format=20'csv',=20= filename=20'@abs_srcdir@/data/list1.csv',=20delimiter=20',',=20header=20= 'match');=09--=20ERROR=0A+=0A=20--=20per-column=20options=20tests=0A=20= CREATE=20FOREIGN=20TABLE=20text_csv=20(=0A=20=20=20=20=20word1=20text=20= OPTIONS=20(force_not_null=20'true'),=0Adiff=20--git=20= a/contrib/file_fdw/output/file_fdw.source=20= b/contrib/file_fdw/output/file_fdw.source=0Aindex=20= 547b81fd16..ebe826b9f4=20100644=0A---=20= a/contrib/file_fdw/output/file_fdw.source=0A+++=20= b/contrib/file_fdw/output/file_fdw.source=0A@@=20-93,6=20+93,11=20@@=20= CREATE=20FOREIGN=20TABLE=20agg_bad=20(=0A=20)=20SERVER=20file_server=0A=20= OPTIONS=20(format=20'csv',=20filename=20'@abs_srcdir@/data/agg.bad',=20= header=20'true',=20delimiter=20';',=20quote=20'@',=20escape=20'"',=20= null=20'');=0A=20ALTER=20FOREIGN=20TABLE=20agg_bad=20ADD=20CHECK=20(a=20= >=3D=200);=0A+--=20test=20header=20matching=0A+CREATE=20FOREIGN=20TABLE=20= header_match=20("1"=20int,=20foo=20text)=20SERVER=20file_server=0A= +OPTIONS=20(format=20'csv',=20filename=20'@abs_srcdir@/data/list1.csv',=20= delimiter=20',',=20header=20'match');=0A+CREATE=20FOREIGN=20TABLE=20= header_dont_match=20(a=20int,=20foo=20text)=20SERVER=20file_server=0A= +OPTIONS=20(format=20'csv',=20filename=20'@abs_srcdir@/data/list1.csv',=20= delimiter=20',',=20header=20'match');=09--=20ERROR=0A=20--=20per-column=20= options=20tests=0A=20CREATE=20FOREIGN=20TABLE=20text_csv=20(=0A=20=20=20=20= =20word1=20text=20OPTIONS=20(force_not_null=20'true'),=0A@@=20-439,12=20= +444,14=20@@=20SET=20ROLE=20regress_file_fdw_superuser;=0A=20--=20= cleanup=0A=20RESET=20ROLE;=0A=20DROP=20EXTENSION=20file_fdw=20CASCADE;=0A= -NOTICE:=20=20drop=20cascades=20to=207=20other=20objects=0A+NOTICE:=20=20= drop=20cascades=20to=209=20other=20objects=0A=20DETAIL:=20=20drop=20= cascades=20to=20server=20file_server=0A=20drop=20cascades=20to=20user=20= mapping=20for=20regress_file_fdw_superuser=20on=20server=20file_server=0A= =20drop=20cascades=20to=20user=20mapping=20for=20regress_no_priv_user=20= on=20server=20file_server=0A=20drop=20cascades=20to=20foreign=20table=20= agg_text=0A=20drop=20cascades=20to=20foreign=20table=20agg_csv=0A=20drop=20= cascades=20to=20foreign=20table=20agg_bad=0A+drop=20cascades=20to=20= foreign=20table=20header_match=0A+drop=20cascades=20to=20foreign=20table=20= header_dont_match=0A=20drop=20cascades=20to=20foreign=20table=20text_csv=0A= =20DROP=20ROLE=20regress_file_fdw_superuser,=20regress_file_fdw_user,=20= regress_no_priv_user;=0Adiff=20--git=20a/doc/src/sgml/ref/copy.sgml=20= b/doc/src/sgml/ref/copy.sgml=0Aindex=20fcab594f09..a804d0c35b=20100644=0A= ---=20a/doc/src/sgml/ref/copy.sgml=0A+++=20b/doc/src/sgml/ref/copy.sgml=0A= @@=20-36,7=20+36,7=20@@=20COPY=20{=20table_name=20[=20(=20boolean=20]=0A=20=20=20=20=20DELIMITER=20= 'delimiter_character'=0A= =20=20=20=20=20NULL=20'null_string'=0A-=20=20=20=20HEADER=20[=20= boolean=20]=0A+=20=20=20= =20HEADER=20[=20match=20|=20true=20= |=20false=20]=0A=20=20=20=20=20QUOTE=20'quote_character'=0A=20=20=20=20=20= ESCAPE=20'escape_character'=0A=20=20=20=20=20= FORCE_QUOTE=20{=20(=20column_name=20[,=20...]=20)=20|=20*=20= }=0A@@=20-270,9=20+270,13=20@@=20COPY=20{=20table_name=20[=20(=20=0A=20=20=20=20=20=20=20Specifies=20that=20the=20= file=20contains=20a=20header=20line=20with=20the=20names=20of=20each=0A=20= =20=20=20=20=20=20column=20in=20the=20file.=20=20On=20output,=20the=20= first=20line=20contains=20the=20column=0A-=20=20=20=20=20=20names=20from=20= the=20table,=20and=20on=20input,=20the=20first=20line=20is=20ignored.=0A= -=20=20=20=20=20=20This=20option=20is=20allowed=20only=20when=20using=20= CSV=20or=0A-=20=20=20=20=20=20text=20= format.=0A+=20=20=20=20=20=20names=20from=20the=20table.=20On=20input,=20= the=20first=20line=20is=20discarded=20when=0A+=20=20=20=20=20=20= header=20is=20set=20to=20true=20or=20= required=0A+=20=20=20=20=20=20to=20match=20the=20column=20names=20if=20= set=20to=20match.=20If=20the=0A+=20=20=20=20=20=20= number=20of=20columns=20in=20the=20header=20is=20not=20correct,=20their=20= order=20differs=0A+=20=20=20=20=20=20from=20the=20one=20expected,=20or=20= the=20name=20or=20case=20do=20not=20match,=20the=20copy=20will=0A+=20=20=20= =20=20=20be=20aborted=20with=20an=20error.=20=20This=20option=20is=20= allowed=20only=20when=20using=0A+=20=20=20=20=20=20= CSV=20or=20text=20format.=0A=20=20=20= =20=20=20=0A=20=20=20=20=20=0A=20=20=20=20= =0Adiff=20--git=20a/src/backend/commands/copy.c=20= b/src/backend/commands/copy.c=0Aindex=2036fa4c0c74..0ffa9112fa=20100644=0A= ---=20a/src/backend/commands/copy.c=0A+++=20= b/src/backend/commands/copy.c=0A@@=20-315,7=20+315,66=20@@=20= DoCopy(ParseState=20*pstate,=20const=20CopyStmt=20*stmt,=0A=20}=0A=20=0A=20= /*=0A-=20*=20Process=20the=20statement=20option=20list=20for=20COPY.=0A= +*=20Extract=20a=20CopyHeader=20value=20from=20a=20DefElem.=0A+*/=0A= +static=20CopyHeader=0A+DefGetCopyHeader(DefElem=20*def)=0A+{=0A+=09/*=0A= +=09*=20If=20no=20parameter=20given,=20assume=20"true"=20is=20meant.=0A+=09= */=0A+=09if=20(def->arg=20=3D=3D=20NULL)=0A+=09=09return=20= COPY_HEADER_PRESENT;=0A+=0A+=09/*=0A+=09*=20Allow=200,=201,=20"true",=20= "false",=20"on",=20"off"=20or=20"match".=0A+=09*/=0A+=09switch=20= (nodeTag(def->arg))=0A+=09{=0A+=09=09case=20T_Integer:=0A+=09=09=09= switch=20(intVal(def->arg))=0A+=09=09=09{=0A+=09=09=09=09case=200:=0A+=09= =09=09=09=09return=20COPY_HEADER_ABSENT;=0A+=09=09=09=09case=201:=0A+=09=09= =09=09=09return=20COPY_HEADER_PRESENT;=0A+=09=09=09=09default:=0A+=09=09=09= =09=09/*=20otherwise,=20error=20out=20below=20*/=0A+=09=09=09=09=09= break;=0A+=09=09=09}=0A+=09=09=09break;=0A+=09=09default:=0A+=09=09=09{=0A= +=09=09=09=09char=09*sval=20=3D=20defGetString(def);=0A+=0A+=09=09=09=09= /*=0A+=09=09=09=09*=20The=20set=20of=20strings=20accepted=20here=20= should=20match=20up=20with=20the=0A+=09=09=09=09*=20grammar's=20= opt_boolean_or_string=20production.=0A+=09=09=09=09*/=0A+=09=09=09=09if=20= (pg_strcasecmp(sval,=20"true")=20=3D=3D=200)=0A+=09=09=09=09=09=09return=20= COPY_HEADER_PRESENT;=0A+=09=09=09=09if=20(pg_strcasecmp(sval,=20"false")=20= =3D=3D=200)=0A+=09=09=09=09=09=09return=20COPY_HEADER_ABSENT;=0A+=09=09=09= =09if=20(pg_strcasecmp(sval,=20"on")=20=3D=3D=200)=0A+=09=09=09=09=09=09= return=20COPY_HEADER_PRESENT;=0A+=09=09=09=09if=20(pg_strcasecmp(sval,=20= "off")=20=3D=3D=200)=0A+=09=09=09=09=09=09return=20COPY_HEADER_ABSENT;=0A= +=09=09=09=09if=20(pg_strcasecmp(sval,=20"match")=20=3D=3D=200)=0A+=09=09= =09=09=09=09return=20COPY_HEADER_MATCH;=0A+=0A+=09=09=09}=0A+=09=09=09= break;=0A+=09}=0A+=0A+=09ereport(ERROR,=0A+=09=09=09=09= (errcode(ERRCODE_SYNTAX_ERROR),=0A+=09=09=09=09=20errmsg("%s=20requires=20= a=20boolean=20or=20\"match\"",=0A+=09=09=09=09=09def->defname)));=0A+=09= return=20COPY_HEADER_ABSENT;=09=09=09=09=09=09/*=20keep=20compiler=20= quiet=20*/=0A+}=0A+=0A+/*=0A+*=20Process=20the=20statement=20option=20= list=20for=20COPY.=0A=20=20*=0A=20=20*=20Scan=20the=20options=20list=20= (a=20list=20of=20DefElem)=20and=20transpose=20the=20information=0A=20=20= *=20into=20*opts_out,=20applying=20appropriate=20error=20checking.=0A@@=20= -410,7=20+469,7=20@@=20ProcessCopyOptions(ParseState=20*pstate,=0A=20=09=09= =09=09=09=09=20errmsg("conflicting=20or=20redundant=20options"),=0A=20=09= =09=09=09=09=09=20parser_errposition(pstate,=20defel->location)));=0A=20=09= =09=09header_specified=20=3D=20true;=0A-=09=09=09opts_out->header_line=20= =3D=20defGetBoolean(defel);=0A+=09=09=09opts_out->header_line=20=3D=20= DefGetCopyHeader(defel);=0A=20=09=09}=0A=20=09=09else=20if=20= (strcmp(defel->defname,=20"quote")=20=3D=3D=200)=0A=20=09=09{=0A@@=20= -591,7=20+650,7=20@@=20ProcessCopyOptions(ParseState=20*pstate,=0A=20=09=09= =09=09=20errmsg("COPY=20delimiter=20cannot=20be=20\"%s\"",=20= opts_out->delim)));=0A=20=0A=20=09/*=20Check=20header=20*/=0A-=09if=20= (opts_out->binary=20&&=20opts_out->header_line)=0A+=09if=20= (opts_out->binary=20&&=20opts_out->header_line=20!=3D=20= COPY_HEADER_ABSENT)=0A=20=09=09ereport(ERROR,=0A=20=09=09=09=09= (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),=0A=20=09=09=09=09=20= errmsg("COPY=20HEADER=20available=20only=20in=20CSV=20and=20text=20= mode")));=0Adiff=20--git=20a/src/backend/commands/copyfromparse.c=20= b/src/backend/commands/copyfromparse.c=0Aindex=2034ed3cfcd5..4a76e5fdd5=20= 100644=0A---=20a/src/backend/commands/copyfromparse.c=0A+++=20= b/src/backend/commands/copyfromparse.c=0A@@=20-26,6=20+26,7=20@@=0A=20= #include=20"mb/pg_wchar.h"=0A=20#include=20"miscadmin.h"=0A=20#include=20= "port/pg_bswap.h"=0A+#include=20"utils/builtins.h"=0A=20#include=20= "utils/memutils.h"=0A=20#include=20"utils/rel.h"=0A=20=0A@@=20-455,12=20= +456,53=20@@=20NextCopyFromRawFields(CopyFromState=20cstate,=20char=20= ***fields,=20int=20*nfields)=0A=20=09/*=20only=20available=20for=20text=20= or=20csv=20input=20*/=0A=20=09Assert(!cstate->opts.binary);=0A=20=0A-=09= /*=20on=20input=20just=20throw=20the=20header=20line=20away=20*/=0A-=09= if=20(cstate->cur_lineno=20=3D=3D=200=20&&=20cstate->opts.header_line)=0A= +=09/*=20on=20input=20check=20that=20the=20header=20line=20is=20correct=20= if=20needed=20*/=0A+=09if=20(cstate->cur_lineno=20=3D=3D=200=20&&=20= cstate->opts.header_line=20!=3D=20COPY_HEADER_ABSENT)=0A=20=09{=0A+=09=09= ListCell=20=20=20*cur;=0A+=09=09TupleDesc=20=20=20tupDesc;=0A+=0A+=09=09= tupDesc=20=3D=20RelationGetDescr(cstate->rel);=0A+=0A=20=09=09= cstate->cur_lineno++;=0A-=09=09if=20(CopyReadLine(cstate))=0A-=09=09=09= return=20false;=09=09/*=20done=20*/=0A+=09=09done=20=3D=20= CopyReadLine(cstate);=0A+=0A+=09=09if=20(cstate->opts.header_line=20=3D=3D= =20COPY_HEADER_MATCH)=0A+=09=09{=0A+=09=09=09if=20= (cstate->opts.csv_mode)=0A+=09=09=09=09fldct=20=3D=20= CopyReadAttributesCSV(cstate);=0A+=09=09=09else=0A+=09=09=09=09fldct=20=3D= =20CopyReadAttributesText(cstate);=0A+=0A+=09=09=09if=20(fldct=20<=20= list_length(cstate->attnumlist))=0A+=09=09=09=09ereport(ERROR,=0A+=09=09=09= =09=09=09(errcode(ERRCODE_BAD_COPY_FILE_FORMAT),=0A+=09=09=09=09=09=09=20= errmsg("missing=20header")));=0A+=09=09=09else=20if=20(fldct=20>=20= list_length(cstate->attnumlist))=0A+=09=09=09=09ereport(ERROR,=0A+=09=09=09= =09=09=09(errcode(ERRCODE_BAD_COPY_FILE_FORMAT),=0A+=09=09=09=09=09=09=20= errmsg("extra=20data=20after=20last=20expected=20header")));=0A+=0A+=09=09= =09foreach(cur,=20cstate->attnumlist)=0A+=09=09=09{=0A+=09=09=09=09int=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20attnum=20=3D=20lfirst_int(cur);=0A+=09=09=09=09char=20=20=20=20=20= =20=20=20=20=20=20=20=20=20*colName=20=3D=20cstate->raw_fields[attnum=20= -=201];=0A+=09=09=09=09Form_pg_attribute=20attr=20=3D=20= TupleDescAttr(tupDesc,=20attnum=20-=201);=0A+=0A+=09=09=09=09if=20= (colName=20=3D=3D=20NULL)=0A+=09=09=09=09=09colName=20=3D=20= cstate->opts.null_print;=0A+=0A+=09=09=09=09if=20= (namestrcmp(&attr->attname,=20colName)=20!=3D=200)=20{=0A+=09=09=09=09=09= ereport(ERROR,=0A+=09=09=09=09=09=09=09= (errcode(ERRCODE_BAD_COPY_FILE_FORMAT),=0A+=09=09=09=09=09=09=09=20= errmsg("wrong=20header=20for=20column=20\"%s\":=20got=20\"%s\"",=0A+=09=09= =09=09=09=09=09=09=09NameStr(attr->attname),=20colName)));=0A+=09=09=09=09= }=0A+=09=09=09}=0A+=09=09}=0A+=0A+=09=09if=20(done)=0A+=09=09=09return=20= false;=0A=20=09}=0A=20=0A=20=09cstate->cur_lineno++;=0Adiff=20--git=20= a/src/backend/commands/copyto.c=20b/src/backend/commands/copyto.c=0A= index=20568b2ca46c..3b441c408d=20100644=0A---=20= a/src/backend/commands/copyto.c=0A+++=20b/src/backend/commands/copyto.c=0A= @@=20-897,7=20+897,7=20@@=20CopyTo(CopyToState=20cstate)=0A=20=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=20cstate->file_encoding);=0A=20=0A=20=09=09= /*=20if=20a=20header=20has=20been=20requested=20send=20the=20line=20*/=0A= -=09=09if=20(cstate->opts.header_line)=0A+=09=09if=20= (cstate->opts.header_line=20!=3D=20COPY_HEADER_ABSENT)=0A=20=09=09{=0A=20= =09=09=09bool=09=09hdr_delim=20=3D=20false;=0A=20=0Adiff=20--git=20= a/src/include/commands/copy.h=20b/src/include/commands/copy.h=0Aindex=20= d115f72559..b163bc8eb5=20100644=0A---=20a/src/include/commands/copy.h=0A= +++=20b/src/include/commands/copy.h=0A@@=20-19,6=20+19,16=20@@=0A=20= #include=20"parser/parse_node.h"=0A=20#include=20"tcop/dest.h"=0A=20=0A= +/*=0A+=20*=20Represents=20whether=20the=20header=20must=20be=20absent,=20= present=20or=20present=20and=20match.=0A+=20*/=0A+typedef=20enum=20= CopyHeader=0A+{=0A+=09COPY_HEADER_ABSENT,=0A+=09COPY_HEADER_PRESENT,=0A+=09= COPY_HEADER_MATCH=0A+}=20CopyHeader;=0A+=0A=20/*=0A=20=20*=20A=20struct=20= to=20hold=20COPY=20options,=20in=20a=20parsed=20form.=20All=20of=20these=20= are=20related=0A=20=20*=20to=20formatting,=20except=20for=20'freeze',=20= which=20doesn't=20really=20belong=20here,=20but=0A@@=20-32,7=20+42,7=20= @@=20typedef=20struct=20CopyFormatOptions=0A=20=09bool=09=09binary;=09=09= =09/*=20binary=20format?=20*/=0A=20=09bool=09=09freeze;=09=09=09/*=20= freeze=20rows=20on=20loading?=20*/=0A=20=09bool=09=09csv_mode;=09=09/*=20= Comma=20Separated=20Value=20format?=20*/=0A-=09bool=09=09header_line;=09= /*=20CSV=20or=20text=20header=20line?=20*/=0A+=09CopyHeader=09= header_line;=09/*=20CSV=20or=20text=20header=20line?=20*/=0A=20=09char=09= =20=20=20*null_print;=09=09/*=20NULL=20marker=20string=20(server=20= encoding!)=20*/=0A=20=09int=09=09=09null_print_len;=20/*=20length=20of=20= same=20*/=0A=20=09char=09=20=20=20*null_print_client;=09/*=20same=20= converted=20to=20file=20encoding=20*/=0Adiff=20--git=20= a/src/test/regress/input/copy.source=20= b/src/test/regress/input/copy.source=0Aindex=202368649111..4d21c7d524=20= 100644=0A---=20a/src/test/regress/input/copy.source=0A+++=20= b/src/test/regress/input/copy.source=0A@@=20-213,3=20+213,28=20@@=20= select=20*=20from=20parted_copytest=20where=20b=20=3D=201;=0A=20select=20= *=20from=20parted_copytest=20where=20b=20=3D=202;=0A=20=0A=20drop=20= table=20parted_copytest;=0A+=0A+--=20Test=20header=20matching=20feature=0A= +create=20table=20header_copytest=20(=0A+=09a=20int,=0A+=09b=20int,=0A+=09= c=20text=0A+);=0A+copy=20header_copytest=20from=20stdin=20with=20(header=20= wrong_choice);=0A+copy=20header_copytest=20from=20stdin=20with=20(header=20= match);=0A+a=09b=09c=0A+1=092=09foo=0A+\.=0A+copy=20header_copytest=20= from=20stdin=20with=20(header=20match);=0A+a=09b=0A+1=092=0A+\.=0A+copy=20= header_copytest=20from=20stdin=20with=20(header=20match);=0A+a=09b=09c=09= d=0A+1=092=09foo=09bar=0A+\.=0A+copy=20header_copytest=20from=20stdin=20= with=20(header=20match,=20format=20csv);=0A+a,b,c=0A+1,2,foo=0A+\.=0A= +drop=20table=20header_copytest;=0Adiff=20--git=20= a/src/test/regress/output/copy.source=20= b/src/test/regress/output/copy.source=0Aindex=20c1f7f99747..b792181fe3=20= 100644=0A---=20a/src/test/regress/output/copy.source=0A+++=20= b/src/test/regress/output/copy.source=0A@@=20-173,3=20+173,20=20@@=20= select=20*=20from=20parted_copytest=20where=20b=20=3D=202;=0A=20(1=20= row)=0A=20=0A=20drop=20table=20parted_copytest;=0A+--=20Test=20header=20= matching=20feature=0A+create=20table=20header_copytest=20(=0A+=09a=20= int,=0A+=09b=20int,=0A+=09c=20text=0A+);=0A+copy=20header_copytest=20= from=20stdin=20with=20(header=20wrong_choice);=0A+ERROR:=20=20header=20= requires=20a=20boolean=20or=20"match"=0A+copy=20header_copytest=20from=20= stdin=20with=20(header=20match);=0A+copy=20header_copytest=20from=20= stdin=20with=20(header=20match);=0A+ERROR:=20=20missing=20header=0A= +CONTEXT:=20=20COPY=20header_copytest,=20line=201:=20"a=09b"=0A+copy=20= header_copytest=20from=20stdin=20with=20(header=20match);=0A+ERROR:=20=20= extra=20data=20after=20last=20expected=20header=0A+CONTEXT:=20=20COPY=20= header_copytest,=20line=201:=20"a=09b=09c=09d"=0A+copy=20header_copytest=20= from=20stdin=20with=20(header=20match,=20format=20csv);=0A+drop=20table=20= header_copytest;=0A=0A--------------2.29.2--=0A=0A=0A= --Apple-Mail=_EE29381D-BD82-4479-96AF-5AABBB7EEC9C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 21 Oct 2020, at 19:49, Daniel Verite = wrote: >=20 > R=C3=A9mi Lapeyre wrote: >=20 >> It looks like this is not in the current commitfest and that Cabot = does not >> find it. I=E2=80=99m not yet accustomed to the PostgreSQL workflow, = should I just >> create a new entry in the current commitfest? >=20 > Yes. Because in the last CommitFest it was marked > as "Returned with feedback" > https://commitfest.postgresql.org/29/2504/ >=20 >=20 > Best regards, > --=20 > Daniel V=C3=A9rit=C3=A9 > PostgreSQL-powered mailer: https://www.manitou-mail.org > Twitter: @DanielVerite >=20 >=20 >=20 >=20 --Apple-Mail=_EE29381D-BD82-4479-96AF-5AABBB7EEC9C--