public inbox for [email protected]
help / color / mirror / Atom feedFrom: Rémi Lapeyre <[email protected]>
To: Zhihong Yu <[email protected]>
Cc: David Steele <[email protected]>
Cc: Michael Paquier <[email protected]>
Cc: vignesh C <[email protected]>
Cc: PostgreSQL-development <[email protected]>
Cc: Daniel Verite <[email protected]>
Subject: Re: Add header support to text format and matching feature
Date: Fri, 31 Dec 2021 18:36:33 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
<[email protected]>
<[email protected]>
<[email protected]>
<CALNJ-vRUVcnSO_WDJS5RRjsh26jRK2rVsPoos+xeJ4Uu1fOtWw@mail.gmail.com>
<[email protected]>
<CALNJ-vRCei7pSp6-Atf3Zy3Y88UDzKEVTCxPxA4AVvUebxSfNg@mail.gmail.com>
<[email protected]>
Here’s an updated version of the patch that takes into account the changes in d1029bb5a2. The actual code is the same as v10 which was already marked as ready for committer.
> On 11 Apr 2021, at 16:35, Rémi Lapeyre <[email protected]> wrote:
>
>>
>>>> This now reads "Represents whether the header must be absent, present or match.”.
>>
>> Since match shouldn't be preceded with be, I think we can say:
>>
>> Represents whether the header must match, be absent or be present.
>
> Thanks, here’s a v10 version of the patch that fixes this.
>
> <v10-0001-Add-header-support-to-COPY-TO-text-format.patch><v10-0002-Add-header-matching-mode-to-COPY-FROM.patch>
Attachments:
[application/octet-stream] v11-0001-Add-header-support-to-COPY-TO-text-format.patch (7.4K, ../[email protected]/2-v11-0001-Add-header-support-to-COPY-TO-text-format.patch)
download | inline diff:
From 41b00d9fcb50b5209295020cf1dc679234e1d018 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?R=C3=A9mi=20Lapeyre?= <[email protected]>
Date: Fri, 17 Jul 2020 01:50:06 +0200
Subject: [PATCH v11 1/2] Add header support to "COPY TO" text format
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="------------2.34.1"
This is a multi-part message in MIME format.
--------------2.34.1
Content-Type: text/plain; charset=UTF-8; format=fixed
Content-Transfer-Encoding: 8bit
CSV format supports the HEADER option to output a header in the output,
it is convenient when other programs need to consume the output. This
patch adds the same option to the default text format.
Discussion: https://www.postgresql.org/message-id/flat/CAF1-J-0PtCWMeLtswwGV2M70U26n4g33gpe1rcKQqe6wVQDrFA@mail.gmail.com
---
contrib/file_fdw/expected/file_fdw.out | 4 +---
contrib/file_fdw/sql/file_fdw.sql | 1 -
doc/src/sgml/ref/copy.sgml | 3 ++-
src/backend/commands/copy.c | 4 ++--
src/backend/commands/copyto.c | 5 ++++-
src/include/commands/copy.h | 2 +-
src/test/regress/expected/copy.out | 8 ++++++++
src/test/regress/sql/copy.sql | 12 ++++++++++++
8 files changed, 30 insertions(+), 9 deletions(-)
--------------2.34.1
Content-Type: text/x-patch; name="v11-0001-Add-header-support-to-COPY-TO-text-format.patch"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="v11-0001-Add-header-support-to-COPY-TO-text-format.patch"
diff --git a/contrib/file_fdw/expected/file_fdw.out b/contrib/file_fdw/expected/file_fdw.out
index 891146fef3..ba6fdba598 100644
--- a/contrib/file_fdw/expected/file_fdw.out
+++ b/contrib/file_fdw/expected/file_fdw.out
@@ -50,14 +50,12 @@ CREATE USER MAPPING FOR regress_no_priv_user SERVER file_server;
-- validator tests
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'xml'); -- ERROR
ERROR: COPY format "xml" not recognized
-CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'text', header 'true'); -- ERROR
-ERROR: COPY HEADER available only in CSV mode
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'text', quote ':'); -- ERROR
ERROR: COPY quote available only in CSV mode
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'text', escape ':'); -- ERROR
ERROR: COPY escape available only in CSV mode
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'binary', header 'true'); -- ERROR
-ERROR: COPY HEADER available only in CSV mode
+ERROR: COPY HEADER available only in CSV and text mode
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'binary', quote ':'); -- ERROR
ERROR: COPY quote available only in CSV mode
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'binary', escape ':'); -- ERROR
diff --git a/contrib/file_fdw/sql/file_fdw.sql b/contrib/file_fdw/sql/file_fdw.sql
index 0ea8b14508..86f876d565 100644
--- a/contrib/file_fdw/sql/file_fdw.sql
+++ b/contrib/file_fdw/sql/file_fdw.sql
@@ -56,7 +56,6 @@ CREATE USER MAPPING FOR regress_no_priv_user SERVER file_server;
-- validator tests
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'xml'); -- ERROR
-CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'text', header 'true'); -- ERROR
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'text', quote ':'); -- ERROR
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'text', escape ':'); -- ERROR
CREATE FOREIGN TABLE tbl () SERVER file_server OPTIONS (format 'binary', header 'true'); -- ERROR
diff --git a/doc/src/sgml/ref/copy.sgml b/doc/src/sgml/ref/copy.sgml
index 4624c8f4c9..2d3847e0d1 100644
--- a/doc/src/sgml/ref/copy.sgml
+++ b/doc/src/sgml/ref/copy.sgml
@@ -277,7 +277,8 @@ COPY { <replaceable class="parameter">table_name</replaceable> [ ( <replaceable
Specifies that the file contains a header line with the names of each
column in the file. On output, the first line contains the column
names from the table, and on input, the first line is ignored.
- This option is allowed only when using <literal>CSV</literal> format.
+ This option is allowed only when using <literal>CSV</literal> or
+ <literal>text</literal> format.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 53f4853141..382a12cfb2 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -555,10 +555,10 @@ ProcessCopyOptions(ParseState *pstate,
errmsg("COPY delimiter cannot be \"%s\"", opts_out->delim)));
/* Check header */
- if (!opts_out->csv_mode && opts_out->header_line)
+ if (opts_out->binary && opts_out->header_line)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("COPY HEADER available only in CSV mode")));
+ errmsg("COPY HEADER available only in CSV and text mode")));
/* Check quote */
if (!opts_out->csv_mode && opts_out->quote != NULL)
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index b6eacd5baa..e9b962c7aa 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -863,8 +863,11 @@ DoCopyTo(CopyToState cstate)
colname = NameStr(TupleDescAttr(tupDesc, attnum - 1)->attname);
- CopyAttributeOutCSV(cstate, colname, false,
+ if (cstate->opts.csv_mode)
+ CopyAttributeOutCSV(cstate, colname, false,
list_length(cstate->attnumlist) == 1);
+ else
+ CopyAttributeOutText(cstate, colname);
}
CopySendEndOfRow(cstate);
diff --git a/src/include/commands/copy.h b/src/include/commands/copy.h
index 264895d278..c3f91cdc47 100644
--- a/src/include/commands/copy.h
+++ b/src/include/commands/copy.h
@@ -32,7 +32,7 @@ typedef struct CopyFormatOptions
bool binary; /* binary format? */
bool freeze; /* freeze rows on loading? */
bool csv_mode; /* Comma Separated Value format? */
- bool header_line; /* CSV header line? */
+ bool header_line; /* CSV or text header line? */
char *null_print; /* NULL marker string (server encoding!) */
int null_print_len; /* length of same */
char *null_print_client; /* same converted to file encoding */
diff --git a/src/test/regress/expected/copy.out b/src/test/regress/expected/copy.out
index 931e7b2e69..08de57ee69 100644
--- a/src/test/regress/expected/copy.out
+++ b/src/test/regress/expected/copy.out
@@ -120,6 +120,14 @@ copy copytest3 to stdout csv header;
c1,"col with , comma","col with "" quote"
1,a,1
2,b,2
+create temp table copytest4 (
+ c1 int,
+ "col with tabulation: " text);
+copy copytest4 from stdin (header);
+copy copytest4 to stdout (header);
+c1 col with tabulation: \t
+1 a
+2 b
-- test copy from with a partitioned table
create table parted_copytest (
a int,
diff --git a/src/test/regress/sql/copy.sql b/src/test/regress/sql/copy.sql
index 15e26517ec..6809f43e26 100644
--- a/src/test/regress/sql/copy.sql
+++ b/src/test/regress/sql/copy.sql
@@ -160,6 +160,18 @@ this is just a line full of junk that would error out if parsed
copy copytest3 to stdout csv header;
+create temp table copytest4 (
+ c1 int,
+ "col with tabulation: " text);
+
+copy copytest4 from stdin (header);
+this is just a line full of junk that would error out if parsed
+1 a
+2 b
+\.
+
+copy copytest4 to stdout (header);
+
-- test copy from with a partitioned table
create table parted_copytest (
a int,
--------------2.34.1--
[application/octet-stream] v11-0002-Add-header-matching-mode-to-COPY-FROM.patch (14.5K, ../[email protected]/3-v11-0002-Add-header-matching-mode-to-COPY-FROM.patch)
download | inline diff:
From 7c72199c4eb8e5e9d6a9fea79ecff1e85db543c4 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?R=C3=A9mi=20Lapeyre?= <[email protected]>
Date: Tue, 13 Oct 2020 14:45:56 +0200
Subject: [PATCH v11 2/2] Add header matching mode to "COPY FROM"
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="------------2.34.1"
This is a multi-part message in MIME format.
--------------2.34.1
Content-Type: text/plain; charset=UTF-8; format=fixed
Content-Transfer-Encoding: 8bit
COPY FROM supports the HEADER option to silently discard the header from
a CSV or text file. It is possible to load by mistake a file that
matches the expected format, for example if two text columns have been
swapped, resulting in garbage in the database.
This option adds the possibility to actually check the header to make
sure it matches what is expected and exit immediatly if it does not.
Discussion: https://www.postgresql.org/message-id/flat/CAF1-J-0PtCWMeLtswwGV2M70U26n4g33gpe1rcKQqe6wVQDrFA@mail.gmail.com
---
contrib/file_fdw/expected/file_fdw.out | 10 +++-
contrib/file_fdw/sql/file_fdw.sql | 7 +++
doc/src/sgml/ref/copy.sgml | 12 +++--
src/backend/commands/copy.c | 65 ++++++++++++++++++++++++--
src/backend/commands/copyfromparse.c | 52 +++++++++++++++++++--
src/backend/commands/copyto.c | 2 +-
src/include/commands/copy.h | 12 ++++-
src/test/regress/expected/copy.out | 17 +++++++
src/test/regress/sql/copy.sql | 25 ++++++++++
9 files changed, 188 insertions(+), 14 deletions(-)
--------------2.34.1
Content-Type: text/x-patch; name="v11-0002-Add-header-matching-mode-to-COPY-FROM.patch"
Content-Transfer-Encoding: 8bit
Content-Disposition: attachment; filename="v11-0002-Add-header-matching-mode-to-COPY-FROM.patch"
diff --git a/contrib/file_fdw/expected/file_fdw.out b/contrib/file_fdw/expected/file_fdw.out
index ba6fdba598..74ed8fae46 100644
--- a/contrib/file_fdw/expected/file_fdw.out
+++ b/contrib/file_fdw/expected/file_fdw.out
@@ -113,6 +113,12 @@ CREATE FOREIGN TABLE agg_bad (
) SERVER file_server
OPTIONS (format 'csv', filename :'filename', header 'true', delimiter ';', quote '@', escape '"', null '');
ALTER FOREIGN TABLE agg_bad ADD CHECK (a >= 0);
+-- test header matching
+\set filename :abs_srcdir '/data/list1.csv'
+CREATE FOREIGN TABLE header_match ("1" int, foo text) SERVER file_server
+OPTIONS (format 'csv', filename :'filename', delimiter ',', header 'match');
+CREATE FOREIGN TABLE header_doesnt_match (a int, foo text) SERVER file_server
+OPTIONS (format 'csv', filename :'filename', delimiter ',', header 'match'); -- ERROR
-- per-column options tests
\set filename :abs_srcdir '/data/text.csv'
CREATE FOREIGN TABLE text_csv (
@@ -464,12 +470,14 @@ SET ROLE regress_file_fdw_superuser;
-- cleanup
RESET ROLE;
DROP EXTENSION file_fdw CASCADE;
-NOTICE: drop cascades to 7 other objects
+NOTICE: drop cascades to 9 other objects
DETAIL: drop cascades to server file_server
drop cascades to user mapping for regress_file_fdw_superuser on server file_server
drop cascades to user mapping for regress_no_priv_user on server file_server
drop cascades to foreign table agg_text
drop cascades to foreign table agg_csv
drop cascades to foreign table agg_bad
+drop cascades to foreign table header_match
+drop cascades to foreign table header_doesnt_match
drop cascades to foreign table text_csv
DROP ROLE regress_file_fdw_superuser, regress_file_fdw_user, regress_no_priv_user;
diff --git a/contrib/file_fdw/sql/file_fdw.sql b/contrib/file_fdw/sql/file_fdw.sql
index 86f876d565..9f3c7219d0 100644
--- a/contrib/file_fdw/sql/file_fdw.sql
+++ b/contrib/file_fdw/sql/file_fdw.sql
@@ -103,6 +103,13 @@ CREATE FOREIGN TABLE agg_bad (
OPTIONS (format 'csv', filename :'filename', header 'true', delimiter ';', quote '@', escape '"', null '');
ALTER FOREIGN TABLE agg_bad ADD CHECK (a >= 0);
+-- test header matching
+\set filename :abs_srcdir '/data/list1.csv'
+CREATE FOREIGN TABLE header_match ("1" int, foo text) SERVER file_server
+OPTIONS (format 'csv', filename :'filename', delimiter ',', header 'match');
+CREATE FOREIGN TABLE header_doesnt_match (a int, foo text) SERVER file_server
+OPTIONS (format 'csv', filename :'filename', delimiter ',', header 'match'); -- ERROR
+
-- per-column options tests
\set filename :abs_srcdir '/data/text.csv'
CREATE FOREIGN TABLE text_csv (
diff --git a/doc/src/sgml/ref/copy.sgml b/doc/src/sgml/ref/copy.sgml
index 2d3847e0d1..f36eca02fb 100644
--- a/doc/src/sgml/ref/copy.sgml
+++ b/doc/src/sgml/ref/copy.sgml
@@ -36,7 +36,7 @@ COPY { <replaceable class="parameter">table_name</replaceable> [ ( <replaceable
FREEZE [ <replaceable class="parameter">boolean</replaceable> ]
DELIMITER '<replaceable class="parameter">delimiter_character</replaceable>'
NULL '<replaceable class="parameter">null_string</replaceable>'
- HEADER [ <replaceable class="parameter">boolean</replaceable> ]
+ HEADER [ <literal>match</literal> | <literal>true</literal> | <literal>false</literal> ]
QUOTE '<replaceable class="parameter">quote_character</replaceable>'
ESCAPE '<replaceable class="parameter">escape_character</replaceable>'
FORCE_QUOTE { ( <replaceable class="parameter">column_name</replaceable> [, ...] ) | * }
@@ -276,9 +276,13 @@ COPY { <replaceable class="parameter">table_name</replaceable> [ ( <replaceable
<para>
Specifies that the file contains a header line with the names of each
column in the file. On output, the first line contains the column
- names from the table, and on input, the first line is ignored.
- This option is allowed only when using <literal>CSV</literal> or
- <literal>text</literal> format.
+ names from the table. On input, the first line is discarded when
+ <literal>header</literal> is set to <literal>true</literal> or required
+ to match the column names if set to <literal>match</literal>. If the
+ number of columns in the header is not correct, their order differs
+ from the one expected, or the name or case doesn't match, the copy will
+ be aborted with an error. This option is allowed only when using
+ <literal>CSV</literal> or <literal>text</literal> format.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 382a12cfb2..cd427aaa0f 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -314,7 +314,66 @@ DoCopy(ParseState *pstate, const CopyStmt *stmt,
}
/*
- * Process the statement option list for COPY.
+* Extract a CopyHeader value from a DefElem.
+*/
+static CopyHeader
+DefGetCopyHeader(DefElem *def)
+{
+ /*
+ * If no parameter given, assume "true" is meant.
+ */
+ if (def->arg == NULL)
+ return COPY_HEADER_PRESENT;
+
+ /*
+ * Allow 0, 1, "true", "false", "on", "off" or "match".
+ */
+ switch (nodeTag(def->arg))
+ {
+ case T_Integer:
+ switch (intVal(def->arg))
+ {
+ case 0:
+ return COPY_HEADER_ABSENT;
+ case 1:
+ return COPY_HEADER_PRESENT;
+ default:
+ /* otherwise, error out below */
+ break;
+ }
+ break;
+ default:
+ {
+ char *sval = defGetString(def);
+
+ /*
+ * The set of strings accepted here should match up with the
+ * grammar's opt_boolean_or_string production.
+ */
+ if (pg_strcasecmp(sval, "true") == 0)
+ return COPY_HEADER_PRESENT;
+ if (pg_strcasecmp(sval, "false") == 0)
+ return COPY_HEADER_ABSENT;
+ if (pg_strcasecmp(sval, "on") == 0)
+ return COPY_HEADER_PRESENT;
+ if (pg_strcasecmp(sval, "off") == 0)
+ return COPY_HEADER_ABSENT;
+ if (pg_strcasecmp(sval, "match") == 0)
+ return COPY_HEADER_MATCH;
+
+ }
+ break;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("%s requires a boolean or \"match\"",
+ def->defname)));
+ return COPY_HEADER_ABSENT; /* keep compiler quiet */
+}
+
+/*
+* Process the statement option list for COPY.
*
* Scan the options list (a list of DefElem) and transpose the information
* into *opts_out, applying appropriate error checking.
@@ -394,7 +453,7 @@ ProcessCopyOptions(ParseState *pstate,
if (header_specified)
errorConflictingDefElem(defel, pstate);
header_specified = true;
- opts_out->header_line = defGetBoolean(defel);
+ opts_out->header_line = DefGetCopyHeader(defel);
}
else if (strcmp(defel->defname, "quote") == 0)
{
@@ -555,7 +614,7 @@ ProcessCopyOptions(ParseState *pstate,
errmsg("COPY delimiter cannot be \"%s\"", opts_out->delim)));
/* Check header */
- if (opts_out->binary && opts_out->header_line)
+ if (opts_out->binary && opts_out->header_line != COPY_HEADER_ABSENT)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("COPY HEADER available only in CSV and text mode")));
diff --git a/src/backend/commands/copyfromparse.c b/src/backend/commands/copyfromparse.c
index aac10165ec..e3452c26aa 100644
--- a/src/backend/commands/copyfromparse.c
+++ b/src/backend/commands/copyfromparse.c
@@ -72,6 +72,7 @@
#include "miscadmin.h"
#include "pgstat.h"
#include "port/pg_bswap.h"
+#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -758,12 +759,55 @@ NextCopyFromRawFields(CopyFromState cstate, char ***fields, int *nfields)
/* only available for text or csv input */
Assert(!cstate->opts.binary);
- /* on input just throw the header line away */
- if (cstate->cur_lineno == 0 && cstate->opts.header_line)
+ /* on input check that the header line is correct if needed */
+ if (cstate->cur_lineno == 0 && cstate->opts.header_line != COPY_HEADER_ABSENT)
{
+ ListCell *cur;
+ TupleDesc tupDesc;
+
+ tupDesc = RelationGetDescr(cstate->rel);
+
cstate->cur_lineno++;
- if (CopyReadLine(cstate))
- return false; /* done */
+ done = CopyReadLine(cstate);
+
+ if (cstate->opts.header_line == COPY_HEADER_MATCH)
+ {
+ if (cstate->opts.csv_mode)
+ fldct = CopyReadAttributesCSV(cstate);
+ else
+ fldct = CopyReadAttributesText(cstate);
+
+ if (fldct < list_length(cstate->attnumlist))
+ ereport(ERROR,
+ (errcode(ERRCODE_BAD_COPY_FILE_FORMAT),
+ errmsg("incomplete header, expected %d columns but got %d",
+ list_length(cstate->attnumlist), fldct)));
+ else if (fldct > list_length(cstate->attnumlist))
+ ereport(ERROR,
+ (errcode(ERRCODE_BAD_COPY_FILE_FORMAT),
+ errmsg("extra data after last expected header, expected %d columns but got %d",
+ list_length(cstate->attnumlist), fldct)));
+
+ foreach(cur, cstate->attnumlist)
+ {
+ int attnum = lfirst_int(cur);
+ char *colName = cstate->raw_fields[attnum - 1];
+ Form_pg_attribute attr = TupleDescAttr(tupDesc, attnum - 1);
+
+ if (colName == NULL)
+ colName = cstate->opts.null_print;
+
+ if (namestrcmp(&attr->attname, colName) != 0) {
+ ereport(ERROR,
+ (errcode(ERRCODE_BAD_COPY_FILE_FORMAT),
+ errmsg("wrong header for column \"%s\": got \"%s\"",
+ NameStr(attr->attname), colName)));
+ }
+ }
+ }
+
+ if (done)
+ return false;
}
cstate->cur_lineno++;
diff --git a/src/backend/commands/copyto.c b/src/backend/commands/copyto.c
index e9b962c7aa..a2cf680c67 100644
--- a/src/backend/commands/copyto.c
+++ b/src/backend/commands/copyto.c
@@ -848,7 +848,7 @@ DoCopyTo(CopyToState cstate)
cstate->file_encoding);
/* if a header has been requested send the line */
- if (cstate->opts.header_line)
+ if (cstate->opts.header_line != COPY_HEADER_ABSENT)
{
bool hdr_delim = false;
diff --git a/src/include/commands/copy.h b/src/include/commands/copy.h
index c3f91cdc47..c69eb6753f 100644
--- a/src/include/commands/copy.h
+++ b/src/include/commands/copy.h
@@ -19,6 +19,16 @@
#include "parser/parse_node.h"
#include "tcop/dest.h"
+/*
+ * Represents whether the header must match, be absent or be present.
+ */
+typedef enum CopyHeader
+{
+ COPY_HEADER_ABSENT,
+ COPY_HEADER_PRESENT,
+ COPY_HEADER_MATCH
+} CopyHeader;
+
/*
* A struct to hold COPY options, in a parsed form. All of these are related
* to formatting, except for 'freeze', which doesn't really belong here, but
@@ -32,7 +42,7 @@ typedef struct CopyFormatOptions
bool binary; /* binary format? */
bool freeze; /* freeze rows on loading? */
bool csv_mode; /* Comma Separated Value format? */
- bool header_line; /* CSV or text header line? */
+ CopyHeader header_line; /* CSV or text header line? */
char *null_print; /* NULL marker string (server encoding!) */
int null_print_len; /* length of same */
char *null_print_client; /* same converted to file encoding */
diff --git a/src/test/regress/expected/copy.out b/src/test/regress/expected/copy.out
index 08de57ee69..8461388085 100644
--- a/src/test/regress/expected/copy.out
+++ b/src/test/regress/expected/copy.out
@@ -254,3 +254,20 @@ INFO: progress: {"type": "FILE", "command": "COPY FROM", "relname": "tab_progre
drop trigger check_after_tab_progress_reporting on tab_progress_reporting;
drop function notice_after_tab_progress_reporting();
drop table tab_progress_reporting;
+-- Test header matching feature
+create table header_copytest (
+ a int,
+ b int,
+ c text
+);
+copy header_copytest from stdin with (header wrong_choice);
+ERROR: header requires a boolean or "match"
+copy header_copytest from stdin with (header match);
+copy header_copytest from stdin with (header match);
+ERROR: incomplete header, expected 3 columns but got 2
+CONTEXT: COPY header_copytest, line 1: "a b"
+copy header_copytest from stdin with (header match);
+ERROR: extra data after last expected header, expected 3 columns but got 4
+CONTEXT: COPY header_copytest, line 1: "a b c d"
+copy header_copytest from stdin with (header match, format csv);
+drop table header_copytest;
diff --git a/src/test/regress/sql/copy.sql b/src/test/regress/sql/copy.sql
index 6809f43e26..c98b6dd91c 100644
--- a/src/test/regress/sql/copy.sql
+++ b/src/test/regress/sql/copy.sql
@@ -303,3 +303,28 @@ copy tab_progress_reporting from :'filename'
drop trigger check_after_tab_progress_reporting on tab_progress_reporting;
drop function notice_after_tab_progress_reporting();
drop table tab_progress_reporting;
+
+-- Test header matching feature
+create table header_copytest (
+ a int,
+ b int,
+ c text
+);
+copy header_copytest from stdin with (header wrong_choice);
+copy header_copytest from stdin with (header match);
+a b c
+1 2 foo
+\.
+copy header_copytest from stdin with (header match);
+a b
+1 2
+\.
+copy header_copytest from stdin with (header match);
+a b c d
+1 2 foo bar
+\.
+copy header_copytest from stdin with (header match, format csv);
+a,b,c
+1,2,foo
+\.
+drop table header_copytest;
--------------2.34.1--
view thread (41+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Subject: Re: Add header support to text format and matching feature
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox