pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
5+ messages / 3 participants
[nested] [flat]
Commits from this thread:
2026-10-07 e96efd07c7 Remove "array_nulls" GUC parameter.
* BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-04 05:19 PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 5+ messages in thread
From: PG Bug reporting form @ 2026-10-04 05:19 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: kehan5800@gmail.com
The following bug has been logged on the website:
Bug reference: 19747
Logged by: Ke
Email address: kehan5800@gmail.com
PostgreSQL version: 18.6
Operating system: Ubuntu 22.04.2 x86_64
Description:
pg_dump writes a null array element as the unquoted word NULL. Whether the
array input function reads that as a null depends on array_nulls, which is
PGC_USERSET and can be persisted with ALTER DATABASE/ROLE ... SET. The
dump's preamble (_doSetFixedOutputState(), src/bin/pg_dump/
pg_backup_archiver.c:3431) pins the other input-side settings
(client_encoding, standard_conforming_strings, check_function_bodies,
xmloption, row_security, ...) but not array_nulls, so the restoring session
inherits the target database's value.
Reproduction:
createdb src
psql -d src -c "CREATE TABLE t_txt (id int PRIMARY KEY, v text[]);
INSERT INTO t_txt VALUES (1, ARRAY['x', NULL, 'y']);
CREATE TABLE t_int (id int PRIMARY KEY, v int[]);
INSERT INTO t_int VALUES (1, ARRAY[1, NULL, 3]);"
createdb dst
psql -d postgres -c "ALTER DATABASE dst SET array_nulls = off"
pg_dump -d src -f all.sql
psql -d dst -f all.sql
ERROR: invalid input syntax for type integer: "NULL"
CONTEXT: COPY t_int, line 1, column v: "{1,NULL,3}"
and with only the text[] table dumped (pg_dump -t t_txt) there is no error
at all:
source : {x,NULL,y} v[2] IS NULL -> true
restored : {x,"NULL",y} v[2] IS NULL -> false, v[2] = 'NULL'
psql exits 0 and row counts match; only comparing the array values shows
the change. text[], citext[], domains over text and composites containing
them change silently; int[] and other element types that cannot parse
"NULL" fail the COPY.
Expected: a dump restores to the same data regardless of the target
database's array_nulls, as it already does for DateStyle, IntervalStyle,
standard_conforming_strings, xmloption, etc.
Actual: null elements become the four-character string "NULL" (text-like
element types) or the table fails to load (other element types).
The same omission exists in the two other places that copy pg_dump's list
of pinned settings:
- contrib/postgres_fdw/connection.c:839-845 (configure_remote_session),
whose comment says "This logic should match what pg_dump does": it sets
datestyle, intervalstyle and extra_float_digits on the remote session,
not array_nulls. With array_nulls = off on the remote database, an
INSERT of ARRAY['x', NULL, 'y'] through a foreign table stores
{x,"NULL",y} remotely. In the other direction, with array_nulls = off in
the local database, a remote row {a,NULL,b} reads back through the
foreign table as {a,"NULL",b} (v[2] IS NULL false locally, true on the
remote).
- src/backend/replication/libpqwalreceiver/libpqwalreceiver.c:199 passes
"-c datestyle=ISO -c intervalstyle=postgres -c extra_float_digits=3" to
the publisher; the apply worker parses incoming text values under the
subscriber database's settings. With array_nulls = off on the
subscriber database, publisher {a,NULL,b} arrives as {a,"NULL",b} with
nothing logged; for int[] the apply worker errors with "invalid input
syntax for type integer" and is restarted repeatedly.
All three were reproduced on master by the attached script.
Suggested fix:
pg_backup_archiver.c, _doSetFixedOutputState():
ahprintf(AH, "SET xmloption = content;\n");
+ ahprintf(AH, "SET array_nulls = on;\n");
array_nulls = on is the default and what pg_dump assumed when writing the
data, so existing dump semantics do not change.
postgres_fdw: add "SET array_nulls = on" in configure_remote_session()
for the write direction; for the read direction the local input
conversion (make_tuple_from_result_row) would need the same setting
forced locally, e.g. via the GUC nest level that set_transmission_modes()
already uses.
logical replication: force array_nulls = on in the apply worker (the
parsing side), not in the publisher connection options.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-05 21:34 Tom Lane <tgl@sss.pgh.pa.us>
parent: PG Bug reporting form <noreply@postgresql.org>
0 siblings, 1 reply; 5+ messages in thread
From: Tom Lane @ 2026-10-05 21:34 UTC (permalink / raw)
To: kehan5800@gmail.com; +Cc: pgsql-bugs@lists.postgresql.org
PG Bug reporting form <noreply@postgresql.org> writes:
> pg_dump writes a null array element as the unquoted word NULL. Whether the
> array input function reads that as a null depends on array_nulls, which is
> PGC_USERSET and can be persisted with ALTER DATABASE/ROLE ... SET. The
> dump's preamble (_doSetFixedOutputState(), src/bin/pg_dump/
> pg_backup_archiver.c:3431) pins the other input-side settings
> (client_encoding, standard_conforming_strings, check_function_bodies,
> xmloption, row_security, ...) but not array_nulls, so the restoring session
> inherits the target database's value.
Meh. TBH, that GUC was past its shelf life ten years ago. What
I'd rather do about this report is just summarily remove the GUC.
If we make pg_dump issue a SET for it then we'll never be able
to remove it.
We could alternatively do what we've done with some other GUCs
whose time has passed: force them to have constant values and
accept SET commands only when the value matches. But I doubt
that this one ever got widespread enough use to justify doing that
much work. I'd rather just nuke it from orbit.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-06 20:06 Tom Lane <tgl@sss.pgh.pa.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 5+ messages in thread
From: Tom Lane @ 2026-10-06 20:06 UTC (permalink / raw)
To: kehan5800@gmail.com; +Cc: pgsql-bugs@lists.postgresql.org
I wrote:
> PG Bug reporting form <noreply@postgresql.org> writes:
>> pg_dump writes a null array element as the unquoted word NULL. Whether the
>> array input function reads that as a null depends on array_nulls, which is
>> PGC_USERSET and can be persisted with ALTER DATABASE/ROLE ... SET. The
>> dump's preamble (_doSetFixedOutputState(), src/bin/pg_dump/
>> pg_backup_archiver.c:3431) pins the other input-side settings
>> (client_encoding, standard_conforming_strings, check_function_bodies,
>> xmloption, row_security, ...) but not array_nulls, so the restoring session
>> inherits the target database's value.
> Meh. TBH, that GUC was past its shelf life ten years ago. What
> I'd rather do about this report is just summarily remove the GUC.
> If we make pg_dump issue a SET for it then we'll never be able
> to remove it.
Concretely, about like this. (BTW, another argument for doing
this rather than fixing pg_dump is that there are a ton of other
applications that will likely misbehave if array_nulls is turned
on underneath them.)
regards, tom lane
Attachments:
v1-0001-Remove-array_nulls-GUC-parameter.patch (text/x-diff, 6.3K) download
From 7e560c03b0304aa85c7411396ed634dc62e2a495 Mon Sep 17 00:00:00 2001
From: Tom Lane <tgl@sss.pgh.pa.us>
Date: Tue, 6 Oct 2026 16:01:33 -0400
Subject: [PATCH v1] Remove "array_nulls" GUC parameter.
This backwards-compatibility hack was introduced twenty years ago (in
8.2), and has surely been of no value for at least the last ten years.
However, we shouldn't just leave it sit, because as was just pointed
out it creates a dump/reload hazard: if it's set in the receiving
environment then array contents will not be restored correctly.
Our precedent for retiring some other comparable parameters, such as
standard_conforming_strings, has been to leave the parameter in
existence but force it to a fixed value. I judge however that
that's not worth the trouble here, because there's no reason to
believe that a non-negligible number of applications are either
setting or reading this value. Let's just delete it in the master
branch and be done with it.
Bug: #19747
Reported-by: Ke <kehan5800@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19747-69b01e7fc190cd58@postgresql.org
---
doc/src/sgml/array.sgml | 5 +---
doc/src/sgml/config.sgml | 26 -------------------
src/backend/utils/adt/arrayfuncs.c | 7 +----
src/backend/utils/misc/guc_parameters.dat | 7 -----
src/backend/utils/misc/postgresql.conf.sample | 1 -
src/include/utils/array.h | 5 ----
6 files changed, 2 insertions(+), 49 deletions(-)
diff --git a/doc/src/sgml/array.sgml b/doc/src/sgml/array.sgml
index ce338c770c9..5b8ef5cabe6 100644
--- a/doc/src/sgml/array.sgml
+++ b/doc/src/sgml/array.sgml
@@ -756,10 +756,7 @@ SELECT f1[1][-2][3] AS e1, f1[1][-1][5] AS e2
If the value written for an element is <literal>NULL</literal> (in any case
variant), the element is taken to be NULL. The presence of any quotes
or backslashes disables this and allows the literal string value
- <quote>NULL</quote> to be entered. Also, for backward compatibility with
- pre-8.2 versions of <productname>PostgreSQL</productname>, the <xref
- linkend="guc-array-nulls"/> configuration parameter can be turned
- <literal>off</literal> to suppress recognition of <literal>NULL</literal> as a NULL.
+ <quote>NULL</quote> to be entered.
</para>
<para>
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index f36fbb60101..e537b3b0043 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -11941,32 +11941,6 @@ dynamic_library_path = '/usr/local/lib/postgresql:$libdir'
<variablelist>
- <varlistentry id="guc-array-nulls" xreflabel="array_nulls">
- <term><varname>array_nulls</varname> (<type>boolean</type>)
- <indexterm>
- <primary><varname>array_nulls</varname> configuration parameter</primary>
- </indexterm>
- </term>
- <listitem>
- <para>
- This controls whether the array input parser recognizes
- unquoted <literal>NULL</literal> as specifying a null array element.
- By default, this is <literal>on</literal>, allowing array values containing
- null values to be entered. However, <productname>PostgreSQL</productname> versions
- before 8.2 did not support null values in arrays, and therefore would
- treat <literal>NULL</literal> as specifying a normal array element with
- the string value <quote>NULL</quote>. For backward compatibility with
- applications that require the old behavior, this variable can be
- turned <literal>off</literal>.
- </para>
-
- <para>
- Note that it is possible to create array values containing null values
- even when this variable is <literal>off</literal>.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="guc-backslash-quote" xreflabel="backslash_quote">
<term><varname>backslash_quote</varname> (<type>enum</type>)
<indexterm><primary>strings</primary><secondary>backslash quotes</secondary></indexterm>
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index ef66182b047..f1f6336e5c8 100644
--- a/src/backend/utils/adt/arrayfuncs.c
+++ b/src/backend/utils/adt/arrayfuncs.c
@@ -38,11 +38,6 @@
#include "utils/typcache.h"
-/*
- * GUC parameter
- */
-bool Array_nulls = true;
-
/*
* Local definitions
*/
@@ -925,7 +920,7 @@ unquoted_element:
elembuf->len = dstlen;
*srcptr = p;
/* Check if it's unquoted "NULL" */
- if (Array_nulls && !has_escapes &&
+ if (!has_escapes &&
pg_strcasecmp(elembuf->data, "NULL") == 0)
return ATOK_ELEM_NULL;
else
diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat
index c57441f7d98..380679a76c0 100644
--- a/src/backend/utils/misc/guc_parameters.dat
+++ b/src/backend/utils/misc/guc_parameters.dat
@@ -105,13 +105,6 @@
max => 'INT_MAX / 2',
},
-{ name => 'array_nulls', type => 'bool', context => 'PGC_USERSET', group => 'COMPAT_OPTIONS_PREVIOUS',
- short_desc => 'Enables input of NULL elements in arrays.',
- long_desc => 'When turned on, unquoted NULL in an array input value means a null value; otherwise it is taken literally.',
- variable => 'Array_nulls',
- boot_val => 'true',
-},
-
{ name => 'authentication_timeout', type => 'int', context => 'PGC_SIGHUP', group => 'CONN_AUTH_AUTH',
short_desc => 'Sets the maximum allowed time to complete client authentication.',
flags => 'GUC_UNIT_S',
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index e759f06b50f..01f98de8a9f 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -892,7 +892,6 @@
# - Previous PostgreSQL Versions -
-#array_nulls = on
#backslash_quote = safe_encoding # on, off, or safe_encoding
#lo_compat_privileges = off
#quote_all_identifiers = off
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index 88e4f4d70d8..e7ee184034e 100644
--- a/src/include/utils/array.h
+++ b/src/include/utils/array.h
@@ -343,11 +343,6 @@ typedef struct ArrayIteratorData *ArrayIterator;
(a)->xpn.lbound : ARR_LBOUND((ArrayType *) (a)))
-/*
- * GUC parameter
- */
-extern PGDLLIMPORT bool Array_nulls;
-
/*
* prototypes for functions defined in arrayfuncs.c
*/
--
2.52.0
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-07 06:54 Laurenz Albe <laurenz.albe@cybertec.at>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 5+ messages in thread
From: Laurenz Albe @ 2026-10-07 06:54 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; kehan5800@gmail.com; +Cc: pgsql-bugs@lists.postgresql.org
On Tue, 2026-10-06 at 16:06 -0400, Tom Lane wrote:
> I wrote:
> > PG Bug reporting form <noreply@postgresql.org> writes:
> > > [misbehavior with array_nulls on]
>
> > Meh. TBH, that GUC was past its shelf life ten years ago. What
> > I'd rather do about this report is just summarily remove the GUC.
> > If we make pg_dump issue a SET for it then we'll never be able
> > to remove it.
>
> Concretely, about like this. (BTW, another argument for doing
> this rather than fixing pg_dump is that there are a ton of other
> applications that will likely misbehave if array_nulls is turned
> on underneath them.)
+1
I have never seen this used in the field. One less potential headache!
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-07 15:44 Tom Lane <tgl@sss.pgh.pa.us>
parent: Laurenz Albe <laurenz.albe@cybertec.at>
0 siblings, 0 replies; 5+ messages in thread
From: Tom Lane @ 2026-10-07 15:44 UTC (permalink / raw)
To: Laurenz Albe <laurenz.albe@cybertec.at>; +Cc: kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org
Laurenz Albe <laurenz.albe@cybertec.at> writes:
> On Tue, 2026-10-06 at 16:06 -0400, Tom Lane wrote:
>> Concretely, about like this. (BTW, another argument for doing
>> this rather than fixing pg_dump is that there are a ton of other
>> applications that will likely misbehave if array_nulls is turned
>> on underneath them.)
> +1
> I have never seen this used in the field. One less potential headache!
Pushed, thanks for looking at it!
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2026-10-07 15:44 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-10-04 05:19 BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements PG Bug reporting form <noreply@postgresql.org>
2026-10-05 21:34 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-06 20:06 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-07 06:54 ` Laurenz Albe <laurenz.albe@cybertec.at>
2026-10-07 15:44 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox