pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
5+ messages / 3 participants
[nested] [flat]

Commits from this thread:
  2026-10-07 e96efd07c7  Remove "array_nulls" GUC parameter.
* BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-04 05:19  PG Bug reporting form <noreply@postgresql.org>
  0 siblings, 1 reply; 5+ messages in thread

From: PG Bug reporting form @ 2026-10-04 05:19 UTC (permalink / raw)
  To: pgsql-bugs@lists.postgresql.org; +Cc: kehan5800@gmail.com

The following bug has been logged on the website:

Bug reference:      19747
Logged by:          Ke
Email address:      kehan5800@gmail.com
PostgreSQL version: 18.6
Operating system:   Ubuntu 22.04.2 x86_64
Description:        

pg_dump writes a null array element as the unquoted word NULL. Whether the
array input function reads that as a null depends on array_nulls, which is
PGC_USERSET and can be persisted with ALTER DATABASE/ROLE ... SET. The
dump's preamble (_doSetFixedOutputState(), src/bin/pg_dump/
pg_backup_archiver.c:3431) pins the other input-side settings
(client_encoding, standard_conforming_strings, check_function_bodies,
xmloption, row_security, ...) but not array_nulls, so the restoring session
inherits the target database's value.

Reproduction:

    createdb src
    psql -d src -c "CREATE TABLE t_txt (id int PRIMARY KEY, v text[]);
                    INSERT INTO t_txt VALUES (1, ARRAY['x', NULL, 'y']);
                    CREATE TABLE t_int (id int PRIMARY KEY, v int[]);
                    INSERT INTO t_int VALUES (1, ARRAY[1, NULL, 3]);"
    createdb dst
    psql -d postgres -c "ALTER DATABASE dst SET array_nulls = off"
    pg_dump -d src -f all.sql
    psql -d dst -f all.sql

    ERROR:  invalid input syntax for type integer: "NULL"
    CONTEXT:  COPY t_int, line 1, column v: "{1,NULL,3}"

and with only the text[] table dumped (pg_dump -t t_txt) there is no error
at all:

    source   : {x,NULL,y}     v[2] IS NULL  -> true
    restored : {x,"NULL",y}   v[2] IS NULL  -> false, v[2] = 'NULL'

psql exits 0 and row counts match; only comparing the array values shows
the change. text[], citext[], domains over text and composites containing
them change silently; int[] and other element types that cannot parse
"NULL" fail the COPY.

Expected: a dump restores to the same data regardless of the target
database's array_nulls, as it already does for DateStyle, IntervalStyle,
standard_conforming_strings, xmloption, etc.

Actual: null elements become the four-character string "NULL" (text-like
element types) or the table fails to load (other element types).

The same omission exists in the two other places that copy pg_dump's list
of pinned settings:

  - contrib/postgres_fdw/connection.c:839-845 (configure_remote_session),
    whose comment says "This logic should match what pg_dump does": it sets
    datestyle, intervalstyle and extra_float_digits on the remote session,
    not array_nulls. With array_nulls = off on the remote database, an
    INSERT of ARRAY['x', NULL, 'y'] through a foreign table stores
    {x,"NULL",y} remotely. In the other direction, with array_nulls = off in
    the local database, a remote row {a,NULL,b} reads back through the
    foreign table as {a,"NULL",b} (v[2] IS NULL false locally, true on the
    remote).

  - src/backend/replication/libpqwalreceiver/libpqwalreceiver.c:199 passes
    "-c datestyle=ISO -c intervalstyle=postgres -c extra_float_digits=3" to
    the publisher; the apply worker parses incoming text values under the
    subscriber database's settings. With array_nulls = off on the
    subscriber database, publisher {a,NULL,b} arrives as {a,"NULL",b} with
    nothing logged; for int[] the apply worker errors with "invalid input
    syntax for type integer" and is restarted repeatedly.

All three were reproduced on master by the attached script.

Suggested fix:

  pg_backup_archiver.c, _doSetFixedOutputState():
        ahprintf(AH, "SET xmloption = content;\n");
     +  ahprintf(AH, "SET array_nulls = on;\n");

  array_nulls = on is the default and what pg_dump assumed when writing the
  data, so existing dump semantics do not change.

  postgres_fdw: add "SET array_nulls = on" in configure_remote_session()
  for the write direction; for the read direction the local input
  conversion (make_tuple_from_result_row) would need the same setting
  forced locally, e.g. via the GUC nest level that set_transmission_modes()
  already uses.

  logical replication: force array_nulls = on in the apply worker (the
  parsing side), not in the publisher connection options.








^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-05 21:34  Tom Lane <tgl@sss.pgh.pa.us>
  parent: PG Bug reporting form <noreply@postgresql.org>
  0 siblings, 1 reply; 5+ messages in thread

From: Tom Lane @ 2026-10-05 21:34 UTC (permalink / raw)
  To: kehan5800@gmail.com; +Cc: pgsql-bugs@lists.postgresql.org

PG Bug reporting form <noreply@postgresql.org> writes:
> pg_dump writes a null array element as the unquoted word NULL. Whether the
> array input function reads that as a null depends on array_nulls, which is
> PGC_USERSET and can be persisted with ALTER DATABASE/ROLE ... SET. The
> dump's preamble (_doSetFixedOutputState(), src/bin/pg_dump/
> pg_backup_archiver.c:3431) pins the other input-side settings
> (client_encoding, standard_conforming_strings, check_function_bodies,
> xmloption, row_security, ...) but not array_nulls, so the restoring session
> inherits the target database's value.

Meh.  TBH, that GUC was past its shelf life ten years ago.  What
I'd rather do about this report is just summarily remove the GUC.
If we make pg_dump issue a SET for it then we'll never be able
to remove it.

We could alternatively do what we've done with some other GUCs
whose time has passed: force them to have constant values and
accept SET commands only when the value matches.  But I doubt
that this one ever got widespread enough use to justify doing that
much work.  I'd rather just nuke it from orbit.

			regards, tom lane






^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-06 20:06  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 5+ messages in thread

From: Tom Lane @ 2026-10-06 20:06 UTC (permalink / raw)
  To: kehan5800@gmail.com; +Cc: pgsql-bugs@lists.postgresql.org

I wrote:
> PG Bug reporting form <noreply@postgresql.org> writes:
>> pg_dump writes a null array element as the unquoted word NULL. Whether the
>> array input function reads that as a null depends on array_nulls, which is
>> PGC_USERSET and can be persisted with ALTER DATABASE/ROLE ... SET. The
>> dump's preamble (_doSetFixedOutputState(), src/bin/pg_dump/
>> pg_backup_archiver.c:3431) pins the other input-side settings
>> (client_encoding, standard_conforming_strings, check_function_bodies,
>> xmloption, row_security, ...) but not array_nulls, so the restoring session
>> inherits the target database's value.

> Meh.  TBH, that GUC was past its shelf life ten years ago.  What
> I'd rather do about this report is just summarily remove the GUC.
> If we make pg_dump issue a SET for it then we'll never be able
> to remove it.

Concretely, about like this.  (BTW, another argument for doing
this rather than fixing pg_dump is that there are a ton of other
applications that will likely misbehave if array_nulls is turned
on underneath them.)

			regards, tom lane

Attachments:

v1-0001-Remove-array_nulls-GUC-parameter.patch (text/x-diff, 6.3K) download From 7e560c03b0304aa85c7411396ed634dc62e2a495 Mon Sep 17 00:00:00 2001 From: Tom Lane <tgl@sss.pgh.pa.us> Date: Tue, 6 Oct 2026 16:01:33 -0400 Subject: [PATCH v1] Remove "array_nulls" GUC parameter. This backwards-compatibility hack was introduced twenty years ago (in 8.2), and has surely been of no value for at least the last ten years. However, we shouldn't just leave it sit, because as was just pointed out it creates a dump/reload hazard: if it's set in the receiving environment then array contents will not be restored correctly. Our precedent for retiring some other comparable parameters, such as standard_conforming_strings, has been to leave the parameter in existence but force it to a fixed value. I judge however that that's not worth the trouble here, because there's no reason to believe that a non-negligible number of applications are either setting or reading this value. Let's just delete it in the master branch and be done with it. Bug: #19747 Reported-by: Ke <kehan5800@gmail.com> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19747-69b01e7fc190cd58@postgresql.org --- doc/src/sgml/array.sgml | 5 +--- doc/src/sgml/config.sgml | 26 ------------------- src/backend/utils/adt/arrayfuncs.c | 7 +---- src/backend/utils/misc/guc_parameters.dat | 7 ----- src/backend/utils/misc/postgresql.conf.sample | 1 - src/include/utils/array.h | 5 ---- 6 files changed, 2 insertions(+), 49 deletions(-) diff --git a/doc/src/sgml/array.sgml b/doc/src/sgml/array.sgml index ce338c770c9..5b8ef5cabe6 100644 --- a/doc/src/sgml/array.sgml +++ b/doc/src/sgml/array.sgml @@ -756,10 +756,7 @@ SELECT f1[1][-2][3] AS e1, f1[1][-1][5] AS e2 If the value written for an element is <literal>NULL</literal> (in any case variant), the element is taken to be NULL. The presence of any quotes or backslashes disables this and allows the literal string value - <quote>NULL</quote> to be entered. Also, for backward compatibility with - pre-8.2 versions of <productname>PostgreSQL</productname>, the <xref - linkend="guc-array-nulls"/> configuration parameter can be turned - <literal>off</literal> to suppress recognition of <literal>NULL</literal> as a NULL. + <quote>NULL</quote> to be entered. </para> <para> diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index f36fbb60101..e537b3b0043 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -11941,32 +11941,6 @@ dynamic_library_path = '/usr/local/lib/postgresql:$libdir' <variablelist> - <varlistentry id="guc-array-nulls" xreflabel="array_nulls"> - <term><varname>array_nulls</varname> (<type>boolean</type>) - <indexterm> - <primary><varname>array_nulls</varname> configuration parameter</primary> - </indexterm> - </term> - <listitem> - <para> - This controls whether the array input parser recognizes - unquoted <literal>NULL</literal> as specifying a null array element. - By default, this is <literal>on</literal>, allowing array values containing - null values to be entered. However, <productname>PostgreSQL</productname> versions - before 8.2 did not support null values in arrays, and therefore would - treat <literal>NULL</literal> as specifying a normal array element with - the string value <quote>NULL</quote>. For backward compatibility with - applications that require the old behavior, this variable can be - turned <literal>off</literal>. - </para> - - <para> - Note that it is possible to create array values containing null values - even when this variable is <literal>off</literal>. - </para> - </listitem> - </varlistentry> - <varlistentry id="guc-backslash-quote" xreflabel="backslash_quote"> <term><varname>backslash_quote</varname> (<type>enum</type>) <indexterm><primary>strings</primary><secondary>backslash quotes</secondary></indexterm> diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index ef66182b047..f1f6336e5c8 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -38,11 +38,6 @@ #include "utils/typcache.h" -/* - * GUC parameter - */ -bool Array_nulls = true; - /* * Local definitions */ @@ -925,7 +920,7 @@ unquoted_element: elembuf->len = dstlen; *srcptr = p; /* Check if it's unquoted "NULL" */ - if (Array_nulls && !has_escapes && + if (!has_escapes && pg_strcasecmp(elembuf->data, "NULL") == 0) return ATOK_ELEM_NULL; else diff --git a/src/backend/utils/misc/guc_parameters.dat b/src/backend/utils/misc/guc_parameters.dat index c57441f7d98..380679a76c0 100644 --- a/src/backend/utils/misc/guc_parameters.dat +++ b/src/backend/utils/misc/guc_parameters.dat @@ -105,13 +105,6 @@ max => 'INT_MAX / 2', }, -{ name => 'array_nulls', type => 'bool', context => 'PGC_USERSET', group => 'COMPAT_OPTIONS_PREVIOUS', - short_desc => 'Enables input of NULL elements in arrays.', - long_desc => 'When turned on, unquoted NULL in an array input value means a null value; otherwise it is taken literally.', - variable => 'Array_nulls', - boot_val => 'true', -}, - { name => 'authentication_timeout', type => 'int', context => 'PGC_SIGHUP', group => 'CONN_AUTH_AUTH', short_desc => 'Sets the maximum allowed time to complete client authentication.', flags => 'GUC_UNIT_S', diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample index e759f06b50f..01f98de8a9f 100644 --- a/src/backend/utils/misc/postgresql.conf.sample +++ b/src/backend/utils/misc/postgresql.conf.sample @@ -892,7 +892,6 @@ # - Previous PostgreSQL Versions - -#array_nulls = on #backslash_quote = safe_encoding # on, off, or safe_encoding #lo_compat_privileges = off #quote_all_identifiers = off diff --git a/src/include/utils/array.h b/src/include/utils/array.h index 88e4f4d70d8..e7ee184034e 100644 --- a/src/include/utils/array.h +++ b/src/include/utils/array.h @@ -343,11 +343,6 @@ typedef struct ArrayIteratorData *ArrayIterator; (a)->xpn.lbound : ARR_LBOUND((ArrayType *) (a))) -/* - * GUC parameter - */ -extern PGDLLIMPORT bool Array_nulls; - /* * prototypes for functions defined in arrayfuncs.c */ -- 2.52.0
^ permalink raw reply [nested|flat] 5+ messages in thread

* Re: BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-07 06:54  Laurenz Albe <laurenz.albe@cybertec.at>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 5+ messages in thread

From: Laurenz Albe @ 2026-10-07 06:54 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; kehan5800@gmail.com; +Cc: pgsql-bugs@lists.postgresql.org

On Tue, 2026-10-06 at 16:06 -0400, Tom Lane wrote:
> I wrote:
> > PG Bug reporting form <noreply@postgresql.org> writes:
> > > [misbehavior with array_nulls on]
> 
> > Meh.  TBH, that GUC was past its shelf life ten years ago.  What
> > I'd rather do about this report is just summarily remove the GUC.
> > If we make pg_dump issue a SET for it then we'll never be able
> > to remove it.
> 
> Concretely, about like this.  (BTW, another argument for doing
> this rather than fixing pg_dump is that there are a ton of other
> applications that will likely misbehave if array_nulls is turned
> on underneath them.)

+1

I have never seen this used in the field.  One less potential headache!

Yours,
Laurenz Albe






^ permalink  raw  reply  [nested|flat] 5+ messages in thread

* Re: BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements
@ 2026-10-07 15:44  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Laurenz Albe <laurenz.albe@cybertec.at>
  0 siblings, 0 replies; 5+ messages in thread

From: Tom Lane @ 2026-10-07 15:44 UTC (permalink / raw)
  To: Laurenz Albe <laurenz.albe@cybertec.at>; +Cc: kehan5800@gmail.com; pgsql-bugs@lists.postgresql.org

Laurenz Albe <laurenz.albe@cybertec.at> writes:
> On Tue, 2026-10-06 at 16:06 -0400, Tom Lane wrote:
>> Concretely, about like this.  (BTW, another argument for doing
>> this rather than fixing pg_dump is that there are a ton of other
>> applications that will likely misbehave if array_nulls is turned
>> on underneath them.)

> +1
> I have never seen this used in the field.  One less potential headache!

Pushed, thanks for looking at it!

			regards, tom lane






^ permalink  raw  reply  [nested|flat] 5+ messages in thread


end of thread, other threads:[~2026-10-07 15:44 UTC | newest]

Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-10-04 05:19 BUG #19747: pg_dump does not pin array_nulls, so restore mangles NULL array elements PG Bug reporting form <noreply@postgresql.org>
2026-10-05 21:34 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-06 20:06   ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-07 06:54     ` Laurenz Albe <laurenz.albe@cybertec.at>
2026-10-07 15:44       ` Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox