Subject: Re: Upsert error "column reference is ambiguous"

From: Tim Starling <[email-masked]>
Date: Mon, 28 Apr 2025 21:22:56 +1000

On 28/4/25 20:54, Tom Lane wrote:
> Even if I were on board with arbitrarily adopting one of the two
> possible interpretations, it's far from obvious to me that most people
> would agree that "v" should mean the value from the existing row,
> rather than the new value.  Better to make them say which they want.

OK sure, no way to tell, but if every other DBMS does it the same way 
then that might be a hint.

Also, I'm just saying, the upsert feature is fully useless to me with 
this name resolution policy.

In the single-row case, there's no need for EXCLUDED at all, because 
the client knows everything about the excluded row. Recall my example:

INSERT INTO t VALUES (1,1) ON CONFLICT (k) DO UPDATE SET v=v+1;

If I meant SET v=EXCLUDED.v+1 I would have just written v=2. The 
default policy (in other DBMSes) follows by analogy from the 
single-row case.

-- Tim Starling

---

From: Laurenz Albe <[email-masked]>
Date: Mon, 28 Apr 2025 15:29:51 +0200

> then that might be a hint.

Which DBMS that supports INSERT .. ON CONFLICT do you have in mind?

> this name resolution policy.

Because you cannot write EXCLUDED?

> default policy (in other DBMSes) follows by analogy from the 

Actually, for many people, the DWIM would be the other way around:

INSERT INTO tab (col)
SELECT something FROM othertab
ON CONFLICT (id)
/* "col" should get set to "something" */
DO UPDATE SET col = col;

I can follow your reasoning above, but if the SQL parser tried to
guess the user's intention like that, it is likely to go wrong
sometimes.  As Tom said, better force the user to be explicit.

Yours,
Laurenz Albe

---

From: Tom Lane <[email-masked]>
Date: Mon, 28 Apr 2025 09:54:09 -0400

> then that might be a hint.

AFAIK, "ON CONFLICT" is a Postgres-ism.  Exactly which constructs
in exactly which other databases are you citing as precedent?

> the client knows everything about the excluded row.

Laurenz already provided the counter-example of an INSERT/SELECT,
but there's also the possibility of the INSERT supplying a computed
default value for a column, e.g., CURRENT_TIMESTAMP.  So you won't
get far with that argument.

I do actually have some sympathy for your proposal after thinking
about it a bit more, but the argument I would use is "the behavior
of the ON CONFLICT UPDATE SET list should be as much as possible like
the behavior of an ordinary UPDATE's SET list".  Since "v = v+1" would
refer to the existing row's "v" in regular UPDATE, it's sensible to
let that happen here too.  Of course the counter-argument is that this
should be compared not to a trivial UPDATE, but an "UPDATE ... FROM
othertable" where the othertable supplies some conflicting column
name(s).  In that situation we're going to make you resolve the
conflict by qualifying the column names.  The only thing that makes
that not a precise parallel is that EXCLUDED is not something the user
wrote into the query explicitly, so there's no opportunity to
substitute different column aliases, as a FROM clause would allow.
Perhaps that justifies demoting it to second-class citizenship whereby
EXCLUDED has to be qualified but the target table doesn't.  (I don't
find this argument hugely compelling, but it's an argument.)

BTW, I did wonder how hard it would be to make such a change.
On first glance it seems to be a one-liner:

diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 1f4d6adda52..f11727adbaa 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1306,7 +1306,7 @@ transformOnConflictClause(ParseState *pstate,
 		 * Add the EXCLUDED pseudo relation to the query namespace, making it
 		 * available in the UPDATE subexpressions.
 		 */
-		addNSItemToQuery(pstate, exclNSItem, false, true, true);
+		addNSItemToQuery(pstate, exclNSItem, false, true, false);
 
 		/*
 		 * Now transform the UPDATE subexpressions.

So this isn't about implementation difficulty but about whether
we think it's a good idea.

			regards, tom lane

---

From: "David G. Johnston" <[email-masked]>
Date: Mon, 28 Apr 2025 08:21:55 -0700

> in exactly which other databases are you citing as precedent?

I confirmed the SQLite reference from the original email.

“The upsert above inserts the new vocabulary word "jovial" if that word is
not already in the dictionary, or if it is already in the dictionary, it
increments the counter. The "count+1" expression could also be written as
"vocabulary.count". PostgreSQL requires the second form, but SQLite accepts
either.”

https://sqlite.org/lang_upsert.html

David J.

---

From: Tim Starling <[email-masked]>
Date: Tue, 29 Apr 2025 08:36:18 +1000

> in exactly which other databases are you citing as precedent?

There's a list here:

<https://wiki.postgresql.org/wiki/UPSERT#UPSERT_as_implemented_in_practice>

Since that page was written in 2014, SQLite added upsert support, 
consciously following PG's syntax, except that unqualified names 
resolve to target rows.

My code would be like

function upsert( $table, $names, $values, $key, $set ) {
     if ( $this->type === 'mysql' ) {
         $conflict = 'ON DUPLICATE KEY UPDATE';
     } else {
         $conflict = "ON CONFLICT ($key) DO UPDATE SET";
     }
     return $this->query( "INSERT INTO $table ($names) " .
         "VALUES ($values) $conflict $set" );
}

The parameters are a little bit more structured than that, but that 
gives you the idea.

MediaWiki has supported MySQL's ON DUPLICATE KEY UPDATE since 2013, 
and we've always had the conflict target parameter $key since then as 
a helper for emulation. So it's trivial to produce either MySQL and 
SQLite syntax.

-- Tim Starling

---

From: Christophe Pettus <[email-masked]>
Date: Mon, 28 Apr 2025 15:58:42 -0700

> On Apr 28, 2025, at 15:36, Tim Starling <[email-masked]> wrote:

I'll mention that you can do this without ON CONFLICT in PostgreSQL in a way that, while not nearly as clean as ON CONFLICT, isn't a huge hack, either:

"DO $$ BEGIN INSERT INTO $table($names) VALUES($values); EXCEPTION WHEN integrity_constraint_violation THEN UPDATE $table SET $set WHERE $key=$values[0]; END; $$ LANGUAGE plpgsql;"

It does require knowing which of the VALUES is the key value being inserted (pseudocode syntax above), but if that is stylized to always be the first value, that does not seem insurmountable.

---

From: Christophe Pettus <[email-masked]>
Date: Mon, 28 Apr 2025 16:01:23 -0700

> On Apr 28, 2025, at 15:58, Christophe Pettus <[email-masked]> wrote:
> It does require knowing which of the VALUES is the key value being inserted (pseudocode syntax above) [...]

The instant after I hit send, I realized that information is available to the function by lining up the $names and $values array, since the name of the key column is passed in.

---

From: Laurenz Albe <[email-masked]>
Date: Tue, 29 Apr 2025 08:36:14 +0200

> The parameters are a little bit more structured than that, but that 

Another litle "if" to cater for PostgreSQL's "EXCLUDED." would be
such a big problem?

Yours,
Laurenz Albe

---

From: Tim Starling <[email-masked]>
Date: Tue, 29 Apr 2025 21:07:47 +1000

> Another litle "if" to cater for PostgreSQL's "EXCLUDED." would be

I don't understand what you mean. EXCLUDED is not needed. "$table." 
needs to be prefixed to every column reference in the string $set. How 
do you find the column references amongst the string literals, 
function calls, etc.? You would need to parse the expression.

This is a public interface and there may be callers in code that I 
don't have access to.

Part of the reason for wanting to replace the existing emulation with 
a native upsert is to simplify the code. Parsing the expression is 
definitely not a simplification.

-- Tim Starling

---

From: David Rowley <[email-masked]>
Date: Wed, 30 Apr 2025 00:43:17 +1200

> find this argument hugely compelling, but it's an argument.)

Not arguing for or against, but... I think there are some cases where
it would be more dangerous to relax this.  Here's one case where not
qualifying the column can be dangerous:

create table a1 (a int);
insert into a1 values(1),(2);
create table a2 (a int);
insert into a2 values(1);
select * from a1 where a in(select a from a2); -- as expected.

-- application changes, a2.a isn't needed anymore. column gets dropped
but someone forgets to update a query in the app...
alter table a2 drop column a;
select * from a1 where a in(select a from a2); -- silently returns
unexpected results.

If the original author of that query had been thoughtful enough to
qualify the column in the subquery then someone would probably have
gotten an error and fixed it. The moral of that story is that
sometimes forcing the query author to qualify the column is a good
idea. (not that there's much we can do about that one...)

Now the question is, do any similar hazards exist with ON CONFLICT DO
UPDATE? I don't think so as any columns being dropped will disappear
from the insert target table and the EXCLUDED work table at the same
time.

Another thought is that you can have an UPDATE with a RETURNING
clause. An unqualified column defaults to NEW even though you could
argue it's ambiguous due to OLD (as of 80feb727c). Likely we were
forced into making it work that way through not wanting to force
everyone to rewrite their RETURNING statements when upgrading to v18.
The moral of that story is, UPDATE isn't exactly consistent already
about when it requires column qualifications. Maybe it's weird to
insist that users qualify columns with their ON CONFLICT UPDATE SET
when RETURNING is happy to assume you must have meant NEW.

David

---

From: Tom Lane <[email-masked]>
Date: Tue, 29 Apr 2025 10:51:58 -0400

> when RETURNING is happy to assume you must have meant NEW.

That's an analogy I hadn't thought of, and it does seem on-point.
You might be right that we would not have done it like that if
we'd invented RETURNING's ability to support "OLD" at the get-go.
Nonetheless, it's there now and is a pretty similar precedent.

			regards, tom lane

---

From: "David G. Johnston" <[email-masked]>
Date: Tue, 29 Apr 2025 08:29:28 -0700

On Tuesday, April 29, 2025, Tim Starling <[email-masked]> wrote:
>
>
> This is a public interface and there may be callers in code that I don't
> have access to.
>

You might help your cause by sharing examples of how client code uses your
driver to perform upsert that runs into this limitation.

David J.