Message-ID: <900ad3fe-151e-4876-a399-fa12faa2c653@vondra.me>
Date: Tue, 13 Aug 2024 20:31:54 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: FDW INSERT batching can change behavior
To: git@jasonk.me, pgsql-bugs@lists.postgresql.org
References: <20240809030755.jubqv6f6vpxkfkzv@jasonk.me>
Content-Language: en-US
From: Tomas Vondra <tomas@vondra.me>
In-Reply-To: <20240809030755.jubqv6f6vpxkfkzv@jasonk.me>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/900ad3fe-151e-4876-a399-fa12faa2c653%40vondra.me>
Precedence: bulk

Hi,

I took a closer look at this, planning to way to fix this, but I think
it's actually a bit worse than reported - both in impact and ways how to
fix that.

The problem is it's not really specific to DEFAULT values. The exact
same issue exists whenever the insert uses the expressions directly.
That is, if you do this:


    insert into grem1 (a) values (counter()), (counter()),
                                 (counter()), (counter()),
                                 (counter());

it will misbehave in exactly the same way as with the default values. Of
course, this also means that my original idea to disable batching if the
foreign table has (volatile) expression in the DEFAULT value won't fly.

This can happen whenever the to-be-inserted rows have any expressions.
But those expressions are evaluated *outside* ModifyTable - in the nodes
that produce the rows. In the above example it's ValueScan. But it could
be any other node. For example:

    insert into grem1 (a) select counter() from generate_series(1,5);

does that in a subquery. But AFAICS it could be any other node.

Ideally we'd simply set batch_size=1 for those cases, but at this point
I have no idea how to check this from ModifyTable :-(

In retrospect the issue is pretty obvious, yet I haven't thought about
this while working on the batching. This is embarrassing.


regards

-- 
Tomas Vondra