Message-ID: <3ad31f4c-8caf-4b9e-9f1f-b949c6330005@illuminatedcomputing.com>
Date: Wed, 5 Jun 2024 13:57:40 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: SQL:2011 application time
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Peter Eisentraut <peter@eisentraut.org>,
 Robert Haas <robertmhaas@gmail.com>, jian he <jian.universality@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: <CA+renyUApHgSZF9-nd-a0+OPGharLQLO=mDHcY4_qQ0+noCUVg@mail.gmail.com>
 <49c07eac-18ec-4aff-929c-e81b87df1092@eisentraut.org>
 <CACJufxHsYDfHK62FTHdLsgDenvJd_bEuxCiyH62Lacj7NvzX4w@mail.gmail.com>
 <d579a67c-e8d7-4eb7-ae14-f6fca01489cf@illuminatedcomputing.com>
 <1edb3b79-a4bc-43b1-80aa-2e090472fe32@eisentraut.org>
 <47550967-260b-4180-9791-b224859fe63e@illuminatedcomputing.com>
 <CA+TgmoaH-2fAttd24pA3p45TUHbM9hRU6ud=brhKsp9ENHXWDg@mail.gmail.com>
 <1b2ab118-82c1-4e79-a094-04dab382e2c4@illuminatedcomputing.com>
 <CA+TgmobfDJuXJATNUco+g6AiUb3WS2Pi9FrJe9hKAHGw37RX-g@mail.gmail.com>
 <3775839b-3f0f-4c8a-ac03-a253222e6a4b@illuminatedcomputing.com>
 <731ef81e-d4c3-41c6-ba8f-f6131f4f6407@eisentraut.org>
 <1426589a-83cb-4a89-bf40-713970c07e63@illuminatedcomputing.com>
 <CAEze2WiD+U1BuJDLGL=FXxa8hDxNALVE6Jij0cNXjp10Q=nZHw@mail.gmail.com>
 <5aefa87e-ba20-4a34-88aa-233255643c1b@illuminatedcomputing.com>
 <CAEze2Wh21V66udM8cbvBBsAgyQ_5x9nfR0d3sWzbmZk++ey7xw@mail.gmail.com>
Content-Language: en-US
From: Paul Jungwirth <pj@illuminatedcomputing.com>
In-Reply-To: <CAEze2Wh21V66udM8cbvBBsAgyQ_5x9nfR0d3sWzbmZk++ey7xw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/3ad31f4c-8caf-4b9e-9f1f-b949c6330005%40illuminatedcomputing.com>
Precedence: bulk

On Thu, May 9, 2024 at 5:44 PM Matthias van de Meent <boekewurm+postgres@gmail.com> wrote:
 > Additionally, because I can't create my own non-constraint-backing
 > unique GIST indexes, I can't pre-create my unique constraints
 > CONCURRENTLY as one could do for the non-temporal case

We talked about this a bit at pgconf.dev. I would like to implement it, since I agree it is an 
important workflow to support. Here are some thoughts about what would need to be done.

First we could take a small step: allow non-temporal UNIQUE GiST indexes. This is possible according 
to [1], but in the past we had no way of knowing which strategy number an opclass was using for 
equality. With the stratnum support proc introduced by 6db4598fcb (reverted for v17), we could 
change amcanunique to true for the GiST AM handler. If the index's opclasses had that sproc and it 
gave non-zero for RTEqualStrategyNumber, we would have a reliable "definition of uniqueness". UNIQUE 
GiST indexes would raise an error if they detected a duplicate record.

Incidentally, this would also let us correct the error message about GiST not supporting unique, 
fixing the problem you raised here:

On Sun, May 12, 2024 at 8:51 AM Paul Jungwirth <pj@illuminatedcomputing.com> wrote:
 >
 > On 5/12/24 05:55, Matthias van de Meent wrote:
 > >>   > pg=# CREATE UNIQUE INDEX ON temporal_testing USING gist (id, valid_during);
 > >>   > ERROR:  access method "gist" does not support unique indexes
 > >>
 > >> To me that error message seems correct. The programmer hasn't said anything about the special
 > >> temporal behavior they are looking for.
 > >
 > > But I showed that I had a GIST index that does have the indisunique
 > > flag set, which shows that GIST does support indexes with unique
 > > semantics.
 > >
 > > That I can't use CREATE UNIQUE INDEX to create such an index doesn't
 > > mean the feature doesn't exist, which is what the error message
 > > implies.
 >
 > True, the error message is not really telling the truth anymore.

But that is just regular non-temporal indexes. To avoid a long table lock you'd need a way to build 
the index that is not just unique, but also does exclusion based on &&.  We could borrow syntax from 
SQL:2011 and allow `CREATE INDEX idx ON t (id, valid_at WITHOUT OVERLAPS)`. But since CREATE INDEX 
is a lower-level concept than a constraint, it'd be better to do something more general. You can 
already give opclasses for each indexed column. How about allowing operators as well? For instance 
`CREATE UNIQUE INDEX idx ON t (id WITH =, valid_at WITH &&)`? Then the index would know to enforce 
those rules. This is the same data we store today in pg_constraint.conexclops. So that would get 
moved/copied to pg_index (probably moved).

Then when you add the constraint, what is the syntax? Today when you say PRIMARY KEY/UNIQUE USING 
INDEX, you don't give the column names. So how do we know it's WITHOUT OVERLAPS? I guess if the 
underlying index has (foo WITH = [, bar WITH =], baz WITH &&) we just assume the user wants WITHOUT 
OVERLAPS, and otherwise they want a regular PK/UQ constraint?

In addition this workflow only works if you can CREATE INDEX CONCURRENTLY. I'm not sure yet if we'll 
have problems there. I noticed that for REINDEX at least, there were plans in 2012 to support 
exclusion-constraint indexes,[2] but when the patch was committed in 2019 they had been dropped, 
with plans to add support eventually.[3] Today they are still not supported. Maybe whatever caused 
problems for REINDEX isn't an issue for just INDEX, but it would take more research to find out.

[1] https://dsf.berkeley.edu/papers/sigmod97-gist.pdf
[2] Original patch thread from 2012: 
https://www.postgresql.org/message-id/flat/CAB7nPqS%2BWYN021oQHd9GPe_5dSVcVXMvEBW_E2AV9OOEwggMHw%40mail.gmail.com#e1a372074cfdf37bf9e5b4e29ddf7b2d
[3] Revised patch thread, committed in 2019: 
https://www.postgresql.org/message-id/flat/60052986-956b-4478-45ed-8bd119e9b9cf%402ndquadrant.com#74948a1044c56c5e817a5050f554ddee

Yours,

-- 
Paul              ~{:-)
pj@illuminatedcomputing.com