($INBOX_DIR/description missing)
help / color / mirror / Atom feedRe: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c)
61+ messages / 20 participants
[nested] [flat]
* Re: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c)
@ 2018-09-11 20:13 Andrew Dunstan <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrew Dunstan @ 2018-09-11 20:13 UTC (permalink / raw)
To: Tom Lane <[email protected]>; Andres Freund <[email protected]>; +Cc: Sandeep Thakkar <[email protected]>; Peter Eisentraut <[email protected]>; Joshua D. Drake <[email protected]>; Thomas Munro <[email protected]>; David Steele <[email protected]>; PostgreSQL Hackers <[email protected]>; Craig Ringer <[email protected]>
On 08/24/2018 03:42 PM, Andrew Dunstan wrote:
>
>
> On 08/24/2018 02:38 PM, Tom Lane wrote:
>> Andres Freund <[email protected]> writes:
>>> On 2018-08-24 14:09:09 -0400, Andrew Dunstan wrote:
>>>> However, we only support VS2017 down to 9.6 and Vs2015 down to 9.5.
>>>> Perhaps
>>>> we should consider backpatching support for those down to 9.3.
>>> Hm, I have no strong objections to that. I don't think it's strictly
>>> necessary, given 2013 is supported across the board, but for the non
>>> MSVC
>>> world, we do fix compiler issues in older branches. There's not that
>>> much code for the newer versions afaict?
>> +1 for taking a look at how big a patch it would be. But I kind of
>> thought we'd intentionally rejected back-patching some of those changes
>> to begin with, so I'm not sure the end decision will change.
>
> The VS2017 patch applies cleanly to 9.5, so that seems easy. The
> VS2015 patch from 9.5 needs a very small amount of adjustment by the
> look of it for 9.3 and 9.4, after which I hope the VS2017 patch would
> again apply cleanly.
>
> I'll try to put this together.
>
> The trouble with not back patching support to all live branches as new
> versions come in is that it acts as a significant discouragement to
> buildfarm owners to use the latest Visual Studio versions. I've never
> argued stringly on this point before, but I think i'm goiung to ber
> inclined to in future.
>
> Meanwhile, I will turn bowerbird back on but just for >= 9.6 for now.
>
I've pushed support for the latest MSVC compilers back to all live branches.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c)
@ 2018-09-11 20:27 Andres Freund <[email protected]>
parent: Andrew Dunstan <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Andres Freund @ 2018-09-11 20:27 UTC (permalink / raw)
To: Andrew Dunstan <[email protected]>; +Cc: Tom Lane <[email protected]>; Sandeep Thakkar <[email protected]>; Peter Eisentraut <[email protected]>; Joshua D. Drake <[email protected]>; Thomas Munro <[email protected]>; David Steele <[email protected]>; PostgreSQL Hackers <[email protected]>; Craig Ringer <[email protected]>
Hi,
On 2018-09-11 16:13:15 -0400, Andrew Dunstan wrote:
> I've pushed support for the latest MSVC compilers back to all live branches.
Thanks!
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 61+ messages in thread
* [PATCH v1] Cleanup/remove/update references to OID column...
@ 2019-04-03 00:13 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Justin Pryzby @ 2019-04-03 00:13 UTC (permalink / raw)
To: [email protected]
..in wake of 578b229718e8f.
See also
93507e67c9ca54026019ebec3026de35d30370f9
1464755fc490a9911214817fe83077a3689250ab
---
doc/src/sgml/ddl.sgml | 9 ++++-----
doc/src/sgml/ref/insert.sgml | 12 +++++-------
doc/src/sgml/ref/psql-ref.sgml | 3 +++
3 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 9e761db..db044c5 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3672,11 +3672,10 @@ VALUES ('Albany', NULL, NULL, 'NY');
<para>
Partitions cannot have columns that are not present in the parent. It
is not possible to specify columns when creating partitions with
- <command>CREATE TABLE</command>, nor is it possible to add columns to
- partitions after-the-fact using <command>ALTER TABLE</command>. Tables may be
- added as a partition with <command>ALTER TABLE ... ATTACH PARTITION</command>
- only if their columns exactly match the parent, including any
- <literal>oid</literal> column.
+ <command>CREATE TABLE</command>, to add columns to
+ partitions after-the-fact using <command>ALTER TABLE</command>, nor to
+ add a partition with <command>ALTER TABLE ... ATTACH PARTITION</command>
+ if its columns would not exactly match those of the parent.
</para>
</listitem>
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 62e142f..3e1be4c 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -552,13 +552,11 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
INSERT <replaceable>oid</replaceable> <replaceable class="parameter">count</replaceable>
</screen>
The <replaceable class="parameter">count</replaceable> is the
- number of rows inserted or updated. If <replaceable
- class="parameter">count</replaceable> is exactly one, and the
- target table has OIDs, then <replaceable
- class="parameter">oid</replaceable> is the <acronym>OID</acronym>
- assigned to the inserted row. The single row must have been
- inserted rather than updated. Otherwise <replaceable
- class="parameter">oid</replaceable> is zero.
+ number of rows inserted or updated.
+ <replaceable>oid</replaceable> used to be the object ID of the inserted row
+ if <replaceable>rows</replaceable> was 1 and the target table had OIDs, but
+ OIDs system columns are not supported anymore; therefore
+ <replaceable>oid</replaceable> is always 0.
</para>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 08f4bab..0e6e792 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3794,6 +3794,9 @@ bar
command. This variable is only guaranteed to be valid until
after the result of the next <acronym>SQL</acronym> command has
been displayed.
+ <productname>PostgreSQL</productname> servers since version 12 do not
+ support OID system columns in user tables, and LASTOID will always be 0
+ following <command>INSERT</command>.
</para>
</listitem>
</varlistentry>
--
2.1.4
--fUYQa+Pmc3FrFX/N--
^ permalink raw reply [nested|flat] 61+ messages in thread
* [PATCH v3] Cleanup/remove/update references to OID column...
@ 2019-04-03 00:13 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Justin Pryzby @ 2019-04-03 00:13 UTC (permalink / raw)
..in wake of 578b229718e8f.
See also
93507e67c9ca54026019ebec3026de35d30370f9
1464755fc490a9911214817fe83077a3689250ab
Author: Justin Pryzby <[email protected]>
Author: Daniel Verite <[email protected]>
---
doc/src/sgml/information_schema.sgml | 4 ++--
doc/src/sgml/ref/insert.sgml | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/information_schema.sgml b/doc/src/sgml/information_schema.sgml
index 234a3bb..9c618b1 100644
--- a/doc/src/sgml/information_schema.sgml
+++ b/doc/src/sgml/information_schema.sgml
@@ -1312,8 +1312,8 @@
<para>
The view <literal>columns</literal> contains information about all
table columns (or view columns) in the database. System columns
- (<literal>ctid</literal>, etc.) are not included. Only those columns are
- shown that the current user has access to (by way of being the
+ (<literal>ctid</literal>, etc.) are not included. The only columns shown
+ are those to which the current user has access (by way of being the
owner or having some privilege).
</para>
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 189ce2a..f995a76 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -554,7 +554,7 @@ INSERT <replaceable>oid</replaceable> <replaceable class="parameter">count</repl
The <replaceable class="parameter">count</replaceable> is the number of
rows inserted or updated. <replaceable>oid</replaceable> is always 0 (it
used to be the <acronym>OID</acronym> assigned to the inserted row if
- <replaceable>rows</replaceable> was exactly one and the target table was
+ <replaceable>count</replaceable> was exactly one and the target table was
declared <literal>WITH OIDS</literal> and 0 otherwise, but creating a table
<literal>WITH OIDS</literal> is not supported anymore).
</para>
--
2.7.4
--TKYYegg/GYAC5JIZ--
^ permalink raw reply [nested|flat] 61+ messages in thread
* [PATCH v2] Cleanup/remove/update references to OID column...
@ 2019-04-03 00:13 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Justin Pryzby @ 2019-04-03 00:13 UTC (permalink / raw)
..in wake of 578b229718e8f.
See also
93507e67c9ca54026019ebec3026de35d30370f9
1464755fc490a9911214817fe83077a3689250ab
Reviewed-by: Daniel Verite <[email protected]>
---
doc/src/sgml/catalogs.sgml | 2 +-
doc/src/sgml/ddl.sgml | 12 +++++-------
doc/src/sgml/information_schema.sgml | 4 ++--
doc/src/sgml/ref/create_trigger.sgml | 2 +-
doc/src/sgml/ref/insert.sgml | 12 +++++-------
doc/src/sgml/ref/psql-ref.sgml | 3 +++
doc/src/sgml/spi.sgml | 2 +-
7 files changed, 18 insertions(+), 19 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 40ddec4..d544e60 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1047,7 +1047,7 @@
<entry></entry>
<entry>
The number of the column. Ordinary columns are numbered from 1
- up. System columns, such as <structfield>oid</structfield>,
+ up. System columns, such as <structfield>ctid</structfield>,
have (arbitrary) negative numbers.
</entry>
</row>
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 9e761db..244d5ce 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -1202,8 +1202,7 @@ CREATE TABLE circles (
<structfield>ctid</structfield> will change if it is
updated or moved by <command>VACUUM FULL</command>. Therefore
<structfield>ctid</structfield> is useless as a long-term row
- identifier. The OID, or even better a user-defined serial
- number, should be used to identify logical rows.
+ identifier. A primary key should be used to identify logical rows.
</para>
</listitem>
</varlistentry>
@@ -3672,11 +3671,10 @@ VALUES ('Albany', NULL, NULL, 'NY');
<para>
Partitions cannot have columns that are not present in the parent. It
is not possible to specify columns when creating partitions with
- <command>CREATE TABLE</command>, nor is it possible to add columns to
- partitions after-the-fact using <command>ALTER TABLE</command>. Tables may be
- added as a partition with <command>ALTER TABLE ... ATTACH PARTITION</command>
- only if their columns exactly match the parent, including any
- <literal>oid</literal> column.
+ <command>CREATE TABLE</command>, to add columns to
+ partitions after-the-fact using <command>ALTER TABLE</command>, nor to
+ add a partition with <command>ALTER TABLE ... ATTACH PARTITION</command>
+ if its columns would not exactly match those of the parent.
</para>
</listitem>
diff --git a/doc/src/sgml/information_schema.sgml b/doc/src/sgml/information_schema.sgml
index 1321ade..9c618b1 100644
--- a/doc/src/sgml/information_schema.sgml
+++ b/doc/src/sgml/information_schema.sgml
@@ -1312,8 +1312,8 @@
<para>
The view <literal>columns</literal> contains information about all
table columns (or view columns) in the database. System columns
- (<literal>oid</literal>, etc.) are not included. Only those columns are
- shown that the current user has access to (by way of being the
+ (<literal>ctid</literal>, etc.) are not included. The only columns shown
+ are those to which the current user has access (by way of being the
owner or having some privilege).
</para>
diff --git a/doc/src/sgml/ref/create_trigger.sgml b/doc/src/sgml/ref/create_trigger.sgml
index 6456105..3339a4b 100644
--- a/doc/src/sgml/ref/create_trigger.sgml
+++ b/doc/src/sgml/ref/create_trigger.sgml
@@ -465,7 +465,7 @@ UPDATE OF <replaceable>column_name1</replaceable> [, <replaceable>column_name2</
that the <literal>NEW</literal> row seen by the condition is the current value,
as possibly modified by earlier triggers. Also, a <literal>BEFORE</literal>
trigger's <literal>WHEN</literal> condition is not allowed to examine the
- system columns of the <literal>NEW</literal> row (such as <literal>oid</literal>),
+ system columns of the <literal>NEW</literal> row (such as <literal>ctid</literal>),
because those won't have been set yet.
</para>
diff --git a/doc/src/sgml/ref/insert.sgml b/doc/src/sgml/ref/insert.sgml
index 62e142f..3e1be4c 100644
--- a/doc/src/sgml/ref/insert.sgml
+++ b/doc/src/sgml/ref/insert.sgml
@@ -552,13 +552,11 @@ INSERT INTO <replaceable class="parameter">table_name</replaceable> [ AS <replac
INSERT <replaceable>oid</replaceable> <replaceable class="parameter">count</replaceable>
</screen>
The <replaceable class="parameter">count</replaceable> is the
- number of rows inserted or updated. If <replaceable
- class="parameter">count</replaceable> is exactly one, and the
- target table has OIDs, then <replaceable
- class="parameter">oid</replaceable> is the <acronym>OID</acronym>
- assigned to the inserted row. The single row must have been
- inserted rather than updated. Otherwise <replaceable
- class="parameter">oid</replaceable> is zero.
+ number of rows inserted or updated.
+ <replaceable>oid</replaceable> used to be the object ID of the inserted row
+ if <replaceable>rows</replaceable> was 1 and the target table had OIDs, but
+ OIDs system columns are not supported anymore; therefore
+ <replaceable>oid</replaceable> is always 0.
</para>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 08f4bab..0e6e792 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3794,6 +3794,9 @@ bar
command. This variable is only guaranteed to be valid until
after the result of the next <acronym>SQL</acronym> command has
been displayed.
+ <productname>PostgreSQL</productname> servers since version 12 do not
+ support OID system columns in user tables, and LASTOID will always be 0
+ following <command>INSERT</command>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/spi.sgml b/doc/src/sgml/spi.sgml
index 9b2f516..66eced6 100644
--- a/doc/src/sgml/spi.sgml
+++ b/doc/src/sgml/spi.sgml
@@ -3105,7 +3105,7 @@ int SPI_fnumber(TupleDesc <parameter>rowdesc</parameter>, const char * <paramete
<para>
If <parameter>colname</parameter> refers to a system column (e.g.,
- <literal>oid</literal>) then the appropriate negative column number will
+ <literal>ctid</literal>) then the appropriate negative column number will
be returned. The caller should be careful to test the return value
for exact equality to <symbol>SPI_ERROR_NOATTRIBUTE</symbol> to
detect an error; testing the result for less than or equal to 0 is
--
2.1.4
--dFWYt1i2NyOo1oI9--
^ permalink raw reply [nested|flat] 61+ messages in thread
* [PATCH v3 12/12] Cleanup/remove/update references to OID column...
@ 2019-04-03 00:13 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Justin Pryzby @ 2019-04-03 00:13 UTC (permalink / raw)
..in wake of 578b229718e8f.
See also
93507e67c9ca54026019ebec3026de35d30370f9
1464755fc490a9911214817fe83077a3689250ab
f6b39171f3d65155b9390c2c69bc5b3469f923a8
---
doc/src/sgml/catalogs.sgml | 2 +-
doc/src/sgml/information_schema.sgml | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d544e60..0f9c6f2 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -610,7 +610,7 @@
<entry><structfield>oid</structfield></entry>
<entry><type>oid</type></entry>
<entry></entry>
- <entry>Row identifier (hidden attribute; must be explicitly selected)</entry>
+ <entry>Row identifier</entry>
</row>
<row>
diff --git a/doc/src/sgml/information_schema.sgml b/doc/src/sgml/information_schema.sgml
index 234a3bb..9c618b1 100644
--- a/doc/src/sgml/information_schema.sgml
+++ b/doc/src/sgml/information_schema.sgml
@@ -1312,8 +1312,8 @@
<para>
The view <literal>columns</literal> contains information about all
table columns (or view columns) in the database. System columns
- (<literal>ctid</literal>, etc.) are not included. Only those columns are
- shown that the current user has access to (by way of being the
+ (<literal>ctid</literal>, etc.) are not included. The only columns shown
+ are those to which the current user has access (by way of being the
owner or having some privilege).
</para>
--
2.7.4
--cWoXeonUoKmBZSoM--
^ permalink raw reply [nested|flat] 61+ messages in thread
* [PATCH v1] Cleanup/remove/update references to OID column...
@ 2019-05-08 18:57 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Justin Pryzby @ 2019-05-08 18:57 UTC (permalink / raw)
..in wake of 578b229718e8f.
See also
93507e67c9ca54026019ebec3026de35d30370f9
1464755fc490a9911214817fe83077a3689250ab
f6b39171f3d65155b9390c2c69bc5b3469f923a8
Author: Justin Pryzby <[email protected]>
---
doc/src/sgml/catalogs.sgml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d544e60..0f9c6f2 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -610,7 +610,7 @@
<entry><structfield>oid</structfield></entry>
<entry><type>oid</type></entry>
<entry></entry>
- <entry>Row identifier (hidden attribute; must be explicitly selected)</entry>
+ <entry>Row identifier</entry>
</row>
<row>
--
2.7.4
--eDB11BtaWSyaBkpc--
^ permalink raw reply [nested|flat] 61+ messages in thread
* drop postmaster symlink
@ 2022-11-23 07:52 Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Peter Eisentraut @ 2022-11-23 07:52 UTC (permalink / raw)
To: pgsql-hackers
A little while ago we discussed briefly over in the meson thread whether
we could remove the postmaster symlink [0]. The meson build system
currently does not install a postmaster symlink. (AFAICT, the MSVC
build system does not either.) So if we want to elevate the meson build
system, we either need to add the postmaster symlink, or remove it from
the other build system(s) as well. Seeing that it's been deprecated for
a long time, I propose we just remove it. See attached patches.
[0]:
https://www.postgresql.org/message-id/[email protected]
From 179eaab96dc5215970b2b3485cf77f1a58f9d337 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sun, 13 Nov 2022 20:52:00 +0100
Subject: [PATCH 1/2] Remove gratuitous references to postmaster instead of
postgres
---
contrib/start-scripts/freebsd | 2 +-
contrib/start-scripts/linux | 2 +-
src/port/path.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/contrib/start-scripts/freebsd b/contrib/start-scripts/freebsd
index 3323237a54b4..5e22b2d2f0e4 100644
--- a/contrib/start-scripts/freebsd
+++ b/contrib/start-scripts/freebsd
@@ -29,7 +29,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# What to use to start up the postmaster. (If you want the script to wait
# until the server has started, you could use "pg_ctl start" here.)
-DAEMON="$prefix/bin/postmaster"
+DAEMON="$prefix/bin/postgres"
# What to use to shut down the postmaster
PGCTL="$prefix/bin/pg_ctl"
diff --git a/contrib/start-scripts/linux b/contrib/start-scripts/linux
index a7757162fc4b..9292855df716 100644
--- a/contrib/start-scripts/linux
+++ b/contrib/start-scripts/linux
@@ -61,7 +61,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# What to use to start up the postmaster. (If you want the script to wait
# until the server has started, you could use "pg_ctl start" here.)
-DAEMON="$prefix/bin/postmaster"
+DAEMON="$prefix/bin/postgres"
# What to use to shut down the postmaster
PGCTL="$prefix/bin/pg_ctl"
diff --git a/src/port/path.c b/src/port/path.c
index 05fe812f757b..fb64873c7a43 100644
--- a/src/port/path.c
+++ b/src/port/path.c
@@ -651,7 +651,7 @@ dir_strcmp(const char *s1, const char *s2)
* For example:
* target_path = '/usr/local/share/postgresql'
* bin_path = '/usr/local/bin'
- * my_exec_path = '/opt/pgsql/bin/postmaster'
+ * my_exec_path = '/opt/pgsql/bin/postgres'
* Given these inputs, the common prefix is '/usr/local/', the tail of
* bin_path is 'bin' which does match the last directory component of
* my_exec_path, so we would return '/opt/pgsql/share/postgresql'
--
2.38.1
From 6118b4098a50b78ca7437938117cc3ab41b72e08 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sun, 13 Nov 2022 20:52:56 +0100
Subject: [PATCH 2/2] Don't install postmaster symlink anymore
Also remove man page.
---
doc/src/sgml/ref/postmaster.sgml | 44 --------------------------------
doc/src/sgml/reference.sgml | 1 -
src/backend/Makefile | 8 +-----
3 files changed, 1 insertion(+), 52 deletions(-)
delete mode 100644 doc/src/sgml/ref/postmaster.sgml
diff --git a/doc/src/sgml/ref/postmaster.sgml b/doc/src/sgml/ref/postmaster.sgml
deleted file mode 100644
index 7b544ed0b613..000000000000
--- a/doc/src/sgml/ref/postmaster.sgml
+++ /dev/null
@@ -1,44 +0,0 @@
-<!--
-doc/src/sgml/ref/postmaster.sgml
-PostgreSQL documentation
--->
-
-<refentry id="app-postmaster">
- <indexterm zone="app-postmaster">
- <primary>postmaster</primary>
- </indexterm>
-
- <refmeta>
- <refentrytitle><application>postmaster</application></refentrytitle>
- <manvolnum>1</manvolnum>
- <refmiscinfo>Application</refmiscinfo>
- </refmeta>
-
- <refnamediv>
- <refname>postmaster</refname>
- <refpurpose><productname>PostgreSQL</productname> database server</refpurpose>
- </refnamediv>
-
- <refsynopsisdiv>
- <cmdsynopsis>
- <command>postmaster</command>
- <arg rep="repeat"><replaceable>option</replaceable></arg>
- </cmdsynopsis>
- </refsynopsisdiv>
-
- <refsect1>
- <title>Description</title>
-
- <para>
- <command>postmaster</command> is a deprecated alias of <command>postgres</command>.
- </para>
- </refsect1>
-
- <refsect1>
- <title>See Also</title>
-
- <para>
- <xref linkend="app-postgres"/>
- </para>
- </refsect1>
-</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index a3b743e8c1e7..e11b4b613075 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -289,7 +289,6 @@ <title>PostgreSQL Server Applications</title>
&pgupgrade;
&pgwaldump;
&postgres;
- &postmaster;
</reference>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 181c217fae4c..ed364543b74e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -208,12 +208,6 @@ endif
install-bin: postgres $(POSTGRES_IMP) installdirs
$(INSTALL_PROGRAM) postgres$(X) '$(DESTDIR)$(bindir)/postgres$(X)'
-ifneq ($(PORTNAME), win32)
- @rm -f '$(DESTDIR)$(bindir)/postmaster$(X)'
- ln -s postgres$(X) '$(DESTDIR)$(bindir)/postmaster$(X)'
-else
- $(INSTALL_PROGRAM) postgres$(X) '$(DESTDIR)$(bindir)/postmaster$(X)'
-endif
ifeq ($(MAKE_EXPORTS), true)
$(INSTALL_DATA) $(POSTGRES_IMP) '$(DESTDIR)$(pkglibdir)/$(POSTGRES_IMP)'
$(INSTALL_PROGRAM) $(MKLDEXPORT) '$(DESTDIR)$(pgxsdir)/$(MKLDEXPORT_DIR)/mkldexport.sh'
@@ -242,7 +236,7 @@ endif
##########################################################################
uninstall:
- rm -f '$(DESTDIR)$(bindir)/postgres$(X)' '$(DESTDIR)$(bindir)/postmaster'
+ rm -f '$(DESTDIR)$(bindir)/postgres$(X)'
ifeq ($(MAKE_EXPORTS), true)
rm -f '$(DESTDIR)$(pkglibdir)/$(POSTGRES_IMP)'
rm -f '$(DESTDIR)$(pgxsdir)/$(MKLDEXPORT_DIR)/mkldexport.sh'
--
2.38.1
Attachments:
[text/plain] 0001-Remove-gratuitous-references-to-postmaster-instead-o.patch (2.1K, ../../[email protected]/2-0001-Remove-gratuitous-references-to-postmaster-instead-o.patch)
download | inline diff:
From 179eaab96dc5215970b2b3485cf77f1a58f9d337 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sun, 13 Nov 2022 20:52:00 +0100
Subject: [PATCH 1/2] Remove gratuitous references to postmaster instead of
postgres
---
contrib/start-scripts/freebsd | 2 +-
contrib/start-scripts/linux | 2 +-
src/port/path.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/contrib/start-scripts/freebsd b/contrib/start-scripts/freebsd
index 3323237a54b4..5e22b2d2f0e4 100644
--- a/contrib/start-scripts/freebsd
+++ b/contrib/start-scripts/freebsd
@@ -29,7 +29,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# What to use to start up the postmaster. (If you want the script to wait
# until the server has started, you could use "pg_ctl start" here.)
-DAEMON="$prefix/bin/postmaster"
+DAEMON="$prefix/bin/postgres"
# What to use to shut down the postmaster
PGCTL="$prefix/bin/pg_ctl"
diff --git a/contrib/start-scripts/linux b/contrib/start-scripts/linux
index a7757162fc4b..9292855df716 100644
--- a/contrib/start-scripts/linux
+++ b/contrib/start-scripts/linux
@@ -61,7 +61,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# What to use to start up the postmaster. (If you want the script to wait
# until the server has started, you could use "pg_ctl start" here.)
-DAEMON="$prefix/bin/postmaster"
+DAEMON="$prefix/bin/postgres"
# What to use to shut down the postmaster
PGCTL="$prefix/bin/pg_ctl"
diff --git a/src/port/path.c b/src/port/path.c
index 05fe812f757b..fb64873c7a43 100644
--- a/src/port/path.c
+++ b/src/port/path.c
@@ -651,7 +651,7 @@ dir_strcmp(const char *s1, const char *s2)
* For example:
* target_path = '/usr/local/share/postgresql'
* bin_path = '/usr/local/bin'
- * my_exec_path = '/opt/pgsql/bin/postmaster'
+ * my_exec_path = '/opt/pgsql/bin/postgres'
* Given these inputs, the common prefix is '/usr/local/', the tail of
* bin_path is 'bin' which does match the last directory component of
* my_exec_path, so we would return '/opt/pgsql/share/postgresql'
--
2.38.1
[text/plain] 0002-Don-t-install-postmaster-symlink-anymore.patch (3.0K, ../../[email protected]/3-0002-Don-t-install-postmaster-symlink-anymore.patch)
download | inline diff:
From 6118b4098a50b78ca7437938117cc3ab41b72e08 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <[email protected]>
Date: Sun, 13 Nov 2022 20:52:56 +0100
Subject: [PATCH 2/2] Don't install postmaster symlink anymore
Also remove man page.
---
doc/src/sgml/ref/postmaster.sgml | 44 --------------------------------
doc/src/sgml/reference.sgml | 1 -
src/backend/Makefile | 8 +-----
3 files changed, 1 insertion(+), 52 deletions(-)
delete mode 100644 doc/src/sgml/ref/postmaster.sgml
diff --git a/doc/src/sgml/ref/postmaster.sgml b/doc/src/sgml/ref/postmaster.sgml
deleted file mode 100644
index 7b544ed0b613..000000000000
--- a/doc/src/sgml/ref/postmaster.sgml
+++ /dev/null
@@ -1,44 +0,0 @@
-<!--
-doc/src/sgml/ref/postmaster.sgml
-PostgreSQL documentation
--->
-
-<refentry id="app-postmaster">
- <indexterm zone="app-postmaster">
- <primary>postmaster</primary>
- </indexterm>
-
- <refmeta>
- <refentrytitle><application>postmaster</application></refentrytitle>
- <manvolnum>1</manvolnum>
- <refmiscinfo>Application</refmiscinfo>
- </refmeta>
-
- <refnamediv>
- <refname>postmaster</refname>
- <refpurpose><productname>PostgreSQL</productname> database server</refpurpose>
- </refnamediv>
-
- <refsynopsisdiv>
- <cmdsynopsis>
- <command>postmaster</command>
- <arg rep="repeat"><replaceable>option</replaceable></arg>
- </cmdsynopsis>
- </refsynopsisdiv>
-
- <refsect1>
- <title>Description</title>
-
- <para>
- <command>postmaster</command> is a deprecated alias of <command>postgres</command>.
- </para>
- </refsect1>
-
- <refsect1>
- <title>See Also</title>
-
- <para>
- <xref linkend="app-postgres"/>
- </para>
- </refsect1>
-</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index a3b743e8c1e7..e11b4b613075 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -289,7 +289,6 @@ <title>PostgreSQL Server Applications</title>
&pgupgrade;
&pgwaldump;
&postgres;
- &postmaster;
</reference>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 181c217fae4c..ed364543b74e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -208,12 +208,6 @@ endif
install-bin: postgres $(POSTGRES_IMP) installdirs
$(INSTALL_PROGRAM) postgres$(X) '$(DESTDIR)$(bindir)/postgres$(X)'
-ifneq ($(PORTNAME), win32)
- @rm -f '$(DESTDIR)$(bindir)/postmaster$(X)'
- ln -s postgres$(X) '$(DESTDIR)$(bindir)/postmaster$(X)'
-else
- $(INSTALL_PROGRAM) postgres$(X) '$(DESTDIR)$(bindir)/postmaster$(X)'
-endif
ifeq ($(MAKE_EXPORTS), true)
$(INSTALL_DATA) $(POSTGRES_IMP) '$(DESTDIR)$(pkglibdir)/$(POSTGRES_IMP)'
$(INSTALL_PROGRAM) $(MKLDEXPORT) '$(DESTDIR)$(pgxsdir)/$(MKLDEXPORT_DIR)/mkldexport.sh'
@@ -242,7 +236,7 @@ endif
##########################################################################
uninstall:
- rm -f '$(DESTDIR)$(bindir)/postgres$(X)' '$(DESTDIR)$(bindir)/postmaster'
+ rm -f '$(DESTDIR)$(bindir)/postgres$(X)'
ifeq ($(MAKE_EXPORTS), true)
rm -f '$(DESTDIR)$(pkglibdir)/$(POSTGRES_IMP)'
rm -f '$(DESTDIR)$(pgxsdir)/$(MKLDEXPORT_DIR)/mkldexport.sh'
--
2.38.1
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-23 14:18 Joe Conway <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Joe Conway @ 2022-11-23 14:18 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; pgsql-hackers; Devrim GUNDUZ <[email protected]>
On 11/23/22 02:52, Peter Eisentraut wrote:
> A little while ago we discussed briefly over in the meson thread whether
> we could remove the postmaster symlink [0]. The meson build system
> currently does not install a postmaster symlink. (AFAICT, the MSVC
> build system does not either.) So if we want to elevate the meson build
> system, we either need to add the postmaster symlink, or remove it from
> the other build system(s) as well. Seeing that it's been deprecated for
> a long time, I propose we just remove it. See attached patches.
I am a big +1 on removing the symlink, however it is worth pointing out
that the PGDG RPMs still use the symlink in the included systemd service
file:
8<----------
ExecStart=/usr/pgsql-15/bin/postmaster -D ${PGDATA}
8<----------
--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-23 14:28 Devrim Gündüz <[email protected]>
parent: Joe Conway <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Devrim Gündüz @ 2022-11-23 14:28 UTC (permalink / raw)
To: Joe Conway <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Hi,
On Wed, 2022-11-23 at 09:18 -0500, Joe Conway wrote:
> I am a big +1 on removing the symlink, however it is worth pointing
> out
> that the PGDG RPMs still use the symlink in the included systemd
> service
> file:
>
> 8<----------
> ExecStart=/usr/pgsql-15/bin/postmaster -D ${PGDATA}
...and it helps us to find the "main" process a bit easily.
Regards,
--
Devrim Gündüz
Open Source Solution Architect, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-23 15:07 Tom Lane <[email protected]>
parent: Devrim Gündüz <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Tom Lane @ 2022-11-23 15:07 UTC (permalink / raw)
To: Devrim Gündüz <[email protected]>; +Cc: Joe Conway <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <[email protected]> writes:
> ...and it helps us to find the "main" process a bit easily.
Hmm, that's a nontrivial point perhaps. It's certain that this
will break some other people's start scripts too. On the whole,
is it really that hard to add the symlink to the meson build?
regards, tom lane
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-23 19:50 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 2 replies; 61+ messages in thread
From: Andres Freund @ 2022-11-23 19:50 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Devrim Gündüz <[email protected]>; Joe Conway <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Hi,
On 2022-11-23 10:07:49 -0500, Tom Lane wrote:
> Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <[email protected]> writes:
> > ...and it helps us to find the "main" process a bit easily.
>
> Hmm, that's a nontrivial point perhaps. It's certain that this
> will break some other people's start scripts too.
OTOH, postmaster has been deprecated for ~15 years.
> On the whole, is it really that hard to add the symlink to the meson build?
No. Meson has a builtin command for it, just not in the meson version we're
currently requiring. We can create the symlink ourselves instead. The problem
is just detecting systems where we can't symlink and what to fall back to
there.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-23 20:10 Robert Haas <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 2 replies; 61+ messages in thread
From: Robert Haas @ 2022-11-23 20:10 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Devrim Gündüz <[email protected]>; Joe Conway <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On Wed, Nov 23, 2022 at 2:50 PM Andres Freund <[email protected]> wrote:
> On 2022-11-23 10:07:49 -0500, Tom Lane wrote:
> > Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <[email protected]> writes:
> > > ...and it helps us to find the "main" process a bit easily.
> >
> > Hmm, that's a nontrivial point perhaps. It's certain that this
> > will break some other people's start scripts too.
>
> OTOH, postmaster has been deprecated for ~15 years.
Yeah. Also, I don't think it's generally too hard to find the parent
process anyway, because at least on my system, the other ones end up
with ps display that looks like "postgres: logical replication
launcher" or whatever. The main process doesn't set the ps status
display, so that's the only one that shows a full path to the
executable in the ps status, which is how I usually spot it. That has
the advantage that it doesn't matter which name was used to launch it,
too.
I don't actually care very much whether we get rid of the postmaster
symlink or not, but if we aren't going to, we should stop calling it
deprecated. If 15 years isn't enough time to remove it, what ever will
be? I tend to think it's fairly pointless and perhaps also a bit
confusing, because the product is postgres not postmaster and people
can reasonably expect the binary name to match the product name. But
if we keep it, I don't think anything too dire will happen, either.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-23 20:32 Joe Conway <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 0 replies; 61+ messages in thread
From: Joe Conway @ 2022-11-23 20:32 UTC (permalink / raw)
To: Robert Haas <[email protected]>; Andres Freund <[email protected]>; +Cc: Tom Lane <[email protected]>; Devrim Gündüz <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On 11/23/22 15:10, Robert Haas wrote:
> On Wed, Nov 23, 2022 at 2:50 PM Andres Freund <[email protected]> wrote:
>> On 2022-11-23 10:07:49 -0500, Tom Lane wrote:
>> > Devrim =?ISO-8859-1?Q?G=FCnd=FCz?= <[email protected]> writes:
>> > > ...and it helps us to find the "main" process a bit easily.
>> >
>> > Hmm, that's a nontrivial point perhaps. It's certain that this
>> > will break some other people's start scripts too.
>>
>> OTOH, postmaster has been deprecated for ~15 years.
>
> Yeah. Also, I don't think it's generally too hard to find the parent
> process anyway, because at least on my system, the other ones end up
> with ps display that looks like "postgres: logical replication
> launcher" or whatever. The main process doesn't set the ps status
> display, so that's the only one that shows a full path to the
> executable in the ps status, which is how I usually spot it. That has
> the advantage that it doesn't matter which name was used to launch it,
> too.
Same here
> I don't actually care very much whether we get rid of the postmaster
> symlink or not, but if we aren't going to, we should stop calling it
> deprecated. If 15 years isn't enough time to remove it, what ever will
> be? I tend to think it's fairly pointless and perhaps also a bit
> confusing, because the product is postgres not postmaster and people
> can reasonably expect the binary name to match the product name. But
> if we keep it, I don't think anything too dire will happen, either.
FWIW, the reason I took note of the postmaster symlink in the first
place a few years ago was because selinux treats execution of programs
from symlinks differently than from actual files.
--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-23 20:48 Tom Lane <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 61+ messages in thread
From: Tom Lane @ 2022-11-23 20:48 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Devrim Gündüz <[email protected]>; Joe Conway <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Andres Freund <[email protected]> writes:
> On 2022-11-23 10:07:49 -0500, Tom Lane wrote:
>> On the whole, is it really that hard to add the symlink to the meson build?
> No. Meson has a builtin command for it, just not in the meson version we're
> currently requiring. We can create the symlink ourselves instead. The problem
> is just detecting systems where we can't symlink and what to fall back to
> there.
This isn't a hill I want to die on, either way. But "it's a bit
more complicated in meson" seems like a poor reason for changing
the user-visible installed fileset.
regards, tom lane
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-23 21:08 Andres Freund <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Andres Freund @ 2022-11-23 21:08 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Devrim Gündüz <[email protected]>; Joe Conway <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Hi,
On 2022-11-23 15:48:04 -0500, Tom Lane wrote:
> Andres Freund <[email protected]> writes:
> > On 2022-11-23 10:07:49 -0500, Tom Lane wrote:
> >> On the whole, is it really that hard to add the symlink to the meson build?
>
> > No. Meson has a builtin command for it, just not in the meson version we're
> > currently requiring. We can create the symlink ourselves instead. The problem
> > is just detecting systems where we can't symlink and what to fall back to
> > there.
>
> This isn't a hill I want to die on, either way. But "it's a bit
> more complicated in meson" seems like a poor reason for changing
> the user-visible installed fileset.
I wouldn't even have thought about proposing dropping the symlink if it hadn't
been deprecated forever, and I suspect Peter wouldn't have either...
I think this is a bit more more complicated than "changing the user-visible
installed fileset" because we didn't have logic to create 'postmaster' on
windows before, afaik the only OS without reliable symlink support.
Anyway, my current thinking is to have dumb OS dependent behaviour and create
a symlink everywhere but windows, where we'd just copy the file.
Or we could just continue to not install 'postmaster' on windows, because of
the potential confusion that 'postmaster.exe' differing from 'postgres.exe'
could cause.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: drop postmaster symlink
@ 2022-11-24 00:15 Daniel Gustafsson <[email protected]>
parent: Robert Haas <[email protected]>
1 sibling, 0 replies; 61+ messages in thread
From: Daniel Gustafsson @ 2022-11-24 00:15 UTC (permalink / raw)
To: Robert Haas <[email protected]>; +Cc: Andres Freund <[email protected]>; Tom Lane <[email protected]>; Devrim Gündüz <[email protected]>; Joe Conway <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
> On 23 Nov 2022, at 21:10, Robert Haas <[email protected]> wrote:
> I don't actually care very much whether we get rid of the postmaster
> symlink or not, but if we aren't going to, we should stop calling it
> deprecated. If 15 years isn't enough time to remove it, what ever will
> be?
+1. If we actively add support for something then it isn't really all that
deprecated IMHO.
--
Daniel Gustafsson https://vmware.com/
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-11 20:20 Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-11 20:20 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Sat, Nov 9, 2024 at 9:07 AM Sergey Prokhorenko
<[email protected]> wrote:
>
> On Saturday 9 November 2024 at 01:00:15 am GMT+3, Masahiko Sawada <[email protected]> wrote:
>
> > the microsecond part is working also as a counter in a sense. IT seems fine to me but I'm slightly concerned that there is no guidance of such implementation in RFC 9562.
>
> In fact, there is guidance of similar implementation in RFC 9562:
> https://datatracker.ietf.org/doc/html/rfc9562#name-monotonicity-and-counters
> "Counter Rollover Handling:"
> "Alternatively, implementations MAY increment the timestamp ahead of the actual time and reinitialize the counter."
>
Indeed, thank you.
> But in the near future, this may not be enough for the highest-performance systems.
Yeah, I'm concerned about this. That time might gradually come. That
being said, as long as rand_a part works also as a counter, it's fine.
Also, 12 bits does not differ much as Andrey Borodin mentioned. I
think in the first version it's better to start with a simple
implementation rather than over-engineering it.
Regarding the implementation, the v30 patch uses only microseconds
precision time even on platforms where nanoseconds precision is
available such as Linux. I think it's better to store the value of
(sub-milliseconds * 4096) into 12-bits of rand_a space instead of
directly storing microseconds into 10 bits space. That way, we can use
nanoseconds precision timestamps where available. On some platforms
such as macOS, the sub-milliseconds precision timestamp is restricted
to microseconds, we can consider it as a kind of special case. If
12-bits of rand_a space is not enough to guarantee monotonically in
the future, it is also possible to improve it by putting a (random)
counter into rand_b.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-15 01:44 Masahiko Sawada <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-15 01:44 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Mon, Nov 11, 2024 at 12:20 PM Masahiko Sawada <[email protected]> wrote:
>
> On Sat, Nov 9, 2024 at 9:07 AM Sergey Prokhorenko
> <[email protected]> wrote:
> >
> > On Saturday 9 November 2024 at 01:00:15 am GMT+3, Masahiko Sawada <[email protected]> wrote:
> >
> > > the microsecond part is working also as a counter in a sense. IT seems fine to me but I'm slightly concerned that there is no guidance of such implementation in RFC 9562.
> >
> > In fact, there is guidance of similar implementation in RFC 9562:
> > https://datatracker.ietf.org/doc/html/rfc9562#name-monotonicity-and-counters
> > "Counter Rollover Handling:"
> > "Alternatively, implementations MAY increment the timestamp ahead of the actual time and reinitialize the counter."
> >
>
> Indeed, thank you.
>
> > But in the near future, this may not be enough for the highest-performance systems.
>
> Yeah, I'm concerned about this. That time might gradually come. That
> being said, as long as rand_a part works also as a counter, it's fine.
> Also, 12 bits does not differ much as Andrey Borodin mentioned. I
> think in the first version it's better to start with a simple
> implementation rather than over-engineering it.
>
> Regarding the implementation, the v30 patch uses only microseconds
> precision time even on platforms where nanoseconds precision is
> available such as Linux. I think it's better to store the value of
> (sub-milliseconds * 4096) into 12-bits of rand_a space instead of
> directly storing microseconds into 10 bits space.
IIUC v29 patch implements UUIDv7 generation in this way. So I've
reviewed v29 patch and here are some review comments:
---
* Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * http://tools.ietf.org/html/rfc9562#section-4.4
*/
The new RFC doesn't have section 4.4.
---
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant 0b10 bits.
I'm concerned that "version and variant 0b10 bits" is not very clear
to readers. I think we can just mention "... except version and
variant bits".
---
+
+#ifndef WIN32
+#include <time.h>
+
+static uint64 get_real_time_ns()
+{
+ struct timespec tmp;
+
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ return tmp.tv_sec * 1000000000L + tmp.tv_nsec;
+}
+#else /* WIN32 */
+
+#include "c.h"
+#include <sysinfoapi.h>
+#include <sys/time.h>
+
+/* FILETIME of Jan 1 1970 00:00:00, the PostgreSQL epoch */
+static const unsigned __int64 epoch = UINT64CONST(116444736000000000);
+
+/*
+ * FILETIME represents the number of 100-nanosecond intervals since
+ * January 1, 1601 (UTC).
+ */
+#define FILETIME_UNITS_TO_NS UINT64CONST(100)
+
+
+/*
+ * timezone information is stored outside the kernel so tzp isn't used anymore.
+ *
+ * Note: this function is not for Win32 high precision timing purposes. See
+ * elapsed_time().
+ */
+static uint64
+get_real_time_ns()
+{
+ FILETIME file_time;
+ ULARGE_INTEGER ularge;
+
+ GetSystemTimePreciseAsFileTime(&file_time);
+ ularge.LowPart = file_time.dwLowDateTime;
+ ularge.HighPart = file_time.dwHighDateTime;
+
+ return (ularge.QuadPart - epoch) * FILETIME_UNITS_TO_NS;
+}
+#endif
I think that it's better to implement these functions in instr_time.h
or another file.
---
+/* minimum amount of ns that guarantees step of increased_clock_precision */
+#define SUB_MILLISECOND_STEP (1000000/4096 + 1)
I think we can rewrite it to:
#define NS_PER_MS INT64CONST(1000000)
#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
Which improves the readability.
Also, I think "#define NS_PER_US INT64CONST(1000)" can also be used in
many places.
---
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
I think we can make an inline function to set both variant and version
so we can use it for generating UUIDv4 and UUIDv7.
--
+ tms = uuid->data[5];
+ tms += ((uint64) uuid->data[4]) << 8;
+ tms += ((uint64) uuid->data[3]) << 16;
+ tms += ((uint64) uuid->data[2]) << 24;
+ tms += ((uint64) uuid->data[1]) << 32;
+ tms += ((uint64) uuid->data[0]) << 40;
How about rewriting these to the following for consistency with UUIDv1 codes?
tms = uuid->data[5]
+ ((uint64) uuid->data[4] << 8)
+ ((uint64) uuid->data[3] << 16)
+ ((uint64) uuid->data[2] << 24)
+ ((uint64) uuid->data[1] << 32)
+ ((uint64) uuid->data[0] << 40);
---
Thinking about the function structures more, I think we can refactor
generate_uuidv7(), uuidv7() and uuidv7_interval():
- create a function, get_clock_timestamp_ns(), that provides a
nanosecond-precision timestamp
- the returned timestamp is guaranteed to be greater than the
previous returned value.
- this function can be inlined.
- create a function, generate_uuidv7(), that takes a
nanosecond-precision timestamp as a function argument, and generate
UUIDv7 based on it.
- this function can be inlined too.
- uuidv7() gets the timestamp from get_clock_timestamp_ns() and passes
it to generate_uuidv7().
- uuidv7() gets the timestamp from get_clock_timestamp_ns(), adjusts
it based on the given interval, and passes it to generate_uuidv7().
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-16 19:06 Andrey M. Borodin <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-16 19:06 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 15 Nov 2024, at 06:44, Masahiko Sawada <[email protected]> wrote:
>
> IIUC v29 patch implements UUIDv7 generation in this way. So I've
> reviewed v29 patch and here are some review comments:
I believe I've addressed all your comments in v31 with one exception:
get_clock_timestamp_ns() does not return ascending values, because it resides in too generic place. So there's get_real_time_ns_ascending() in uuid.c.
Should we make a note about get_clock_timestamp_ns() returning only microseconds somewhere?
Also, maybe let's steal 2 random bits (from version or variant) and mix it into increased_clock_precision on MacOS?
Thank you!
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v31-0001-Implement-UUID-v7.patch (18.8K, ../../[email protected]/2-v31-0001-Implement-UUID-v7.patch)
download | inline diff:
From c4f919d4d15ec362f157736ea7a4bb49a890a437 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v31] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
For code readability this commit adds alias uuidv4() to function
gen_random_uuid().
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 ++-
src/backend/utils/adt/uuid.c | 165 +++++++++++++++++++++--
src/include/catalog/pg_proc.dat | 11 +-
src/include/portability/instr_time.h | 22 +++
src/port/win32gettimeofday.c | 17 +++
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 41 +++++-
src/test/regress/sql/uuid.sql | 18 ++-
9 files changed, 283 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1..3e6751d64c 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 73979f20ff..03161b3f87 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14213,6 +14213,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14222,12 +14230,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random).
</para>
<para>
@@ -14251,7 +14264,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc..6c5aee2e31 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,10 +13,13 @@
#include "postgres.h"
+#include <sys/time.h>
+
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
+#include "portability/instr_time.h"
#include "utils/fmgrprotos.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
@@ -37,6 +40,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline uint64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +406,24 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/*
+ * Set magic numbers for a UUID variant 3
+ * https://www.rfc-editor.org/rfc/rfc9562
+ */
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -413,20 +436,129 @@ gen_random_uuid(PG_FUNCTION_ARGS)
/*
* Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ uuid_set_version(uuid, 4);
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Aquire nanosecond reading and ensure it is ascending (on this backend)
+ */
+static inline uint64 get_real_time_ns_ascending()
+{
+ static uint64 previous_ns = 0;
+ uint64 ns = get_real_time_ns();
+
+ /* minimum amount of ns that guarantees step of UUID increased clock precision */
+#define NS_PER_MS INT64CONST(1000000)
+#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
+ if (previous_ns + SUB_MILLISECOND_STEP >= ns)
+ ns = previous_ns + SUB_MILLISECOND_STEP;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562.
+ *
+ * Monotonicity (regarding generation on given backend) is ensured with method
+ * "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * We use 12 bits in "rand_a" bits to store 1/4096 fractions of millisecond.
+ * Usage of pg_testtime indicates that such precision is available on most
+ * systems. If timestamp is not advancing between two consecutive UUID
+ * generations, previous timestamp is incremented and used instead of current
+ * timestamp.
+ */
+static Datum
+generate_uuidv7(uint64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ uint64 unix_ts_ms;
+ uint16 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /* sub-millisecond timestamp fraction (12 bits) */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << 12)) / NS_PER_MS;
+
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://www.rfc-editor.org/rfc/rfc9562#name-version-field
+ */
+ uuid_set_version(uuid, 7);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Entry point for uuidv7()
+ */
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ return generate_uuidv7(get_real_time_ns_ascending());
+}
+
+/*
+ * Entry point for uuidv7(interval)
+ */
+Datum
+uuidv7_interval(PG_FUNCTION_ARGS)
+{
+ uint64 ns = get_real_time_ns_ascending();
+ /*
+ * We are given a time shift interval as an argument.
+ * The interval represent days, monthes and years, that are not fixed
+ * number of nanoseconds. To make correct computations we call
+ * timestamptz_pl_interval() with corresponding logic. This logic is
+ * implemented with microsecond precision. So we carry nanoseconds
+ * between computations.
+ */
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ /* Convert time part of UUID to Timestamptz (ms since Postgres epoch) */
+ TimestampTz ts = (TimestampTz) (ns / 1000) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Copmute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+ /* Convert TimestampTz back and carry nanoseconds. */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * 1000 + ns % 1000;
+ return generate_uuidv7(ns);
+}
+
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +568,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +587,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * 1000) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +614,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +622,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd38..3353e9d6e3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index e66ecf34cd..9acf547b64 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -194,4 +194,26 @@ GetTimerFrequency(void)
#define INSTR_TIME_GET_MICROSEC(t) \
(INSTR_TIME_GET_NANOSEC(t) / NS_PER_US)
+#ifndef WIN32
+
+/*
+ * Read real time with high resolution. Trimmed to microseconds on MacOS.
+ */
+static inline uint64 get_real_time_ns()
+{
+ struct timespec tmp;
+
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ return tmp.tv_sec * 1000000000L + tmp.tv_nsec;
+}
+#else /* WIN32 */
+
+/*
+ * Function to read real time with all available preciscion.
+ * Prototype-only, implementation in win32gettimeofday.c
+ */
+uint64 get_real_time_ns();
+
+#endif
+
#endif /* INSTR_TIME_H */
diff --git a/src/port/win32gettimeofday.c b/src/port/win32gettimeofday.c
index 1e00f7ee14..ec46cc00fd 100644
--- a/src/port/win32gettimeofday.c
+++ b/src/port/win32gettimeofday.c
@@ -41,6 +41,7 @@ static const unsigned __int64 epoch = UINT64CONST(116444736000000000);
*/
#define FILETIME_UNITS_PER_SEC 10000000L
#define FILETIME_UNITS_PER_USEC 10
+#define FILETIME_UNITS_TO_NS 100L
/*
@@ -73,3 +74,19 @@ gettimeofday(struct timeval *tp, void *tzp)
return 0;
}
+
+/*
+ * Function to read real time with all available preciscion.
+ */
+uint64
+get_real_time_ns()
+{
+ FILETIME file_time;
+ ULARGE_INTEGER ularge;
+
+ GetSystemTimePreciseAsFileTime(&file_time);
+ ularge.LowPart = file_time.dwLowDateTime;
+ ularge.HighPart = file_time.dwHighDateTime;
+
+ return (ularge.QuadPart - epoch) * FILETIME_UNITS_TO_NS;
+}
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d..43e7180a16 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuidv4()
+uuidv7()
+uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 6026e15ed3..aa6224e81b 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,6 +168,27 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -188,8 +209,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuidv4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuidv7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index c88f6d087a..eec7f160f8 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,6 +85,19 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
-- extract functions
@@ -92,9 +105,12 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-17 18:39 Andrey M. Borodin <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-17 18:39 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 17 Nov 2024, at 00:06, Andrey M. Borodin <[email protected]> wrote:
>
> v31
There was a problem with MingWG build. I've considered all options and decided to include all necessary stuff into instr_time.h. So much fuss for these 2 bits about nanoseconds :)
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v32-0001-Implement-UUID-v7.patch (18.3K, ../../[email protected]/2-v32-0001-Implement-UUID-v7.patch)
download | inline diff:
From 4b540ae77fd3ee215e0482e520059f36d1ac0c15 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v32] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
For code readability this commit adds alias uuidv4() to function
gen_random_uuid().
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 ++-
src/backend/utils/adt/uuid.c | 163 +++++++++++++++++++++--
src/include/catalog/pg_proc.dat | 11 +-
src/include/portability/instr_time.h | 41 ++++++
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 41 +++++-
src/test/regress/sql/uuid.sql | 18 ++-
8 files changed, 283 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1..3e6751d64c 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 73979f20ff..03161b3f87 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14213,6 +14213,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14222,12 +14230,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random).
</para>
<para>
@@ -14251,7 +14264,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc..d909ae4907 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -17,6 +17,7 @@
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
+#include "portability/instr_time.h"
#include "utils/fmgrprotos.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
@@ -37,6 +38,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline uint64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +404,24 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/*
+ * Set magic numbers for a UUID variant 3
+ * https://www.rfc-editor.org/rfc/rfc9562
+ */
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -413,20 +434,129 @@ gen_random_uuid(PG_FUNCTION_ARGS)
/*
* Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
+ */
+ uuid_set_version(uuid, 4);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Aquire nanosecond reading and ensure it is ascending (on this backend)
+ */
+static inline uint64 get_real_time_ns_ascending()
+{
+ static uint64 previous_ns = 0;
+ uint64 ns = get_real_time_ns();
+
+ /* minimum amount of ns that guarantees step of UUID increased clock precision */
+#define NS_PER_MS INT64CONST(1000000)
+#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
+ if (previous_ns + SUB_MILLISECOND_STEP >= ns)
+ ns = previous_ns + SUB_MILLISECOND_STEP;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562.
+ *
+ * Monotonicity (regarding generation on given backend) is ensured with method
+ * "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * We use 12 bits in "rand_a" bits to store 1/4096 fractions of millisecond.
+ * Usage of pg_testtime indicates that such precision is available on most
+ * systems. If timestamp is not advancing between two consecutive UUID
+ * generations, previous timestamp is incremented and used instead of current
+ * timestamp.
+ */
+static Datum
+generate_uuidv7(uint64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ uint64 unix_ts_ms;
+ uint16 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /* sub-millisecond timestamp fraction (12 bits) */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << 12)) / NS_PER_MS;
+
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://www.rfc-editor.org/rfc/rfc9562#name-version-field
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ uuid_set_version(uuid, 7);
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Entry point for uuidv7()
+ */
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ return generate_uuidv7(get_real_time_ns_ascending());
+}
+
+/*
+ * Entry point for uuidv7(interval)
+ */
+Datum
+uuidv7_interval(PG_FUNCTION_ARGS)
+{
+ uint64 ns = get_real_time_ns_ascending();
+ /*
+ * We are given a time shift interval as an argument.
+ * The interval represent days, monthes and years, that are not fixed
+ * number of nanoseconds. To make correct computations we call
+ * timestamptz_pl_interval() with corresponding logic. This logic is
+ * implemented with microsecond precision. So we carry nanoseconds
+ * between computations.
+ */
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ /* Convert time part of UUID to Timestamptz (ms since Postgres epoch) */
+ TimestampTz ts = (TimestampTz) (ns / 1000) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Copmute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+ /* Convert TimestampTz back and carry nanoseconds. */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * 1000 + ns % 1000;
+ return generate_uuidv7(ns);
+}
+
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +566,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +585,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * 1000) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +612,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +620,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd38..3353e9d6e3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index e66ecf34cd..baf5678f8c 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -194,4 +194,45 @@ GetTimerFrequency(void)
#define INSTR_TIME_GET_MICROSEC(t) \
(INSTR_TIME_GET_NANOSEC(t) / NS_PER_US)
+#ifdef WIN32
+
+#include <sysinfoapi.h>
+
+#include <sys/time.h>
+
+/* FILETIME of Jan 1 1970 00:00:00, the PostgreSQL epoch */
+static const unsigned __int64 epoch = UINT64CONST(116444736000000000);
+
+#define FILETIME_UNITS_TO_NS 100L
+
+/*
+ * Read real time with high resolution. Trimmed to 100ns.
+ */
+static inline uint64 get_real_time_ns()
+{
+ FILETIME file_time;
+ ULARGE_INTEGER ularge;
+
+ GetSystemTimePreciseAsFileTime(&file_time);
+ ularge.LowPart = file_time.dwLowDateTime;
+ ularge.HighPart = file_time.dwHighDateTime;
+
+ return (ularge.QuadPart - epoch) * FILETIME_UNITS_TO_NS;
+}
+
+#else /* not WIN32 */
+
+/*
+ * Read real time with high resolution. Trimmed to microseconds on MacOS.
+ */
+static inline uint64 get_real_time_ns()
+{
+ struct timespec tmp;
+
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ return tmp.tv_sec * 1000000000L + tmp.tv_nsec;
+}
+
+#endif
+
#endif /* INSTR_TIME_H */
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d..43e7180a16 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuidv4()
+uuidv7()
+uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 6026e15ed3..aa6224e81b 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,6 +168,27 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -188,8 +209,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuidv4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuidv7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index c88f6d087a..eec7f160f8 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,6 +85,19 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
-- extract functions
@@ -92,9 +105,12 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-18 21:16 Masahiko Sawada <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-18 21:16 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Sun, Nov 17, 2024 at 10:39 AM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 17 Nov 2024, at 00:06, Andrey M. Borodin <[email protected]> wrote:
> >
> > v31
>
> There was a problem with MingWG build. I've considered all options and decided to include all necessary stuff into instr_time.h. So much fuss for these 2 bits about nanoseconds :)
>
I realized that what we do in get_real_time_ns() on Windows is
essentially the same as what we do in gettimeofday(). Probably we can
just do either clock_gettime() with CLOCK_REALTIME on unix-like
systems and gettimeofday() on Windows, and then don't change anything
in instr_time.h? We need to explain why we don't use gettimeofday() on
unix-like systems in get_real_time_ns() function.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-19 09:31 Andrey M. Borodin <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-19 09:31 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 19 Nov 2024, at 02:16, Masahiko Sawada <[email protected]> wrote:
>
> I realized that what we do in get_real_time_ns() on Windows is
> essentially the same as what we do in gettimeofday(). Probably we can
> just do either clock_gettime() with CLOCK_REALTIME on unix-like
> systems and gettimeofday() on Windows, and then don't change anything
> in instr_time.h? We need to explain why we don't use gettimeofday() on
> unix-like systems in get_real_time_ns() function.
Done.
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v33-0001-Implement-UUID-v7.patch (18.9K, ../../[email protected]/2-v33-0001-Implement-UUID-v7.patch)
download | inline diff:
From 19527327ff6be78e50d275e95beaf3ffd8365542 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v33] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
For code readability this commit adds alias uuidv4() to function
gen_random_uuid().
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 ++-
src/backend/utils/adt/uuid.c | 167 +++++++++++++++++++++--
src/include/catalog/pg_proc.dat | 11 +-
src/include/port/win32_port.h | 14 ++
src/port/win32gettimeofday.c | 22 +++
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 41 +++++-
src/test/regress/sql/uuid.sql | 18 ++-
9 files changed, 282 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1c..3e6751d64cc 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 73979f20fff..03161b3f874 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14213,6 +14213,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14222,12 +14230,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random).
</para>
<para>
@@ -14251,7 +14264,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc4..f8b8b590216 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -17,6 +17,7 @@
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
+#include "portability/instr_time.h"
#include "utils/fmgrprotos.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
@@ -37,6 +38,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline uint64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +404,24 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/*
+ * Set magic numbers for a UUID variant 3
+ * https://www.rfc-editor.org/rfc/rfc9562
+ */
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -413,20 +434,133 @@ gen_random_uuid(PG_FUNCTION_ARGS)
/*
* Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ uuid_set_version(uuid, 4);
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Aquire nanosecond reading and ensure it is ascending (on this backend)
+ */
+static inline uint64 get_real_time_ns_ascending()
+{
+ static uint64 previous_ns = 0;
+ struct timespec tmp;
+ uint64 ns;
+
+ /* We use some bits of nanosecond precision, so we cannot resort to gettimeofday() */
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ ns = tmp.tv_sec * 1000000000L + tmp.tv_nsec;
+
+ /* minimum amount of ns that guarantees step of UUID increased clock precision */
+#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
+ if (previous_ns + SUB_MILLISECOND_STEP >= ns)
+ ns = previous_ns + SUB_MILLISECOND_STEP;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562.
+ *
+ * Monotonicity (regarding generation on given backend) is ensured with method
+ * "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * We use 12 bits in "rand_a" bits to store 1/4096 fractions of millisecond.
+ * Usage of pg_testtime indicates that such precision is available on most
+ * systems. If timestamp is not advancing between two consecutive UUID
+ * generations, previous timestamp is incremented and used instead of current
+ * timestamp.
+ */
+static Datum
+generate_uuidv7(uint64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ uint64 unix_ts_ms;
+ uint16 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /* sub-millisecond timestamp fraction (12 bits) */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << 12)) / NS_PER_MS;
+
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://www.rfc-editor.org/rfc/rfc9562#name-version-field
+ */
+ uuid_set_version(uuid, 7);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Entry point for uuidv7()
+ */
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ return generate_uuidv7(get_real_time_ns_ascending());
+}
+
+/*
+ * Entry point for uuidv7(interval)
+ */
+Datum
+uuidv7_interval(PG_FUNCTION_ARGS)
+{
+ uint64 ns = get_real_time_ns_ascending();
+ /*
+ * We are given a time shift interval as an argument.
+ * The interval represent days, monthes and years, that are not fixed
+ * number of nanoseconds. To make correct computations we call
+ * timestamptz_pl_interval() with corresponding logic. This logic is
+ * implemented with microsecond precision. So we carry nanoseconds
+ * between computations.
+ */
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ /* Convert time part of UUID to Timestamptz (ms since Postgres epoch) */
+ TimestampTz ts = (TimestampTz) (ns / 1000) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Copmute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+ /* Convert TimestampTz back and carry nanoseconds. */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * 1000 + ns % 1000;
+ return generate_uuidv7(ns);
+}
+
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +570,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +589,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * 1000) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +616,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +624,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd382..3353e9d6e36 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/include/port/win32_port.h b/src/include/port/win32_port.h
index 7789e0431aa..d343e6c1875 100644
--- a/src/include/port/win32_port.h
+++ b/src/include/port/win32_port.h
@@ -184,6 +184,20 @@
#ifdef _MSC_VER
/* Last parameter not used */
extern int gettimeofday(struct timeval *tp, void *tzp);
+
+/*
+ * Windows implementation is limited to CLOCK_REALTIME
+ */
+typedef enum {
+ CLOCK_REALTIME
+} clockid_t;
+
+#include <time.h> /* for timespec */
+
+extern int clock_gettime(clockid_t clock_id, struct timespec *tp);
+#else
+/* MinGW */
+#include "pthread_time.h"
#endif
/* for setitimer in backend/port/win32/timer.c */
diff --git a/src/port/win32gettimeofday.c b/src/port/win32gettimeofday.c
index 1e00f7ee149..93ec3bf731c 100644
--- a/src/port/win32gettimeofday.c
+++ b/src/port/win32gettimeofday.c
@@ -41,6 +41,7 @@ static const unsigned __int64 epoch = UINT64CONST(116444736000000000);
*/
#define FILETIME_UNITS_PER_SEC 10000000L
#define FILETIME_UNITS_PER_USEC 10
+#define FILETIME_UNITS_TO_NS 100L
/*
@@ -73,3 +74,24 @@ gettimeofday(struct timeval *tp, void *tzp)
return 0;
}
+
+/*
+ * This function is ported for UUID purposes.
+ */
+int
+clock_gettime(clockid_t clock_id, struct timespec *tp)
+{
+ Assert(clock_id == CLOCK_REALTIME);
+
+ FILETIME file_time;
+ ULARGE_INTEGER ularge;
+ GetSystemTimePreciseAsFileTime(&file_time);
+ ularge.LowPart = file_time.dwLowDateTime;
+ ularge.HighPart = file_time.dwHighDateTime;
+
+ tp->tv_sec = (long) ((ularge.QuadPart - epoch) / FILETIME_UNITS_PER_SEC);
+ tp->tv_nsec = (long) (((ularge.QuadPart - epoch) % FILETIME_UNITS_PER_SEC)
+ * FILETIME_UNITS_TO_NS);
+
+ return 0;
+}
\ No newline at end of file
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d2..43e7180a161 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuidv4()
+uuidv7()
+uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 6026e15ed31..aa6224e81bb 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,6 +168,27 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -188,8 +209,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuidv4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuidv7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index c88f6d087a7..eec7f160f81 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,6 +85,19 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
-- extract functions
@@ -92,9 +105,12 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
--
2.42.0
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-19 17:44 Andrey M. Borodin <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-19 17:44 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 19 Nov 2024, at 14:31, Andrey M. Borodin <[email protected]> wrote:
>
> Done.
Here's v33 intact + one more patch to add 2 bits of entropy on MacOS (to compensate lack of nanoseconds).
What do you think?
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v33-0001-Implement-UUID-v7.patch (18.9K, ../../[email protected]/2-v33-0001-Implement-UUID-v7.patch)
download | inline diff:
From 19527327ff6be78e50d275e95beaf3ffd8365542 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v33] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
For code readability this commit adds alias uuidv4() to function
gen_random_uuid().
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 ++-
src/backend/utils/adt/uuid.c | 167 +++++++++++++++++++++--
src/include/catalog/pg_proc.dat | 11 +-
src/include/port/win32_port.h | 14 ++
src/port/win32gettimeofday.c | 22 +++
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 41 +++++-
src/test/regress/sql/uuid.sql | 18 ++-
9 files changed, 282 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1..3e6751d64c 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 73979f20ff..03161b3f87 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14213,6 +14213,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14222,12 +14230,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random).
</para>
<para>
@@ -14251,7 +14264,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc..f8b8b59021 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -17,6 +17,7 @@
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
+#include "portability/instr_time.h"
#include "utils/fmgrprotos.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
@@ -37,6 +38,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline uint64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +404,24 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/*
+ * Set magic numbers for a UUID variant 3
+ * https://www.rfc-editor.org/rfc/rfc9562
+ */
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -413,20 +434,133 @@ gen_random_uuid(PG_FUNCTION_ARGS)
/*
* Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ uuid_set_version(uuid, 4);
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Aquire nanosecond reading and ensure it is ascending (on this backend)
+ */
+static inline uint64 get_real_time_ns_ascending()
+{
+ static uint64 previous_ns = 0;
+ struct timespec tmp;
+ uint64 ns;
+
+ /* We use some bits of nanosecond precision, so we cannot resort to gettimeofday() */
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ ns = tmp.tv_sec * 1000000000L + tmp.tv_nsec;
+
+ /* minimum amount of ns that guarantees step of UUID increased clock precision */
+#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
+ if (previous_ns + SUB_MILLISECOND_STEP >= ns)
+ ns = previous_ns + SUB_MILLISECOND_STEP;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562.
+ *
+ * Monotonicity (regarding generation on given backend) is ensured with method
+ * "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * We use 12 bits in "rand_a" bits to store 1/4096 fractions of millisecond.
+ * Usage of pg_testtime indicates that such precision is available on most
+ * systems. If timestamp is not advancing between two consecutive UUID
+ * generations, previous timestamp is incremented and used instead of current
+ * timestamp.
+ */
+static Datum
+generate_uuidv7(uint64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ uint64 unix_ts_ms;
+ uint16 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /* sub-millisecond timestamp fraction (12 bits) */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << 12)) / NS_PER_MS;
+
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://www.rfc-editor.org/rfc/rfc9562#name-version-field
+ */
+ uuid_set_version(uuid, 7);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Entry point for uuidv7()
+ */
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ return generate_uuidv7(get_real_time_ns_ascending());
+}
+
+/*
+ * Entry point for uuidv7(interval)
+ */
+Datum
+uuidv7_interval(PG_FUNCTION_ARGS)
+{
+ uint64 ns = get_real_time_ns_ascending();
+ /*
+ * We are given a time shift interval as an argument.
+ * The interval represent days, monthes and years, that are not fixed
+ * number of nanoseconds. To make correct computations we call
+ * timestamptz_pl_interval() with corresponding logic. This logic is
+ * implemented with microsecond precision. So we carry nanoseconds
+ * between computations.
+ */
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ /* Convert time part of UUID to Timestamptz (ms since Postgres epoch) */
+ TimestampTz ts = (TimestampTz) (ns / 1000) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Copmute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+ /* Convert TimestampTz back and carry nanoseconds. */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * 1000 + ns % 1000;
+ return generate_uuidv7(ns);
+}
+
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +570,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +589,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * 1000) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +616,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +624,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd38..3353e9d6e3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/include/port/win32_port.h b/src/include/port/win32_port.h
index 7789e0431a..d343e6c187 100644
--- a/src/include/port/win32_port.h
+++ b/src/include/port/win32_port.h
@@ -184,6 +184,20 @@
#ifdef _MSC_VER
/* Last parameter not used */
extern int gettimeofday(struct timeval *tp, void *tzp);
+
+/*
+ * Windows implementation is limited to CLOCK_REALTIME
+ */
+typedef enum {
+ CLOCK_REALTIME
+} clockid_t;
+
+#include <time.h> /* for timespec */
+
+extern int clock_gettime(clockid_t clock_id, struct timespec *tp);
+#else
+/* MinGW */
+#include "pthread_time.h"
#endif
/* for setitimer in backend/port/win32/timer.c */
diff --git a/src/port/win32gettimeofday.c b/src/port/win32gettimeofday.c
index 1e00f7ee14..93ec3bf731 100644
--- a/src/port/win32gettimeofday.c
+++ b/src/port/win32gettimeofday.c
@@ -41,6 +41,7 @@ static const unsigned __int64 epoch = UINT64CONST(116444736000000000);
*/
#define FILETIME_UNITS_PER_SEC 10000000L
#define FILETIME_UNITS_PER_USEC 10
+#define FILETIME_UNITS_TO_NS 100L
/*
@@ -73,3 +74,24 @@ gettimeofday(struct timeval *tp, void *tzp)
return 0;
}
+
+/*
+ * This function is ported for UUID purposes.
+ */
+int
+clock_gettime(clockid_t clock_id, struct timespec *tp)
+{
+ Assert(clock_id == CLOCK_REALTIME);
+
+ FILETIME file_time;
+ ULARGE_INTEGER ularge;
+ GetSystemTimePreciseAsFileTime(&file_time);
+ ularge.LowPart = file_time.dwLowDateTime;
+ ularge.HighPart = file_time.dwHighDateTime;
+
+ tp->tv_sec = (long) ((ularge.QuadPart - epoch) / FILETIME_UNITS_PER_SEC);
+ tp->tv_nsec = (long) (((ularge.QuadPart - epoch) % FILETIME_UNITS_PER_SEC)
+ * FILETIME_UNITS_TO_NS);
+
+ return 0;
+}
\ No newline at end of file
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d..43e7180a16 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuidv4()
+uuidv7()
+uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 6026e15ed3..aa6224e81b 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,6 +168,27 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -188,8 +209,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuidv4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuidv7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index c88f6d087a..eec7f160f8 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,6 +85,19 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
-- extract functions
@@ -92,9 +105,12 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
--
2.39.5 (Apple Git-154)
[application/octet-stream] v34-0002-Mix-in-2-bits-of-entropy-into-timestampt-of-UUID.patch (1.2K, ../../[email protected]/3-v34-0002-Mix-in-2-bits-of-entropy-into-timestampt-of-UUID.patch)
download | inline diff:
From 2deed80fbd15edeff04f9c5925b7897715cfe437 Mon Sep 17 00:00:00 2001
From: Andrey Borodin <[email protected]>
Date: Tue, 19 Nov 2024 22:41:32 +0500
Subject: [PATCH v34 2/2] Mix in 2 bits of entropy into timestampt of UUID on
MacOS
---
src/backend/utils/adt/uuid.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index f8b8b59021..b251f2b10d 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -503,6 +503,17 @@ generate_uuidv7(uint64 ns)
(errcode(ERRCODE_INTERNAL_ERROR),
errmsg("could not generate random values")));
+#if defined(__darwin__)
+ /*
+ * On MacOS real time is truncted to microseconds. Thus, 2 least
+ * significant bits of increased_clock_precision are neither random
+ * (CSPRNG), nor time-dependent (in a sense - truly random). These 2 bits
+ * are dependent on other time-specific bits, thus they do not contribute
+ * to uniqueness. To make these bit random we mix in two bits from CSPRNG.
+ */
+ uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
+#endif
+
/*
* Set magic numbers for a "version 7" (pseudorandom) UUID, see
* https://www.rfc-editor.org/rfc/rfc9562#name-version-field
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-19 19:06 Masahiko Sawada <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-19 19:06 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Tue, Nov 19, 2024 at 9:45 AM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 19 Nov 2024, at 14:31, Andrey M. Borodin <[email protected]> wrote:
> >
> > Done.
>
> Here's v33 intact + one more patch to add 2 bits of entropy on MacOS (to compensate lack of nanoseconds).
> What do you think?
>
Thank you for updating the patch!
I've reviewed the v33 patch and made some changes mostly for cosmetic
things. Please review it to see if we accept these changes.
I have one question about the additional patch:
+#if defined(__darwin__)
+ /*
+ * On MacOS real time is truncted to microseconds. Thus, 2 least
+ * significant bits of increased_clock_precision are neither random
+ * (CSPRNG), nor time-dependent (in a sense - truly random). These 2 bits
+ * are dependent on other time-specific bits, thus they do not contribute
+ * to uniqueness. To make these bit random we mix in two bits from CSPRNG.
+ */
+ uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
+#endif
I thought that the whole 12 bits in "rand_a" is actually
time-dependent since we store 1/4096 fraction of sub-milliseconds. Am
I missing something?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
[application/octet-stream] change_v33.patch (12.6K, ../../CAD21AoBTPXU5s6WZyANZdwdvfRNjUHjFxWD1xnvUG9gJeQDfGg@mail.gmail.com/2-change_v33.patch)
download | inline diff:
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index f8b8b590216..ac094ac5901 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,17 +13,30 @@
#include "postgres.h"
+#include <time.h> /* for clock_gettime() */
+
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
-#include "portability/instr_time.h"
#include "utils/fmgrprotos.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
#include "utils/timestamp.h"
#include "utils/uuid.h"
+/* helper macros */
+#define NS_PER_S INT64CONST(1000000000)
+#define NS_PER_MS INT64CONST(1000000)
+#define NS_PER_US INT64CONST(1000)
+
+/*
+ * In UUID version 7, we use 12 bits in "rand_a" to store 1/4096 fractions of
+ * sub-millisecond. This is the minimum amount of nanoseconds that guarantees
+ * step of UUID increased clock precision.
+ */
+#define SUBMS_MINIMAL_STEP ((NS_PER_MS / (1 << 12)) + 1)
+
/* sortsupport for uuid */
typedef struct
{
@@ -39,7 +52,7 @@ static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
-static inline uint64 get_real_time_ns_ascending();
+static inline int64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -404,14 +417,13 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
-/*
- * Set magic numbers for a UUID variant 3
- * https://www.rfc-editor.org/rfc/rfc9562
- */
-static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+/* Set the given UUID version and the variant bits */
+static inline void
+uuid_set_version(pg_uuid_t *uuid, unsigned char version)
{
/* set version field, top four bits */
uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+
/* set variant field, top two bits are 1, 0 */
uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
}
@@ -433,8 +445,8 @@ gen_random_uuid(PG_FUNCTION_ARGS)
errmsg("could not generate random values")));
/*
- * Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
+ * Set magic numbers for a "version 4" (pseudorandom) UUID and
+ * variant, see https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
*/
uuid_set_version(uuid, 4);
@@ -442,44 +454,63 @@ gen_random_uuid(PG_FUNCTION_ARGS)
}
/*
- * Aquire nanosecond reading and ensure it is ascending (on this backend)
+ * Get the current timestamp with nanosecond precision for UUID generation.
+ * The returned timestamp is ensured to be at least SUBMS_MINIMAL_STEP greater
+ * than the previous returned timestamp (on this backend).
*/
-static inline uint64 get_real_time_ns_ascending()
+static inline int64
+get_real_time_ns_ascending()
{
- static uint64 previous_ns = 0;
+ static int64 previous_ns = 0;
+ int64 ns;
+
+ /* Get the current real timestamp */
+
+#ifdef WIN32
+ struct timeval tmp;
+
+ gettimeofday(&tp, NULL);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_usec * NS_PER_US;
+#else
struct timespec tmp;
- uint64 ns;
- /* We use some bits of nanosecond precision, so we cannot resort to gettimeofday() */
+ /*
+ * We don't use gettimeofday() where available, instead use clock_gettime()
+ * with CLOCK_REALTIME in order to get a high-precision (nanoseconds) real
+ * timestamp.
+ *
+ * Note that a timestamp returned by clock_gettime() with CLOCK_REALTIME
+ * is nanosecond-precision on most Unix-like platforms. On some platforms
+ * such as macOS, it's restricted to microsecond-precision.
+ */
clock_gettime(CLOCK_REALTIME, &tmp);
- ns = tmp.tv_sec * 1000000000L + tmp.tv_nsec;
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+#endif
- /* minimum amount of ns that guarantees step of UUID increased clock precision */
-#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
- if (previous_ns + SUB_MILLISECOND_STEP >= ns)
- ns = previous_ns + SUB_MILLISECOND_STEP;
+ /* Guarantee the minimal step advancement of the timestamp */
+ if (previous_ns + SUBMS_MINIMAL_STEP >= ns)
+ ns = previous_ns + SUBMS_MINIMAL_STEP;
previous_ns = ns;
return ns;
}
/*
- * Generate UUID version 7 per RFC 9562.
+ * Generate UUID version 7 per RFC 9562, with the given timestamp.
*
- * Monotonicity (regarding generation on given backend) is ensured with method
- * "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
- * We use 12 bits in "rand_a" bits to store 1/4096 fractions of millisecond.
- * Usage of pg_testtime indicates that such precision is available on most
- * systems. If timestamp is not advancing between two consecutive UUID
- * generations, previous timestamp is incremented and used instead of current
- * timestamp.
+ * UUID version 7 consists of a Unix timestamp in milliseconds (48 bits) and
+ * 74 random bits, excluding the required version and variant bits. To ensure
+ * monotonicity in scenarios of high-frequency UUID generation, we employ the
+ * method "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * This method utilizes 12 bits from the "rand_a" bits to store a 1/4096
+ * (or 2^12) fraction of sub-millisecond precision.
*/
-static Datum
-generate_uuidv7(uint64 ns)
+static pg_attribute_always_inline pg_uuid_t *
+generate_uuidv7(int64 ns)
{
pg_uuid_t *uuid = palloc(UUID_LEN);
- uint64 unix_ts_ms;
- uint16 increased_clock_precision;
+ int64 unix_ts_ms;
+ int32 increased_clock_precision;
unix_ts_ms = ns / NS_PER_MS;
@@ -494,6 +525,7 @@ generate_uuidv7(uint64 ns)
/* sub-millisecond timestamp fraction (12 bits) */
increased_clock_precision = ((ns % NS_PER_MS) * (1 << 12)) / NS_PER_MS;
+ /* Fill the increased clock precision to "rand_a" bits */
uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
uuid->data[7] = (unsigned char) (increased_clock_precision);
@@ -504,51 +536,62 @@ generate_uuidv7(uint64 ns)
errmsg("could not generate random values")));
/*
- * Set magic numbers for a "version 7" (pseudorandom) UUID, see
- * https://www.rfc-editor.org/rfc/rfc9562#name-version-field
+ * Set magic numbers for a "version 7" (pseudorandom) UUID and
+ * variant, see https://www.rfc-editor.org/rfc/rfc9562#name-version-field
*/
uuid_set_version(uuid, 7);
- PG_RETURN_UUID_P(uuid);
+ return uuid;
}
/*
- * Entry point for uuidv7()
+ * Generate UUID version 7 with the current timestamp.
*/
Datum
uuidv7(PG_FUNCTION_ARGS)
{
- return generate_uuidv7(get_real_time_ns_ascending());
+ pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
+
+ PG_RETURN_UUID_P(uuid);
}
/*
- * Entry point for uuidv7(interval)
+ * Similar to uuidv7() but with the timestamp adjusted by the given interval.
*/
Datum
uuidv7_interval(PG_FUNCTION_ARGS)
{
- uint64 ns = get_real_time_ns_ascending();
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ TimestampTz ts;
+ pg_uuid_t *uuid;
+ int64 ns = get_real_time_ns_ascending();
+
/*
- * We are given a time shift interval as an argument.
- * The interval represent days, monthes and years, that are not fixed
- * number of nanoseconds. To make correct computations we call
- * timestamptz_pl_interval() with corresponding logic. This logic is
- * implemented with microsecond precision. So we carry nanoseconds
- * between computations.
+ * Shift the current timestamp by the given interval. To make correct
+ * calculating the time shift, we convert the UNIX epoch to TimestampTz
+ * and use timestamptz_pl_interval(). Since this calculation is done with
+ * microsecond precision, we carry back the nanoseconds.
*/
- Interval *span = PG_GETARG_INTERVAL_P(0);
- /* Convert time part of UUID to Timestamptz (ms since Postgres epoch) */
- TimestampTz ts = (TimestampTz) (ns / 1000) -
+
+ ts = (TimestampTz) (ns / NS_PER_US) -
(POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
/* Copmute time shift */
ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
- TimestampTzGetDatum(ts),
- IntervalPGetDatum(span)));
- /* Convert TimestampTz back and carry nanoseconds. */
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+
+ /*
+ * Convert a TimestampTz value back to an UNIX epoch and carry back
+ * nanoseconds.
+ */
ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
- * 1000 + ns % 1000;
- return generate_uuidv7(ns);
+ * NS_PER_US + ns % NS_PER_US;
+
+ /* Generate an UUID */
+ uuid = generate_uuidv7(ns);
+
+ PG_RETURN_UUID_P(uuid);
}
/*
@@ -603,7 +646,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
+ (((uint64) uuid->data[0]) << 40);
/* convert ms to us, then adjust */
- ts = (TimestampTz) (tms * 1000) -
+ ts = (TimestampTz) (tms * NS_PER_US) -
(POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
diff --git a/src/include/port/win32_port.h b/src/include/port/win32_port.h
index d343e6c1875..7789e0431aa 100644
--- a/src/include/port/win32_port.h
+++ b/src/include/port/win32_port.h
@@ -184,20 +184,6 @@
#ifdef _MSC_VER
/* Last parameter not used */
extern int gettimeofday(struct timeval *tp, void *tzp);
-
-/*
- * Windows implementation is limited to CLOCK_REALTIME
- */
-typedef enum {
- CLOCK_REALTIME
-} clockid_t;
-
-#include <time.h> /* for timespec */
-
-extern int clock_gettime(clockid_t clock_id, struct timespec *tp);
-#else
-/* MinGW */
-#include "pthread_time.h"
#endif
/* for setitimer in backend/port/win32/timer.c */
diff --git a/src/port/win32gettimeofday.c b/src/port/win32gettimeofday.c
index 93ec3bf731c..1e00f7ee149 100644
--- a/src/port/win32gettimeofday.c
+++ b/src/port/win32gettimeofday.c
@@ -41,7 +41,6 @@ static const unsigned __int64 epoch = UINT64CONST(116444736000000000);
*/
#define FILETIME_UNITS_PER_SEC 10000000L
#define FILETIME_UNITS_PER_USEC 10
-#define FILETIME_UNITS_TO_NS 100L
/*
@@ -74,24 +73,3 @@ gettimeofday(struct timeval *tp, void *tzp)
return 0;
}
-
-/*
- * This function is ported for UUID purposes.
- */
-int
-clock_gettime(clockid_t clock_id, struct timespec *tp)
-{
- Assert(clock_id == CLOCK_REALTIME);
-
- FILETIME file_time;
- ULARGE_INTEGER ularge;
- GetSystemTimePreciseAsFileTime(&file_time);
- ularge.LowPart = file_time.dwLowDateTime;
- ularge.HighPart = file_time.dwHighDateTime;
-
- tp->tv_sec = (long) ((ularge.QuadPart - epoch) / FILETIME_UNITS_PER_SEC);
- tp->tv_nsec = (long) (((ularge.QuadPart - epoch) % FILETIME_UNITS_PER_SEC)
- * FILETIME_UNITS_TO_NS);
-
- return 0;
-}
\ No newline at end of file
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index aa6224e81bb..bd83f6b0763 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
INSERT INTO guid1(guid_field) VALUES('11111111-1111-1111-1111-111111111111F');
@@ -189,6 +194,14 @@ SELECT count(DISTINCT guid_field) FROM guid1;
3
(1 row)
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
+ array_agg
+------------------------
+ {1,2,3,4,5,6,7,8,9,10}
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -247,4 +260,4 @@ SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index eec7f160f81..8e54217a75c 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
@@ -98,6 +103,9 @@ INSERT INTO guid1 (guid_field) VALUES (uuidv7());
INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
-- extract functions
@@ -116,4 +124,4 @@ SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-20 03:48 Andrey M. Borodin <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-20 03:48 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 20 Nov 2024, at 00:06, Masahiko Sawada <[email protected]> wrote:
>
> On Tue, Nov 19, 2024 at 9:45 AM Andrey M. Borodin <[email protected]> wrote:
>>
>>
>>
>>> On 19 Nov 2024, at 14:31, Andrey M. Borodin <[email protected]> wrote:
>>>
>>> Done.
>>
>> Here's v33 intact + one more patch to add 2 bits of entropy on MacOS (to compensate lack of nanoseconds).
>> What do you think?
>>
>
> Thank you for updating the patch!
>
> I've reviewed the v33 patch and made some changes mostly for cosmetic
> things. Please review it to see if we accept these changes.
Your changes look good to me. I particularly like sortability test.
I see that you removed implementation of clock_gettime() for Windows. Well, this makes sense.
>
> I have one question about the additional patch:
>
> +#if defined(__darwin__)
> + /*
> + * On MacOS real time is truncted to microseconds. Thus, 2 least
> + * significant bits of increased_clock_precision are neither random
> + * (CSPRNG), nor time-dependent (in a sense - truly random). These 2 bits
> + * are dependent on other time-specific bits, thus they do not contribute
> + * to uniqueness. To make these bit random we mix in two bits from CSPRNG.
> + */
> + uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
> +#endif
>
> I thought that the whole 12 bits in "rand_a" is actually
> time-dependent since we store 1/4096 fraction of sub-milliseconds. Am
> I missing something?
We have 12 bits in increaesd_clock_precission but only 1000 possible values of these bits. 2 least significant bits are defined by other 10 bits.
These bits are not equal to 0, they are changing.
True, these bits are time-dependent in a sense that these bits are be computed from a full timestamp. I wanted to express the fact that timestamp cannot be altered in a way so only these 2 bits are changed.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-20 23:24 Masahiko Sawada <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-20 23:24 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Tue, Nov 19, 2024 at 7:54 PM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 20 Nov 2024, at 00:06, Masahiko Sawada <[email protected]> wrote:
> >
> > On Tue, Nov 19, 2024 at 9:45 AM Andrey M. Borodin <[email protected]> wrote:
> >>
> >>
> >>
> >>> On 19 Nov 2024, at 14:31, Andrey M. Borodin <[email protected]> wrote:
> >>>
> >>> Done.
> >>
> >> Here's v33 intact + one more patch to add 2 bits of entropy on MacOS (to compensate lack of nanoseconds).
> >> What do you think?
> >>
> >
> > Thank you for updating the patch!
> >
> > I've reviewed the v33 patch and made some changes mostly for cosmetic
> > things. Please review it to see if we accept these changes.
>
> Your changes look good to me. I particularly like sortability test.
> I see that you removed implementation of clock_gettime() for Windows. Well, this makes sense.
>
>
> >
> > I have one question about the additional patch:
> >
> > +#if defined(__darwin__)
> > + /*
> > + * On MacOS real time is truncted to microseconds. Thus, 2 least
> > + * significant bits of increased_clock_precision are neither random
> > + * (CSPRNG), nor time-dependent (in a sense - truly random). These 2 bits
> > + * are dependent on other time-specific bits, thus they do not contribute
> > + * to uniqueness. To make these bit random we mix in two bits from CSPRNG.
> > + */
> > + uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
> > +#endif
> >
> > I thought that the whole 12 bits in "rand_a" is actually
> > time-dependent since we store 1/4096 fraction of sub-milliseconds. Am
> > I missing something?
>
> We have 12 bits in increaesd_clock_precission but only 1000 possible values of these bits. 2 least significant bits are defined by other 10 bits.
> These bits are not equal to 0, they are changing.
> True, these bits are time-dependent in a sense that these bits are be computed from a full timestamp. I wanted to express the fact that timestamp cannot be altered in a way so only these 2 bits are changed.
Understood the idea. But does replacing the least significant 2 bits
with random 2 bits really not affect monotonicity? The ensured minimal
timestamp step is 245, ((NS_PER_MS / (1 << 12)) + 1), meaning that if
two UUIDs are generated within a microsecond on macOS, the two
timestamps differ by 245 ns. After calculating the increased clock
precision with these two timestamps, they differ only by 1, which
seems to be problematic to me.
Suppose the two timestamps are:
ns1: 1732142033754429000 (Nov 20, 2024 10:33:53.754429000)
ns2: 1732142033754429245 (Nov 20, 2024 10:33:53.754429245)
Their sub-milliseconds are calculated (by multiplying by 4096) to:
subms1: 1757 (0b011011011101)
subms2: 1758 (0b011011011110)
If we replace the least significant bits '01' of subms1 with random
bits '11' and replace '10' of subms2 with '00', we cannot guarantee
the monotonicity.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-21 21:22 Andrey M. Borodin <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-21 21:22 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 21 Nov 2024, at 02:24, Masahiko Sawada <[email protected]> wrote:
>
> But does replacing the least significant 2 bits
> with random 2 bits really not affect monotonicity?
You are right. We have to take into account this when calculating monotonicity. PFA another version.
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v35-0002-Mix-in-2-bits-of-entropy-into-timestampt-of-UUID.patch (1.8K, ../../[email protected]/2-v35-0002-Mix-in-2-bits-of-entropy-into-timestampt-of-UUID.patch)
download | inline diff:
From 328032b5ec850e7eacc135e00e8d5f63c23d8058 Mon Sep 17 00:00:00 2001
From: Andrey Borodin <[email protected]>
Date: Tue, 19 Nov 2024 22:41:32 +0500
Subject: [PATCH v35 2/2] Mix in 2 bits of entropy into timestampt of UUID on
MacOS
---
src/backend/utils/adt/uuid.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index f8b8b59021..25cbe62803 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -455,7 +455,12 @@ static inline uint64 get_real_time_ns_ascending()
ns = tmp.tv_sec * 1000000000L + tmp.tv_nsec;
/* minimum amount of ns that guarantees step of UUID increased clock precision */
-#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
+#if defined(__darwin__) || _MSC_VER
+#define SUB_MILLISECOND_BITS 10
+#else
+#define SUB_MILLISECOND_BITS 12
+#endif
+#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << SUB_MILLISECOND_BITS)) + 1)
if (previous_ns + SUB_MILLISECOND_STEP >= ns)
ns = previous_ns + SUB_MILLISECOND_STEP;
previous_ns = ns;
@@ -503,6 +508,17 @@ generate_uuidv7(uint64 ns)
(errcode(ERRCODE_INTERNAL_ERROR),
errmsg("could not generate random values")));
+#if defined(__darwin__) || _MSC_VER
+ /*
+ * On MacOS real time is truncted to microseconds. Thus, 2 least
+ * significant bits of increased_clock_precision are neither random
+ * (CSPRNG), nor time-dependent (in a sense - truly random). These 2 bits
+ * are dependent on other time-specific bits, thus they do not contribute
+ * to uniqueness. To make these bit random we mix in two bits from CSPRNG.
+ */
+ uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
+#endif
+
/*
* Set magic numbers for a "version 7" (pseudorandom) UUID, see
* https://www.rfc-editor.org/rfc/rfc9562#name-version-field
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-22 22:37 Masahiko Sawada <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-22 22:37 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Thu, Nov 21, 2024 at 1:22 PM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 21 Nov 2024, at 02:24, Masahiko Sawada <[email protected]> wrote:
> >
> > But does replacing the least significant 2 bits
> > with random 2 bits really not affect monotonicity?
>
> You are right. We have to take into account this when calculating monotonicity. PFA another version.
>
While it works fine, I think we need a comment for this change:
-#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
+#if defined(__darwin__) || _MSC_VER
+#define SUB_MILLISECOND_BITS 10
+#else
+#define SUB_MILLISECOND_BITS 12
+#endif
+#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << SUB_MILLISECOND_BITS)) + 1)
because the reader might think we should use SUB_MILLISECOND_BITS
here too at a glance:
+ /* sub-millisecond timestamp fraction (12 bits) */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << 12)) / NS_PER_MS;
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-23 07:58 Masahiko Sawada <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-23 07:58 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Fri, Nov 22, 2024 at 2:37 PM Masahiko Sawada <[email protected]> wrote:
>
> On Thu, Nov 21, 2024 at 1:22 PM Andrey M. Borodin <[email protected]> wrote:
> >
> >
> >
> > > On 21 Nov 2024, at 02:24, Masahiko Sawada <[email protected]> wrote:
> > >
> > > But does replacing the least significant 2 bits
> > > with random 2 bits really not affect monotonicity?
> >
> > You are right. We have to take into account this when calculating monotonicity. PFA another version.
> >
>
> While it works fine, I think we need a comment for this change:
>
> -#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << 12)) + 1)
> +#if defined(__darwin__) || _MSC_VER
> +#define SUB_MILLISECOND_BITS 10
> +#else
> +#define SUB_MILLISECOND_BITS 12
> +#endif
> +#define SUB_MILLISECOND_STEP ((NS_PER_MS / (1 << SUB_MILLISECOND_BITS)) + 1)
>
> because the reader might think we should use SUB_MILLISECOND_BITS
> here too at a glance:
>
> + /* sub-millisecond timestamp fraction (12 bits) */
> + increased_clock_precision = ((ns % NS_PER_MS) * (1 << 12)) / NS_PER_MS;
>
I've attached an updated patch that squashed changes I made for v33.
We're still discussing increasing entropy on Windows and macOS, but
the patch seems to be in good shape.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
[application/octet-stream] v36-0001-Implement-UUID-v7.patch (19.7K, ../../CAD21AoAe9b_GxxyRGXuWHXh8-K4KdqupV-38SCttEc-gpGWGMQ@mail.gmail.com/2-v36-0001-Implement-UUID-v7.patch)
download | inline diff:
From d93ab6dc33b8623c1ba4aac6faf2f0c1f9cd2473 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v36] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
For code readability this commit adds alias uuidv4() to function
gen_random_uuid().
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 ++-
src/backend/utils/adt/uuid.c | 213 +++++++++++++++++++++--
src/include/catalog/pg_proc.dat | 11 +-
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 56 +++++-
src/test/regress/sql/uuid.sql | 28 ++-
7 files changed, 314 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1c..3e6751d64cc 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 13ccbe7d78c..a1af74b69cc 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14213,6 +14213,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14222,12 +14230,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random).
</para>
<para>
@@ -14251,7 +14264,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc4..b137805696f 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,6 +13,8 @@
#include "postgres.h"
+#include <time.h> /* for clock_gettime() */
+
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
@@ -23,6 +25,19 @@
#include "utils/timestamp.h"
#include "utils/uuid.h"
+/* helper macros */
+#define NS_PER_S INT64CONST(1000000000)
+#define NS_PER_MS INT64CONST(1000000)
+#define NS_PER_US INT64CONST(1000)
+
+/*
+ * In UUID version 7, we use 12 bits in "rand_a" to store 1/4096 fractions of
+ * sub-millisecond. This is the minimum amount of nanoseconds that guarantees
+ * step advancement of sub-millisecond part.
+ */
+#define SUBMS_BITS 12
+#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_BITS)) + 1)
+
/* sortsupport for uuid */
typedef struct
{
@@ -37,6 +52,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline int64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +418,23 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/* Set the given UUID version and the variant bits */
+static inline void
+uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -412,21 +446,165 @@ gen_random_uuid(PG_FUNCTION_ARGS)
errmsg("could not generate random values")));
/*
- * Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * Set magic numbers for a "version 4" (pseudorandom) UUID and variant,
+ * see https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
+ */
+ uuid_set_version(uuid, 4);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Get the current timestamp with nanosecond precision for UUID generation.
+ * The returned timestamp is ensured to be at least SUBMS_MINIMAL_STEP greater
+ * than the previous returned timestamp (on this backend).
+ */
+static inline int64
+get_real_time_ns_ascending()
+{
+ static int64 previous_ns = 0;
+ int64 ns;
+
+ /* Get the current real timestamp */
+
+#ifdef _MSC_VER
+ struct timeval tmp;
+
+ gettimeofday(&tmp, NULL);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_usec * NS_PER_US;
+#else
+ struct timespec tmp;
+
+ /*
+ * We don't use gettimeofday() where available, instead use
+ * clock_gettime() with CLOCK_REALTIME in order to get a high-precision
+ * (nanoseconds) real timestamp.
+ *
+ * Note that a timestamp returned by clock_gettime() with CLOCK_REALTIME
+ * is nanosecond-precision on most Unix-like platforms. On some platforms
+ * such as macOS, it's restricted to microsecond-precision.
+ */
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+#endif
+
+ /* Guarantee the minimal step advancement of the timestamp */
+ if (previous_ns + SUBMS_MINIMAL_STEP_NS >= ns)
+ ns = previous_ns + SUBMS_MINIMAL_STEP_NS;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562, with the given timestamp.
+ *
+ * UUID version 7 consists of a Unix timestamp in milliseconds (48 bits) and
+ * 74 random bits, excluding the required version and variant bits. To ensure
+ * monotonicity in scenarios of high-frequency UUID generation, we employ the
+ * method "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * This method utilizes 12 bits from the "rand_a" bits to store a 1/4096
+ * (or 2^12) fraction of sub-millisecond precision.
+ */
+static pg_attribute_always_inline pg_uuid_t *
+generate_uuidv7(int64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ int64 unix_ts_ms;
+ int32 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /* sub-millisecond timestamp fraction (12 bits) */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+
+ /* Fill the increased clock precision to "rand_a" bits */
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID and variant,
+ * see https://www.rfc-editor.org/rfc/rfc9562#name-version-field
+ */
+ uuid_set_version(uuid, 7);
+
+ return uuid;
+}
+
+/*
+ * Generate UUID version 7 with the current timestamp.
+ */
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Similar to uuidv7() but with the timestamp adjusted by the given interval.
+ */
+Datum
+uuidv7_interval(PG_FUNCTION_ARGS)
+{
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ TimestampTz ts;
+ pg_uuid_t *uuid;
+ int64 ns = get_real_time_ns_ascending();
+
+ /*
+ * Shift the current timestamp by the given interval. To make correct
+ * calculating the time shift, we convert the UNIX epoch to TimestampTz
+ * and use timestamptz_pl_interval(). Since this calculation is done with
+ * microsecond precision, we carry back the nanoseconds.
+ */
+
+ ts = (TimestampTz) (ns / NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Copmute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+
+ /*
+ * Convert a TimestampTz value back to an UNIX epoch and carry back
+ * nanoseconds.
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * NS_PER_US + ns % NS_PER_US;
+
+ /* Generate an UUID */
+ uuid = generate_uuidv7(ns);
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +614,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +633,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +660,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +668,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd382..3353e9d6e36 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d2..43e7180a161 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuidv4()
+uuidv7()
+uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 6026e15ed31..bd83f6b0763 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
INSERT INTO guid1(guid_field) VALUES('11111111-1111-1111-1111-111111111111F');
@@ -168,6 +173,35 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
+ array_agg
+------------------------
+ {1,2,3,4,5,6,7,8,9,10}
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -188,8 +222,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuidv4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuidv7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
@@ -208,4 +260,4 @@ SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index c88f6d087a7..8e54217a75c 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
@@ -85,6 +90,22 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
-- extract functions
@@ -92,12 +113,15 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
--
2.43.5
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-23 08:20 Andrey M. Borodin <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 2 replies; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-23 08:20 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 23 Nov 2024, at 10:58, Masahiko Sawada <[email protected]> wrote:
>
> I've attached an updated patch that squashed changes I made for v33.
> We're still discussing increasing entropy on Windows and macOS, but
> the patch seems to be in good shape.
+1, thanks!
PFA version with improved comment.
Sergey Prokhorenko just draw my attention to the new release of MariaDB [0]. They are doing very similar UUID v7 generation as we do [1].
Best regards, Andrey Borodin.
[0] https://mariadb.com/resources/blog/announcing-mariadb-community-server-11-7-rc-with-vector-search-an...
[1] https://github.com/mariadb/server/blob/main/plugin/type_uuid/sql_type_uuid_v7.h#L32
Attachments:
[application/octet-stream] v37-0001-Implement-UUID-v7.patch (19.7K, ../../[email protected]/2-v37-0001-Implement-UUID-v7.patch)
download | inline diff:
From 216a6269605f09c363d8e64f0e368588aa5cd4e6 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v37 1/2] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
For code readability this commit adds alias uuidv4() to function
gen_random_uuid().
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 ++-
src/backend/utils/adt/uuid.c | 213 +++++++++++++++++++++--
src/include/catalog/pg_proc.dat | 11 +-
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 56 +++++-
src/test/regress/sql/uuid.sql | 28 ++-
7 files changed, 314 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1..3e6751d64c 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 73979f20ff..03161b3f87 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14213,6 +14213,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14222,12 +14230,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random).
</para>
<para>
@@ -14251,7 +14264,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc..b137805696 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,6 +13,8 @@
#include "postgres.h"
+#include <time.h> /* for clock_gettime() */
+
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
@@ -23,6 +25,19 @@
#include "utils/timestamp.h"
#include "utils/uuid.h"
+/* helper macros */
+#define NS_PER_S INT64CONST(1000000000)
+#define NS_PER_MS INT64CONST(1000000)
+#define NS_PER_US INT64CONST(1000)
+
+/*
+ * In UUID version 7, we use 12 bits in "rand_a" to store 1/4096 fractions of
+ * sub-millisecond. This is the minimum amount of nanoseconds that guarantees
+ * step advancement of sub-millisecond part.
+ */
+#define SUBMS_BITS 12
+#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_BITS)) + 1)
+
/* sortsupport for uuid */
typedef struct
{
@@ -37,6 +52,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline int64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +418,23 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/* Set the given UUID version and the variant bits */
+static inline void
+uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -412,21 +446,165 @@ gen_random_uuid(PG_FUNCTION_ARGS)
errmsg("could not generate random values")));
/*
- * Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * Set magic numbers for a "version 4" (pseudorandom) UUID and variant,
+ * see https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
+ */
+ uuid_set_version(uuid, 4);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Get the current timestamp with nanosecond precision for UUID generation.
+ * The returned timestamp is ensured to be at least SUBMS_MINIMAL_STEP greater
+ * than the previous returned timestamp (on this backend).
+ */
+static inline int64
+get_real_time_ns_ascending()
+{
+ static int64 previous_ns = 0;
+ int64 ns;
+
+ /* Get the current real timestamp */
+
+#ifdef _MSC_VER
+ struct timeval tmp;
+
+ gettimeofday(&tmp, NULL);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_usec * NS_PER_US;
+#else
+ struct timespec tmp;
+
+ /*
+ * We don't use gettimeofday() where available, instead use
+ * clock_gettime() with CLOCK_REALTIME in order to get a high-precision
+ * (nanoseconds) real timestamp.
+ *
+ * Note that a timestamp returned by clock_gettime() with CLOCK_REALTIME
+ * is nanosecond-precision on most Unix-like platforms. On some platforms
+ * such as macOS, it's restricted to microsecond-precision.
+ */
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+#endif
+
+ /* Guarantee the minimal step advancement of the timestamp */
+ if (previous_ns + SUBMS_MINIMAL_STEP_NS >= ns)
+ ns = previous_ns + SUBMS_MINIMAL_STEP_NS;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562, with the given timestamp.
+ *
+ * UUID version 7 consists of a Unix timestamp in milliseconds (48 bits) and
+ * 74 random bits, excluding the required version and variant bits. To ensure
+ * monotonicity in scenarios of high-frequency UUID generation, we employ the
+ * method "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * This method utilizes 12 bits from the "rand_a" bits to store a 1/4096
+ * (or 2^12) fraction of sub-millisecond precision.
+ */
+static pg_attribute_always_inline pg_uuid_t *
+generate_uuidv7(int64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ int64 unix_ts_ms;
+ int32 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /* sub-millisecond timestamp fraction (12 bits) */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+
+ /* Fill the increased clock precision to "rand_a" bits */
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID and variant,
+ * see https://www.rfc-editor.org/rfc/rfc9562#name-version-field
+ */
+ uuid_set_version(uuid, 7);
+
+ return uuid;
+}
+
+/*
+ * Generate UUID version 7 with the current timestamp.
+ */
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Similar to uuidv7() but with the timestamp adjusted by the given interval.
+ */
+Datum
+uuidv7_interval(PG_FUNCTION_ARGS)
+{
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ TimestampTz ts;
+ pg_uuid_t *uuid;
+ int64 ns = get_real_time_ns_ascending();
+
+ /*
+ * Shift the current timestamp by the given interval. To make correct
+ * calculating the time shift, we convert the UNIX epoch to TimestampTz
+ * and use timestamptz_pl_interval(). Since this calculation is done with
+ * microsecond precision, we carry back the nanoseconds.
+ */
+
+ ts = (TimestampTz) (ns / NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Copmute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+
+ /*
+ * Convert a TimestampTz value back to an UNIX epoch and carry back
+ * nanoseconds.
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * NS_PER_US + ns % NS_PER_US;
+
+ /* Generate an UUID */
+ uuid = generate_uuidv7(ns);
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +614,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +633,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +660,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +668,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd38..3353e9d6e3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d..43e7180a16 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuidv4()
+uuidv7()
+uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 6026e15ed3..bd83f6b076 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
INSERT INTO guid1(guid_field) VALUES('11111111-1111-1111-1111-111111111111F');
@@ -168,6 +173,35 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
+ array_agg
+------------------------
+ {1,2,3,4,5,6,7,8,9,10}
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -188,8 +222,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuidv4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuidv7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
@@ -208,4 +260,4 @@ SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index c88f6d087a..8e54217a75 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
@@ -85,6 +90,22 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
-- extract functions
@@ -92,12 +113,15 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
--
2.39.5 (Apple Git-154)
[application/octet-stream] v37-0002-Mix-in-2-bits-of-entropy-into-timestampt-of-UUID.patch (2.7K, ../../[email protected]/3-v37-0002-Mix-in-2-bits-of-entropy-into-timestampt-of-UUID.patch)
download | inline diff:
From 6f16c43c40bda7cf663c56b3efdb7f58ecd367ea Mon Sep 17 00:00:00 2001
From: Andrey Borodin <[email protected]>
Date: Sat, 23 Nov 2024 11:15:49 +0300
Subject: [PATCH v37 2/2] Mix in 2 bits of entropy into timestampt of UUID on
MacOS
---
src/backend/utils/adt/uuid.c | 33 ++++++++++++++++++++++++++++-----
1 file changed, 28 insertions(+), 5 deletions(-)
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index b137805696..ce8cf64908 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -31,12 +31,19 @@
#define NS_PER_US INT64CONST(1000)
/*
- * In UUID version 7, we use 12 bits in "rand_a" to store 1/4096 fractions of
- * sub-millisecond. This is the minimum amount of nanoseconds that guarantees
- * step advancement of sub-millisecond part.
+ * In UUID version 7, we use 12 bits in "rand_a" to store 1/4096
+ * fractions of sub-millisecond. On systems that have only 10 bits of sub-ms
+ * precision we still use 1/4096 parts of a millisecond, but fill lower 2 bits
+ * with random numbers. SUBMS_MINIMAL_STEP is the minimum amount of
+ * nanoseconds that guarantees step of UUID increased clock precision.
*/
+#if defined(__darwin__) || defined(_MSC_VER)
+#define SUBMS_MINIMAL_STEP_BITS 10
+#else
+#define SUBMS_MINIMAL_STEP_BITS 12
+#endif
#define SUBMS_BITS 12
-#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_BITS)) + 1)
+#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_MINIMAL_STEP_BITS)) + 1)
/* sortsupport for uuid */
typedef struct
@@ -523,7 +530,10 @@ generate_uuidv7(int64 ns)
uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
uuid->data[5] = (unsigned char) unix_ts_ms;
- /* sub-millisecond timestamp fraction (12 bits) */
+ /*
+ * sub-millisecond timestamp fraction (SUBMS_BITS bits, not
+ * SUBMS_MINIMAL_STEP_BITS)
+ */
increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
/* Fill the increased clock precision to "rand_a" bits */
@@ -536,6 +546,19 @@ generate_uuidv7(int64 ns)
(errcode(ERRCODE_INTERNAL_ERROR),
errmsg("could not generate random values")));
+#if defined(__darwin__) || defined(WIN32)
+ /*
+ * On MacOS real time is truncted to microseconds. Thus, 2 least
+ * significant are dependent on other time-specific bits, thus they do not
+ * contribute to uniqueness. To make these bit random we mix in two bits
+ * from CSPRNG.
+ *
+ * SUBMS_MINIMAL_STEP is chosen so that we still guarantee monotonicity
+ * despite altering these bits.
+ */
+ uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
+#endif
+
/*
* Set magic numbers for a "version 7" (pseudorandom) UUID and variant,
* see https://www.rfc-editor.org/rfc/rfc9562#name-version-field
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-25 06:08 wenhui qiu <[email protected]>
parent: Andrey M. Borodin <[email protected]>
1 sibling, 0 replies; 61+ messages in thread
From: wenhui qiu @ 2024-11-25 06:08 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
HI Andrey M. Borodin
It's not just mariadb, percona also implements the uuid plugin.
https://docs.percona.com/percona-server/8.4/uuid-versions.html#functions-available-in-uuid_vx
Thanks
Andrey M. Borodin <[email protected]> 于2024年11月23日周六 16:21写道：
>
>
> > On 23 Nov 2024, at 10:58, Masahiko Sawada <[email protected]> wrote:
> >
> > I've attached an updated patch that squashed changes I made for v33.
> > We're still discussing increasing entropy on Windows and macOS, but
> > the patch seems to be in good shape.
>
> +1, thanks!
>
> PFA version with improved comment.
>
> Sergey Prokhorenko just draw my attention to the new release of MariaDB
> [0]. They are doing very similar UUID v7 generation as we do [1].
>
>
> Best regards, Andrey Borodin.
>
>
> [0]
> https://mariadb.com/resources/blog/announcing-mariadb-community-server-11-7-rc-with-vector-search-an...
> [1]
> https://github.com/mariadb/server/blob/main/plugin/type_uuid/sql_type_uuid_v7.h#L32
>
>
>
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-25 17:53 Masahiko Sawada <[email protected]>
parent: Andrey M. Borodin <[email protected]>
1 sibling, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-25 17:53 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Sat, Nov 23, 2024 at 12:20 AM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 23 Nov 2024, at 10:58, Masahiko Sawada <[email protected]> wrote:
> >
> > I've attached an updated patch that squashed changes I made for v33.
> > We're still discussing increasing entropy on Windows and macOS, but
> > the patch seems to be in good shape.
>
> +1, thanks!
>
> PFA version with improved comment.
Thank you for updating the patch!
In the following code, we use "defined(__darwin__) || defined(_MSC_VER)":
+#if defined(__darwin__) || defined(_MSC_VER)
+#define SUBMS_MINIMAL_STEP_BITS 10
+#else
+#define SUBMS_MINIMAL_STEP_BITS 12
+#endif
#define SUBMS_BITS 12
-#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_BITS)) + 1)
+#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 <<
SUBMS_MINIMAL_STEP_BITS)) + 1)
on the other hand, we use "defined(__darwin__) || defined(WIN32)" here:
+#if defined(__darwin__) || defined(WIN32)
+ /*
+ * On MacOS real time is truncted to microseconds. Thus, 2 least
+ * significant are dependent on other time-specific bits, thus
they do not
+ * contribute to uniqueness. To make these bit random we mix in two bits
+ * from CSPRNG.
+ *
+ * SUBMS_MINIMAL_STEP is chosen so that we still guarantee monotonicity
+ * despite altering these bits.
+ */
+ uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
+#endif
Is there a reason for using different macros?
In get_real_time_ns_ascending(), we use _MSC_VER so we use
clock_gettime() on MinGW.
>
> Sergey Prokhorenko just draw my attention to the new release of MariaDB [0]. They are doing very similar UUID v7 generation as we do [1].
>
Thank you for the references. It made me think that we can use the
function name uuid_v7() rather than uuidv7().
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-25 18:14 Andrey M. Borodin <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-25 18:14 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 25 Nov 2024, at 22:53, Masahiko Sawada <[email protected]> wrote:
>
> In the following code, we use "defined(__darwin__) || defined(_MSC_VER)":
>
> +#if defined(__darwin__) || defined(_MSC_VER)
> +#define SUBMS_MINIMAL_STEP_BITS 10
> +#else
> +#define SUBMS_MINIMAL_STEP_BITS 12
> +#endif
> #define SUBMS_BITS 12
> -#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_BITS)) + 1)
> +#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 <<
> SUBMS_MINIMAL_STEP_BITS)) + 1)
>
> on the other hand, we use "defined(__darwin__) || defined(WIN32)" here:
>
> +#if defined(__darwin__) || defined(WIN32)
> + /*
> + * On MacOS real time is truncted to microseconds. Thus, 2 least
> + * significant are dependent on other time-specific bits, thus
> they do not
> + * contribute to uniqueness. To make these bit random we mix in two bits
> + * from CSPRNG.
> + *
> + * SUBMS_MINIMAL_STEP is chosen so that we still guarantee monotonicity
> + * despite altering these bits.
> + */
> + uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
> +#endif
>
> Is there a reason for using different macros?
No, that's an oversight. We should mix these 2 bits if an only if SUBMS_MINIMAL_STEP_BITS=10.
<tldr>
In your review change_v33.patch you used WIN32, but it did not actually compile on Windows.
So on Saturday I squashed v33+change_v33.patch, and composed a message that I think we still should switch to _MSC_VER. And just before sending I received your message with v36 where you used _MSC_VER :)
I think this way:
_MSC_VER - native Windows without clock_gettime, we used gettimeofday() and 10 bits of sub-ms.
MinGW - we use clock_gettime() and 12 bits.
Darwin - we use clock_gettime() and 10 bits.
Anything else - clock_gettime() and 12 bits.
</tldr>
>
> In get_real_time_ns_ascending(), we use _MSC_VER so we use
> clock_gettime() on MinGW.
>
>>
>> Sergey Prokhorenko just draw my attention to the new release of MariaDB [0]. They are doing very similar UUID v7 generation as we do [1].
>>
>
> Thank you for the references. It made me think that we can use the
> function name uuid_v7() rather than uuidv7().
I think it's a good idea if we will be kind of SQL-compatible.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-25 20:11 Masahiko Sawada <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 3 replies; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-25 20:11 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Mon, Nov 25, 2024 at 10:15 AM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 25 Nov 2024, at 22:53, Masahiko Sawada <[email protected]> wrote:
> >
> > In the following code, we use "defined(__darwin__) || defined(_MSC_VER)":
> >
> > +#if defined(__darwin__) || defined(_MSC_VER)
> > +#define SUBMS_MINIMAL_STEP_BITS 10
> > +#else
> > +#define SUBMS_MINIMAL_STEP_BITS 12
> > +#endif
> > #define SUBMS_BITS 12
> > -#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_BITS)) + 1)
> > +#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 <<
> > SUBMS_MINIMAL_STEP_BITS)) + 1)
> >
> > on the other hand, we use "defined(__darwin__) || defined(WIN32)" here:
> >
> > +#if defined(__darwin__) || defined(WIN32)
> > + /*
> > + * On MacOS real time is truncted to microseconds. Thus, 2 least
> > + * significant are dependent on other time-specific bits, thus
> > they do not
> > + * contribute to uniqueness. To make these bit random we mix in two bits
> > + * from CSPRNG.
> > + *
> > + * SUBMS_MINIMAL_STEP is chosen so that we still guarantee monotonicity
> > + * despite altering these bits.
> > + */
> > + uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
> > +#endif
> >
> > Is there a reason for using different macros?
>
> No, that's an oversight. We should mix these 2 bits if an only if SUBMS_MINIMAL_STEP_BITS=10.
>
> <tldr>
> In your review change_v33.patch you used WIN32, but it did not actually compile on Windows.
> So on Saturday I squashed v33+change_v33.patch, and composed a message that I think we still should switch to _MSC_VER. And just before sending I received your message with v36 where you used _MSC_VER :)
>
> I think this way:
> _MSC_VER - native Windows without clock_gettime, we used gettimeofday() and 10 bits of sub-ms.
> MinGW - we use clock_gettime() and 12 bits.
> Darwin - we use clock_gettime() and 10 bits.
> Anything else - clock_gettime() and 12 bits.
> </tldr>
Thank you for the summary.
On MinGW, IIUC we can get 100-ns precision timestamps[1], so using 12
bits for calculating the minimal step would make sense.
Also, if we implement the Windows port of clock_gettime() in the
future, we can remove the part of using gettimeofday() in
get_real_time_ns_ascending(). It seems to me that it's
over-engineering to implement that part only for the UUID v7. So the
current implementation of get_real_time_ns_ascending() makes sense to
me.
>
> >
> > In get_real_time_ns_ascending(), we use _MSC_VER so we use
> > clock_gettime() on MinGW.
> >
> >>
> >> Sergey Prokhorenko just draw my attention to the new release of MariaDB [0]. They are doing very similar UUID v7 generation as we do [1].
> >>
> >
> > Thank you for the references. It made me think that we can use the
> > function name uuid_v7() rather than uuidv7().
>
> I think it's a good idea if we will be kind of SQL-compatible.
>
Okay, let"s rename it.
I've merged patches and renamed functions (also updated the commit
message). Please find the attachment.
Regards,
[1] https://github.com/Alexpux/mingw-w64/blob/d0d7f784833bbb0b2d279310ddc6afb52fe47a46/mingw-w64-librari...
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
[application/octet-stream] v38-0001-Add-UUID-version-7-generation-function.patch (20.8K, ../../CAD21AoCtHQ5F6-T9hUU-s=JpNC7hmCU0EOVtr4FVqOvdPH=Vyg@mail.gmail.com/2-v38-0001-Add-UUID-version-7-generation-function.patch)
download | inline diff:
From c589d9d5ea6ff5d814d2ad4813b8d7cc933bd0e2 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v38] Add UUID version 7 generation function.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit implements the uuid_v7() function for generating UUID
version 7, as defined in RFC 9652. UUID v7 comprises a Unix timestamp
in milliseconds and random bits, providing uniqueness and
sortability.
In our implementation, the 12-bit sub-millisecond timestamp fraction
is stored immediately after the timestamp, referred to as "rand_a" in
the RFC. This ensures additional monotonicity within a millisecond.
Additionally, an alias uuid_v4() is added for the existing
gen_random_uuid() function to maintain consistency.
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 +-
src/backend/utils/adt/uuid.c | 237 ++++++++++++++++++++++-
src/include/catalog/pg_proc.dat | 11 +-
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 56 +++++-
src/test/regress/sql/uuid.sql | 28 ++-
7 files changed, 338 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1c..3e6751d64cc 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 13ccbe7d78c..ab1d42ae759 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14213,6 +14213,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuid_v4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_v7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14222,12 +14230,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv_4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv_7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with millisecond
+ precision + sub-millisecond + random).
</para>
<para>
@@ -14251,7 +14264,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc4..4c907fa80aa 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,6 +13,8 @@
#include "postgres.h"
+#include <time.h> /* for clock_gettime() */
+
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
@@ -23,6 +25,26 @@
#include "utils/timestamp.h"
#include "utils/uuid.h"
+/* helper macros */
+#define NS_PER_S INT64CONST(1000000000)
+#define NS_PER_MS INT64CONST(1000000)
+#define NS_PER_US INT64CONST(1000)
+
+/*
+ * In UUID version 7, we use 12 bits in "rand_a" to store 1/4096
+ * fractions of sub-millisecond. On systems that have only 10 bits of sub-ms
+ * precision we still use 1/4096 parts of a millisecond, but fill lower 2 bits
+ * with random numbers. SUBMS_MINIMAL_STEP is the minimum amount of
+ * nanoseconds that guarantees step of UUID increased clock precision.
+ */
+#if defined(__darwin__) || defined(_MSC_VER)
+#define SUBMS_MINIMAL_STEP_BITS 10
+#else
+#define SUBMS_MINIMAL_STEP_BITS 12
+#endif
+#define SUBMS_BITS 12
+#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_MINIMAL_STEP_BITS)) + 1)
+
/* sortsupport for uuid */
typedef struct
{
@@ -37,6 +59,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline int64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +425,23 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/* Set the given UUID version and the variant bits */
+static inline void
+uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -412,21 +453,182 @@ gen_random_uuid(PG_FUNCTION_ARGS)
errmsg("could not generate random values")));
/*
- * Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * Set magic numbers for a "version 4" (pseudorandom) UUID and variant,
+ * see https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ uuid_set_version(uuid, 4);
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Get the current timestamp with nanosecond precision for UUID generation.
+ * The returned timestamp is ensured to be at least SUBMS_MINIMAL_STEP greater
+ * than the previous returned timestamp (on this backend).
+ */
+static inline int64
+get_real_time_ns_ascending()
+{
+ static int64 previous_ns = 0;
+ int64 ns;
+
+ /* Get the current real timestamp */
+
+#ifdef _MSC_VER
+ struct timeval tmp;
+
+ gettimeofday(&tmp, NULL);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_usec * NS_PER_US;
+#else
+ struct timespec tmp;
+
+ /*
+ * We don't use gettimeofday() where available, instead use
+ * clock_gettime() with CLOCK_REALTIME in order to get a high-precision
+ * (nanoseconds) real timestamp.
+ *
+ * Note that a timestamp returned by clock_gettime() with CLOCK_REALTIME
+ * is nanosecond-precision on most Unix-like platforms. On some platforms
+ * such as macOS, it's restricted to microsecond-precision.
+ */
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+#endif
+
+ /* Guarantee the minimal step advancement of the timestamp */
+ if (previous_ns + SUBMS_MINIMAL_STEP_NS >= ns)
+ ns = previous_ns + SUBMS_MINIMAL_STEP_NS;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562, with the given timestamp.
+ *
+ * UUID version 7 consists of a Unix timestamp in milliseconds (48 bits) and
+ * 74 random bits, excluding the required version and variant bits. To ensure
+ * monotonicity in scenarios of high-frequency UUID generation, we employ the
+ * method "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * This method utilizes 12 bits from the "rand_a" bits to store a 1/4096
+ * (or 2^12) fraction of sub-millisecond precision.
+ */
+static pg_attribute_always_inline pg_uuid_t *
+generate_uuid_v7(int64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ int64 unix_ts_ms;
+ int32 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /*
+ * sub-millisecond timestamp fraction (SUBMS_BITS bits, not
+ * SUBMS_MINIMAL_STEP_BITS)
+ */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+
+ /* Fill the increased clock precision to "rand_a" bits */
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+#if defined(__darwin__) || defined(_MSC_VER)
+
+ /*
+ * On MacOS real time is truncated to microseconds. Thus, 2 least
+ * significant are dependent on other time-specific bits, thus they do not
+ * contribute to uniqueness. To make these bit random we mix in two bits
+ * from CSPRNG.
+ *
+ * SUBMS_MINIMAL_STEP is chosen so that we still guarantee monotonicity
+ * despite altering these bits.
+ */
+ uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
+#endif
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID and variant,
+ * see https://www.rfc-editor.org/rfc/rfc9562#name-version-field
+ */
+ uuid_set_version(uuid, 7);
+
+ return uuid;
+}
+
+/*
+ * Generate UUID version 7 with the current timestamp.
+ */
+Datum
+uuid_v7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = generate_uuid_v7(get_real_time_ns_ascending());
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Similar to uuid_v7() but with the timestamp adjusted by the given interval.
+ */
+Datum
+uuid_v7_interval(PG_FUNCTION_ARGS)
+{
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ TimestampTz ts;
+ pg_uuid_t *uuid;
+ int64 ns = get_real_time_ns_ascending();
+
+ /*
+ * Shift the current timestamp by the given interval. To make correct
+ * calculating the time shift, we convert the UNIX epoch to TimestampTz
+ * and use timestamptz_pl_interval(). Since this calculation is done with
+ * microsecond precision, we carry back the nanoseconds.
+ */
+
+ ts = (TimestampTz) (ns / NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Compute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+
+ /*
+ * Convert a TimestampTz value back to an UNIX epoch and carry back
+ * nanoseconds.
+ */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * NS_PER_US + ns % NS_PER_US;
+
+ /* Generate an UUID */
+ uuid = generate_uuid_v7(ns);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +638,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +657,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +684,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +692,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd382..9210edfec81 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuid_v4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuid_v7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuid_v7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuid_v7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuid_v7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d2..2341907a0b0 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuid_v4()
+uuid_v7()
+uuid_v7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8f4ef0d7a6a..b24e85d4475 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
INSERT INTO guid1(guid_field) VALUES('11111111-1111-1111-1111-111111111111F');
@@ -199,6 +204,35 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuid_v4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuid_v4());
+INSERT INTO guid1 (guid_field) VALUES (uuid_v4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuid_v7());
+INSERT INTO guid1 (guid_field) VALUES (uuid_v7());
+INSERT INTO guid1 (guid_field) VALUES (uuid_v7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuid_v7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
+ array_agg
+------------------------
+ {1,2,3,4,5,6,7,8,9,10}
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -219,8 +253,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuid_v4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuid_v7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
@@ -239,4 +291,4 @@ SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 75ee966ded0..03a5a08be4d 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
@@ -97,6 +102,22 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuid_v4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuid_v4());
+INSERT INTO guid1 (guid_field) VALUES (uuid_v4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuid_v7());
+INSERT INTO guid1 (guid_field) VALUES (uuid_v7());
+INSERT INTO guid1 (guid_field) VALUES (uuid_v7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuid_v7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
-- extract functions
@@ -104,12 +125,15 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuid_v4()); --4
+SELECT uuid_extract_version(uuid_v7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
--
2.43.5
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-26 13:14 Andrey M. Borodin <[email protected]>
parent: Masahiko Sawada <[email protected]>
2 siblings, 0 replies; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-26 13:14 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 26 Nov 2024, at 01:11, Masahiko Sawada <[email protected]> wrote:
>
> I've merged patches and renamed functions (also updated the commit
> message). Please find the attachment.
This comment
* On MacOS real time is truncated to microseconds.
should also note that on Windows we use ported version of gettimeofday(). Interface of this functions limits us with only 10 bits just like MacOS.
Besides this patch looks good to me. Thanks!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-26 15:14 Japin Li <[email protected]>
parent: Masahiko Sawada <[email protected]>
2 siblings, 0 replies; 61+ messages in thread
From: Japin Li @ 2024-11-26 15:14 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Mon, 25 Nov 2024 at 12:11, Masahiko Sawada <[email protected]> wrote:
> On Mon, Nov 25, 2024 at 10:15 AM Andrey M. Borodin <[email protected]> wrote:
>>
>>
>>
>> > On 25 Nov 2024, at 22:53, Masahiko Sawada <[email protected]> wrote:
>> >
>> > In the following code, we use "defined(__darwin__) || defined(_MSC_VER)":
>> >
>> > +#if defined(__darwin__) || defined(_MSC_VER)
>> > +#define SUBMS_MINIMAL_STEP_BITS 10
>> > +#else
>> > +#define SUBMS_MINIMAL_STEP_BITS 12
>> > +#endif
>> > #define SUBMS_BITS 12
>> > -#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_BITS)) + 1)
>> > +#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 <<
>> > SUBMS_MINIMAL_STEP_BITS)) + 1)
>> >
>> > on the other hand, we use "defined(__darwin__) || defined(WIN32)" here:
>> >
>> > +#if defined(__darwin__) || defined(WIN32)
>> > + /*
>> > + * On MacOS real time is truncted to microseconds. Thus, 2 least
>> > + * significant are dependent on other time-specific bits, thus
>> > they do not
>> > + * contribute to uniqueness. To make these bit random we mix in two bits
>> > + * from CSPRNG.
>> > + *
>> > + * SUBMS_MINIMAL_STEP is chosen so that we still guarantee monotonicity
>> > + * despite altering these bits.
>> > + */
>> > + uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
>> > +#endif
>> >
>> > Is there a reason for using different macros?
>>
>> No, that's an oversight. We should mix these 2 bits if an only if SUBMS_MINIMAL_STEP_BITS=10.
>>
>> <tldr>
>> In your review change_v33.patch you used WIN32, but it did not actually compile on Windows.
>> So on Saturday I squashed v33+change_v33.patch, and composed a
>> message that I think we still should switch to _MSC_VER. And just
>> before sending I received your message with v36 where you used
>> _MSC_VER :)
>>
>> I think this way:
>> _MSC_VER - native Windows without clock_gettime, we used gettimeofday() and 10 bits of sub-ms.
>> MinGW - we use clock_gettime() and 12 bits.
>> Darwin - we use clock_gettime() and 10 bits.
>> Anything else - clock_gettime() and 12 bits.
>> </tldr>
>
> Thank you for the summary.
>
> On MinGW, IIUC we can get 100-ns precision timestamps[1], so using 12
> bits for calculating the minimal step would make sense.
>
> Also, if we implement the Windows port of clock_gettime() in the
> future, we can remove the part of using gettimeofday() in
> get_real_time_ns_ascending(). It seems to me that it's
> over-engineering to implement that part only for the UUID v7. So the
> current implementation of get_real_time_ns_ascending() makes sense to
> me.
>
>>
>> >
>> > In get_real_time_ns_ascending(), we use _MSC_VER so we use
>> > clock_gettime() on MinGW.
>> >
>> >>
>> >> Sergey Prokhorenko just draw my attention to the new release of MariaDB [0]. They are doing very similar UUID v7 generation as we do [1].
>> >>
>> >
>> > Thank you for the references. It made me think that we can use the
>> > function name uuid_v7() rather than uuidv7().
>>
>> I think it's a good idea if we will be kind of SQL-compatible.
>>
>
> Okay, let"s rename it.
>
> I've merged patches and renamed functions (also updated the commit
> message). Please find the attachment.
>
It seems a typo about uuid_v{4,7}.
+<function>uuidv_4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv_7</function> () <returnvalue>uuid</returnvalue>
--
Regrads,
Japin Li
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-26 19:11 Sergey Prokhorenko <[email protected]>
parent: Masahiko Sawada <[email protected]>
2 siblings, 1 reply; 61+ messages in thread
From: Sergey Prokhorenko @ 2024-11-26 19:11 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; Masahiko Sawada <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
Changing the name uuidv7() to uuid_v7() is a bad idea because the RFC 9562 uses the term UUIDv7, and therefore code containing uuid_v7() will not be found by searching the web in most cases.
It makes much more sense to rename it to get_uuidv7(), so that a query for "uuidv7" does not return a bunch of other unnecessary functions related to UUIDv7.
Sergey Prokhorenko [email protected]
On Monday 25 November 2024 at 11:12:35 pm GMT+3, Masahiko Sawada <[email protected]> wrote:
On Mon, Nov 25, 2024 at 10:15 AM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 25 Nov 2024, at 22:53, Masahiko Sawada <[email protected]> wrote:
> >
> > In the following code, we use "defined(__darwin__) || defined(_MSC_VER)":
> >
> > +#if defined(__darwin__) || defined(_MSC_VER)
> > +#define SUBMS_MINIMAL_STEP_BITS 10
> > +#else
> > +#define SUBMS_MINIMAL_STEP_BITS 12
> > +#endif
> > #define SUBMS_BITS 12
> > -#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_BITS)) + 1)
> > +#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 <<
> > SUBMS_MINIMAL_STEP_BITS)) + 1)
> >
> > on the other hand, we use "defined(__darwin__) || defined(WIN32)" here:
> >
> > +#if defined(__darwin__) || defined(WIN32)
> > + /*
> > + * On MacOS real time is truncted to microseconds. Thus, 2 least
> > + * significant are dependent on other time-specific bits, thus
> > they do not
> > + * contribute to uniqueness. To make these bit random we mix in two bits
> > + * from CSPRNG.
> > + *
> > + * SUBMS_MINIMAL_STEP is chosen so that we still guarantee monotonicity
> > + * despite altering these bits.
> > + */
> > + uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
> > +#endif
> >
> > Is there a reason for using different macros?
>
> No, that's an oversight. We should mix these 2 bits if an only if SUBMS_MINIMAL_STEP_BITS=10.
>
> <tldr>
> In your review change_v33.patch you used WIN32, but it did not actually compile on Windows.
> So on Saturday I squashed v33+change_v33.patch, and composed a message that I think we still should switch to _MSC_VER. And just before sending I received your message with v36 where you used _MSC_VER :)
>
> I think this way:
> _MSC_VER - native Windows without clock_gettime, we used gettimeofday() and 10 bits of sub-ms.
> MinGW - we use clock_gettime() and 12 bits.
> Darwin - we use clock_gettime() and 10 bits.
> Anything else - clock_gettime() and 12 bits.
> </tldr>
Thank you for the summary.
On MinGW, IIUC we can get 100-ns precision timestamps[1], so using 12
bits for calculating the minimal step would make sense.
Also, if we implement the Windows port of clock_gettime() in the
future, we can remove the part of using gettimeofday() in
get_real_time_ns_ascending(). It seems to me that it's
over-engineering to implement that part only for the UUID v7. So the
current implementation of get_real_time_ns_ascending() makes sense to
me.
>
> >
> > In get_real_time_ns_ascending(), we use _MSC_VER so we use
> > clock_gettime() on MinGW.
> >
> >>
> >> Sergey Prokhorenko just draw my attention to the new release of MariaDB [0]. They are doing very similar UUID v7 generation as we do [1].
> >>
> >
> > Thank you for the references. It made me think that we can use the
> > function name uuid_v7() rather than uuidv7().
>
> I think it's a good idea if we will be kind of SQL-compatible.
>
Okay, let"s rename it.
I've merged patches and renamed functions (also updated the commit
message). Please find the attachment.
Regards,
[1] https://github.com/Alexpux/mingw-w64/blob/d0d7f784833bbb0b2d279310ddc6afb52fe47a46/mingw-w64-librari...
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-26 19:30 Masahiko Sawada <[email protected]>
parent: Sergey Prokhorenko <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-26 19:30 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; Przemysław Sztoch <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Tue, Nov 26, 2024 at 11:11 AM Sergey Prokhorenko
<[email protected]> wrote:
>
> Changing the name uuidv7() to uuid_v7() is a bad idea because the RFC 9562 uses the term UUIDv7, and therefore code containing uuid_v7() will not be found by searching the web in most cases.
>
> It makes much more sense to rename it to get_uuidv7(), so that a query for "uuidv7" does not return a bunch of other unnecessary functions related to UUIDv7.
Thank you for pointing it out. How about gen_uuidv7() and gen_uuidv4()
as we already have gen_random_uuid()?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-26 19:35 Przemysław Sztoch <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Przemysław Sztoch @ 2024-11-26 19:35 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Sergey Prokhorenko <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
A lot of people use https://www.postgresql.org/docs/current/uuid-ossp.html.
|And `uuid_generate_v7()` will be the continuation...|
|From my point of view, absorbing uuid_generate_v5 into mainline would
be a great move too.
|
On 26.11.2024 20:30, Masahiko Sawada wrote:
> On Tue, Nov 26, 2024 at 11:11 AM Sergey Prokhorenko
> <[email protected]> wrote:
>> Changing the name uuidv7() to uuid_v7() is a bad idea because the RFC 9562 uses the term UUIDv7, and therefore code containing uuid_v7() will not be found by searching the web in most cases.
>>
>> It makes much more sense to rename it to get_uuidv7(), so that a query for "uuidv7" does not return a bunch of other unnecessary functions related to UUIDv7.
> Thank you for pointing it out. How about gen_uuidv7() and gen_uuidv4()
> as we already have gen_random_uuid()?
>
> Regards,
>
--
Przemysław Sztoch | Mobile +48 509 99 00 66
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-26 20:47 Sergey Prokhorenko <[email protected]>
parent: Przemysław Sztoch <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Sergey Prokhorenko @ 2024-11-26 20:47 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Przemysław Sztoch <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Jelte Fennema-Nio <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
gen_uuidv7() is OK
uuid-ossp is outdated, slow and not supported by the author. UUIDv7 is the renaissance of UUIDs. So we should not depend on legacy technology names
Sergey Prokhorenko [email protected]
On Tuesday 26 November 2024 at 10:35:20 pm GMT+3, Przemysław Sztoch <[email protected]> wrote:
A lot of people use https://www.postgresql.org/docs/current/uuid-ossp.html.
And `uuid_generate_v7()` will be the continuation...
From my point of view, absorbing uuid_generate_v5 into mainline would be a great move too.
On 26.11.2024 20:30, Masahiko Sawada wrote:
On Tue, Nov 26, 2024 at 11:11 AM Sergey Prokhorenko
<[email protected]> wrote:
Changing the name uuidv7() to uuid_v7() is a bad idea because the RFC 9562 uses the term UUIDv7, and therefore code containing uuid_v7() will not be found by searching the web in most cases.
It makes much more sense to rename it to get_uuidv7(), so that a query for "uuidv7" does not return a bunch of other unnecessary functions related to UUIDv7.
Thank you for pointing it out. How about gen_uuidv7() and gen_uuidv4()
as we already have gen_random_uuid()?
Regards,
--
Przemysław Sztoch | Mobile +48 509 99 00 66
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-26 21:55 Jelte Fennema-Nio <[email protected]>
parent: Sergey Prokhorenko <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Jelte Fennema-Nio @ 2024-11-26 21:55 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Przemysław Sztoch <[email protected]>; Andrey M. Borodin <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Tue, 26 Nov 2024 at 21:48, Sergey Prokhorenko
<[email protected]> wrote:
> gen_uuidv7() is OK
I'd very much prefer to not have a gen_ or get_ prefix as argued before[1][2].
My vote is still for simply uuidv7() and uuidv4()
> uuid-ossp is outdated, slow and not supported by the author. UUIDv7 is the renaissance of UUIDs. So we should not depend on legacy technology names
agreed
[1]: https://www.postgresql.org/message-id/CAGECzQQ%3D38bVUR%3DLZ6vmBCEjaDfOOoQa%2BygFJ1mCG_H2jsC90Q%40ma...
[2]: https://www.postgresql.org/message-id/flat/CAGECzQS%3DEjfLxdX89N95tHFGXS4m1aj2V_%2BxrJppBohgaKQhtQ%4...
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-26 23:11 Masahiko Sawada <[email protected]>
parent: Jelte Fennema-Nio <[email protected]>
0 siblings, 2 replies; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-26 23:11 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Andrey M. Borodin <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Tue, Nov 26, 2024 at 1:55 PM Jelte Fennema-Nio <[email protected]> wrote:
>
> On Tue, 26 Nov 2024 at 21:48, Sergey Prokhorenko
> <[email protected]> wrote:
> > gen_uuidv7() is OK
>
> I'd very much prefer to not have a gen_ or get_ prefix as argued before[1][2].
>
> My vote is still for simply uuidv7() and uuidv4()
>
> > uuid-ossp is outdated, slow and not supported by the author. UUIDv7 is the renaissance of UUIDs. So we should not depend on legacy technology names
>
> agreed
>
It seems that we agreed to use 'uuidv7' instead of 'uuid_v7()'. There
is discussion whether we should add 'gen_' or 'get_' but let's go back
to the previously-agreed function name 'uuidv7()' for now. We can
rename it later if we find a better name.
I've attached the new version patch that incorporated all comments and
renamed the functions. Also I avoided using 'if defined(__darwin__) ||
defined(_MSC_VER)' twice.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
[application/octet-stream] v39-0001-Add-UUID-version-7-generation-function.patch (21.2K, ../../CAD21AoAjRK0-Th2ipCB+xGpMj6OvkMUh_mZ4DST5HKa-AVZVBg@mail.gmail.com/2-v39-0001-Add-UUID-version-7-generation-function.patch)
download | inline diff:
From 7d5b7c2740b242679bc6d7cc90c43a1cde440f64 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v39] Add UUID version 7 generation function.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit implements the uuidv7() SQL function for generating UUID
version 7, as defined in RFC 9652. UUID v7 comprises a Unix timestamp
in milliseconds and random bits, providing uniqueness and
sortability.
In our implementation, the 12-bit sub-millisecond timestamp fraction
is stored immediately after the timestamp, referred to as "rand_a" in
the RFC. This ensures additional monotonicity within a millisecond.
Additionally, an alias uuidv4() is added for the existing
gen_random_uuid() SQL function to maintain consistency.
Bump catalog version.
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 +-
src/backend/utils/adt/uuid.c | 243 ++++++++++++++++++++++-
src/include/catalog/pg_proc.dat | 11 +-
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 56 +++++-
src/test/regress/sql/uuid.sql | 28 ++-
7 files changed, 344 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1c..3e6751d64cc 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 13ccbe7d78c..848ae564540 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14213,6 +14213,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14222,12 +14230,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with millisecond
+ precision + sub-millisecond timestamp + random).
</para>
<para>
@@ -14251,7 +14264,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc4..bb890e9f60d 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,6 +13,8 @@
#include "postgres.h"
+#include <time.h> /* for clock_gettime() */
+
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
@@ -23,6 +25,34 @@
#include "utils/timestamp.h"
#include "utils/uuid.h"
+/* helper macros */
+#define NS_PER_S INT64CONST(1000000000)
+#define NS_PER_MS INT64CONST(1000000)
+#define NS_PER_US INT64CONST(1000)
+
+/*
+ * UUID version 7 uses 12 bits in "rand_a" to store 1/4096 (or 2^12) fractions of
+ * sub-millisecond. While most Unix-like platforms provide nanosecond-precision
+ * timestamps, some systems only offer microsecond precision, limiting us to 10
+ * bits of sub-millisecond information. For example, on macOS, real time is
+ * truncated to microseconds. Additionally, MSVC uses the ported version of
+ * gettimeofday() that returns microsecond precision.
+ *
+ * On systems with only 10 bits of sub-millisecond precision, we still use
+ * 1/4096 parts of a millisecond, but fill lower 2 bits with random numbers
+ * (see generate_uuidv7() for details).
+ *
+ * SUBMS_MINIMAL_STEP defines the minimum number of nanoseconds that guarantees
+ * an increase in the UUID's clock precision.
+ */
+#if defined(__darwin__) || defined(_MSC_VER)
+#define SUBMS_MINIMAL_STEP_BITS 10
+#else
+#define SUBMS_MINIMAL_STEP_BITS 12
+#endif
+#define SUBMS_BITS 12
+#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_MINIMAL_STEP_BITS)) + 1)
+
/* sortsupport for uuid */
typedef struct
{
@@ -37,6 +67,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline int64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +433,23 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/* Set the given UUID version and the variant bits */
+static inline void
+uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -412,21 +461,180 @@ gen_random_uuid(PG_FUNCTION_ARGS)
errmsg("could not generate random values")));
/*
- * Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * Set magic numbers for a "version 4" (pseudorandom) UUID and variant,
+ * see https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
+ */
+ uuid_set_version(uuid, 4);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Get the current timestamp with nanosecond precision for UUID generation.
+ * The returned timestamp is ensured to be at least SUBMS_MINIMAL_STEP greater
+ * than the previous returned timestamp (on this backend).
+ */
+static inline int64
+get_real_time_ns_ascending()
+{
+ static int64 previous_ns = 0;
+ int64 ns;
+
+ /* Get the current real timestamp */
+
+#ifdef _MSC_VER
+ struct timeval tmp;
+
+ gettimeofday(&tmp, NULL);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_usec * NS_PER_US;
+#else
+ struct timespec tmp;
+
+ /*
+ * We don't use gettimeofday() where available, instead use
+ * clock_gettime() with CLOCK_REALTIME in order to get a high-precision
+ * (nanoseconds) real timestamp.
+ *
+ * Note that a timestamp returned by clock_gettime() with CLOCK_REALTIME
+ * is nanosecond-precision on most Unix-like platforms. On some platforms
+ * such as macOS, it's restricted to microsecond-precision.
+ */
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+#endif
+
+ /* Guarantee the minimal step advancement of the timestamp */
+ if (previous_ns + SUBMS_MINIMAL_STEP_NS >= ns)
+ ns = previous_ns + SUBMS_MINIMAL_STEP_NS;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562, with the given timestamp.
+ *
+ * UUID version 7 consists of a Unix timestamp in milliseconds (48 bits) and
+ * 74 random bits, excluding the required version and variant bits. To ensure
+ * monotonicity in scenarios of high-frequency UUID generation, we employ the
+ * method "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * This method utilizes 12 bits from the "rand_a" bits to store a 1/4096
+ * (or 2^12) fraction of sub-millisecond precision.
+ */
+static pg_attribute_always_inline pg_uuid_t *
+generate_uuidv7(int64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ int64 unix_ts_ms;
+ int32 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /*
+ * sub-millisecond timestamp fraction (SUBMS_BITS bits, not
+ * SUBMS_MINIMAL_STEP_BITS)
+ */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+
+ /* Fill the increased clock precision to "rand_a" bits */
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+#if SUBMS_MINIMAL_STEP_BITS == 10
+
+ /*
+ * On systems that have only 10 bits of sub-ms precision, 2 least
+ * significant are dependent on other time-specific bits, and they do not
+ * contribute to uniqueness. To make these bit random we mix in two bits
+ * from CSPRNG. SUBMS_MINIMAL_STEP is chosen so that we still guarantee
+ * monotonicity despite altering these bits.
+ */
+ uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
+#endif
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID and variant,
+ * see https://www.rfc-editor.org/rfc/rfc9562#name-version-field
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ uuid_set_version(uuid, 7);
+
+ return uuid;
+}
+
+/*
+ * Generate UUID version 7 with the current timestamp.
+ */
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Similar to uuidv7() but with the timestamp adjusted by the given interval.
+ */
+Datum
+uuidv7_interval(PG_FUNCTION_ARGS)
+{
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ TimestampTz ts;
+ pg_uuid_t *uuid;
+ int64 ns = get_real_time_ns_ascending();
+
+ /*
+ * Shift the current timestamp by the given interval. To make correct
+ * calculating the time shift, we convert the UNIX epoch to TimestampTz
+ * and use timestamptz_pl_interval(). Since this calculation is done with
+ * microsecond precision, we carry back the nanoseconds.
+ */
+
+ ts = (TimestampTz) (ns / NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Compute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+
+ /*
+ * Convert a TimestampTz value back to an UNIX epoch and carry back
+ * nanoseconds.
+ */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * NS_PER_US + ns % NS_PER_US;
+
+ /* Generate an UUID */
+ uuid = generate_uuidv7(ns);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +644,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +663,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +690,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +698,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd382..3353e9d6e36 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d2..43e7180a161 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuidv4()
+uuidv7()
+uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8f4ef0d7a6a..0059a8c7168 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
INSERT INTO guid1(guid_field) VALUES('11111111-1111-1111-1111-111111111111F');
@@ -199,6 +204,35 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
+ array_agg
+------------------------
+ {1,2,3,4,5,6,7,8,9,10}
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -219,8 +253,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuidv4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuidv7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
@@ -239,4 +291,4 @@ SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 75ee966ded0..6eb8efbd3d3 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
@@ -97,6 +102,22 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
-- extract functions
@@ -104,12 +125,15 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
--
2.43.5
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-27 03:11 Andrey Borodin <[email protected]>
parent: Masahiko Sawada <[email protected]>
1 sibling, 1 reply; 61+ messages in thread
From: Andrey Borodin @ 2024-11-27 03:11 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Peter Eisentraut <[email protected]>; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 27 Nov 2024, at 04:11, Masahiko Sawada <[email protected]> wrote:
>
> On Tue, Nov 26, 2024 at 1:55 PM Jelte Fennema-Nio <[email protected]> wrote:
>>
>> On Tue, 26 Nov 2024 at 21:48, Sergey Prokhorenko
>> <[email protected]> wrote:
>>> gen_uuidv7() is OK
>>
>> I'd very much prefer to not have a gen_ or get_ prefix as argued before[1][2].
>>
>> My vote is still for simply uuidv7() and uuidv4()
>>
>>> uuid-ossp is outdated, slow and not supported by the author. UUIDv7 is the renaissance of UUIDs. So we should not depend on legacy technology names
>>
>> agreed
>>
>
> It seems that we agreed to use 'uuidv7' instead of 'uuid_v7()'. There
> is discussion whether we should add 'gen_' or 'get_' but let's go back
> to the previously-agreed function name 'uuidv7()' for now. We can
> rename it later if we find a better name.
I think uuidv7() is kind of consensual.
> I've attached the new version patch that incorporated all comments and
> renamed the functions. Also I avoided using 'if defined(__darwin__) ||
> defined(_MSC_VER)' twice.
Good, I think now it's a bit easier to understand those 2 bits.
Thanks!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-27 18:24 Masahiko Sawada <[email protected]>
parent: Andrey Borodin <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-27 18:24 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; Peter Eisentraut <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Tue, Nov 26, 2024 at 7:11 PM Andrey Borodin <[email protected]> wrote:
>
>
>
> > On 27 Nov 2024, at 04:11, Masahiko Sawada <[email protected]> wrote:
> >
> > On Tue, Nov 26, 2024 at 1:55 PM Jelte Fennema-Nio <[email protected]> wrote:
> >>
> >> On Tue, 26 Nov 2024 at 21:48, Sergey Prokhorenko
> >> <[email protected]> wrote:
> >>> gen_uuidv7() is OK
> >>
> >> I'd very much prefer to not have a gen_ or get_ prefix as argued before[1][2].
> >>
> >> My vote is still for simply uuidv7() and uuidv4()
> >>
> >>> uuid-ossp is outdated, slow and not supported by the author. UUIDv7 is the renaissance of UUIDs. So we should not depend on legacy technology names
> >>
> >> agreed
> >>
> >
> > It seems that we agreed to use 'uuidv7' instead of 'uuid_v7()'. There
> > is discussion whether we should add 'gen_' or 'get_' but let's go back
> > to the previously-agreed function name 'uuidv7()' for now. We can
> > rename it later if we find a better name.
>
> I think uuidv7() is kind of consensual.
>
> > I've attached the new version patch that incorporated all comments and
> > renamed the functions. Also I avoided using 'if defined(__darwin__) ||
> > defined(_MSC_VER)' twice.
>
> Good, I think now it's a bit easier to understand those 2 bits.
>
Thanks.
I'm going to push the v39 patch (after self review again), barring any
objections and further comments.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-27 23:07 Sergey Prokhorenko <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 2 replies; 61+ messages in thread
From: Sergey Prokhorenko @ 2024-11-27 23:07 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Masahiko Sawada <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
It would be useful to add a standard comparative benchmark with several parameters and use cases to the patch, so that IT departments can compare UUIDv7, ULID, UUIDv4, Snowflake ID and BIGSERIAL for their hardware and conditions.
I know for a fact that IT departments make such benchmarks of low quality. They usually measure the generation rate, which is meaningless because it is usually excessive. It makes sense to measure the rate of single-threaded and multi-threaded insertion of a large number of records (with and without partitioning), as well as the rate of execution of queries to join big tables, to update or delete a large number of records. It is important to measure memory usage, processor load, etc.
Sergey Prokhorenko [email protected]
On Wednesday 27 November 2024 at 09:24:40 pm GMT+3, Masahiko Sawada <[email protected]> wrote:
On Tue, Nov 26, 2024 at 7:11 PM Andrey Borodin <[email protected]> wrote:
>
>
>
> > On 27 Nov 2024, at 04:11, Masahiko Sawada <[email protected]> wrote:
> >
> > On Tue, Nov 26, 2024 at 1:55 PM Jelte Fennema-Nio <[email protected]> wrote:
> >>
> >> On Tue, 26 Nov 2024 at 21:48, Sergey Prokhorenko
> >> <[email protected]> wrote:
> >>> gen_uuidv7() is OK
> >>
> >> I'd very much prefer to not have a gen_ or get_ prefix as argued before[1][2].
> >>
> >> My vote is still for simply uuidv7() and uuidv4()
> >>
> >>> uuid-ossp is outdated, slow and not supported by the author. UUIDv7 is the renaissance of UUIDs. So we should not depend on legacy technology names
> >>
> >> agreed
> >>
> >
> > It seems that we agreed to use 'uuidv7' instead of 'uuid_v7()'. There
> > is discussion whether we should add 'gen_' or 'get_' but let's go back
> > to the previously-agreed function name 'uuidv7()' for now. We can
> > rename it later if we find a better name.
>
> I think uuidv7() is kind of consensual.
>
> > I've attached the new version patch that incorporated all comments and
> > renamed the functions. Also I avoided using 'if defined(__darwin__) ||
> > defined(_MSC_VER)' twice.
>
> Good, I think now it's a bit easier to understand those 2 bits.
>
Thanks.
I'm going to push the v39 patch (after self review again), barring any
objections and further comments.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Отв.: Re: UUID v7
@ 2024-11-27 23:28 Sergey Prokhorenko <[email protected]>
parent: Sergey Prokhorenko <[email protected]>
1 sibling, 0 replies; 61+ messages in thread
From: Sergey Prokhorenko @ 2024-11-27 23:28 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Masahiko Sawada <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
I forgot to mention the incremental download
Отправлено из Yahoo Почты на iPhone
Пользователь четверг, ноября 28, 2024, 2:07 AM написал Sergey Prokhorenko <[email protected]>:
It would be useful to add a standard comparative benchmark with several parameters and use cases to the patch, so that IT departments can compare UUIDv7, ULID, UUIDv4, Snowflake ID and BIGSERIAL for their hardware and conditions.
I know for a fact that IT departments make such benchmarks of low quality. They usually measure the generation rate, which is meaningless because it is usually excessive. It makes sense to measure the rate of single-threaded and multi-threaded insertion of a large number of records (with and without partitioning), as well as the rate of execution of queries to join big tables, to update or delete a large number of records. It is important to measure memory usage, processor load, etc.
Sergey Prokhorenko [email protected]
On Wednesday 27 November 2024 at 09:24:40 pm GMT+3, Masahiko Sawada <[email protected]> wrote:
On Tue, Nov 26, 2024 at 7:11 PM Andrey Borodin <[email protected]> wrote:
>
>
>
> > On 27 Nov 2024, at 04:11, Masahiko Sawada <[email protected]> wrote:
> >
> > On Tue, Nov 26, 2024 at 1:55 PM Jelte Fennema-Nio <[email protected]> wrote:
> >>
> >> On Tue, 26 Nov 2024 at 21:48, Sergey Prokhorenko
> >> <[email protected]> wrote:
> >>> gen_uuidv7() is OK
> >>
> >> I'd very much prefer to not have a gen_ or get_ prefix as argued before[1][2].
> >>
> >> My vote is still for simply uuidv7() and uuidv4()
> >>
> >>> uuid-ossp is outdated, slow and not supported by the author. UUIDv7 is the renaissance of UUIDs. So we should not depend on legacy technology names
> >>
> >> agreed
> >>
> >
> > It seems that we agreed to use 'uuidv7' instead of 'uuid_v7()'. There
> > is discussion whether we should add 'gen_' or 'get_' but let's go back
> > to the previously-agreed function name 'uuidv7()' for now. We can
> > rename it later if we find a better name.
>
> I think uuidv7() is kind of consensual.
>
> > I've attached the new version patch that incorporated all comments and
> > renamed the functions. Also I avoided using 'if defined(__darwin__) ||
> > defined(_MSC_VER)' twice.
>
> Good, I think now it's a bit easier to understand those 2 bits.
>
Thanks.
I'm going to push the v39 patch (after self review again), barring any
objections and further comments.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-28 08:09 Andrey M. Borodin <[email protected]>
parent: Sergey Prokhorenko <[email protected]>
1 sibling, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-28 08:09 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Masahiko Sawada <[email protected]>; Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 28 Nov 2024, at 04:07, Sergey Prokhorenko <[email protected]> wrote:
>
> It would be useful to add a standard comparative benchmark with several parameters and use cases to the patch, so that IT departments can compare UUIDv7, ULID, UUIDv4, Snowflake ID and BIGSERIAL for their hardware and conditions.
>
> I know for a fact that IT departments make such benchmarks of low quality. They usually measure the generation rate, which is meaningless because it is usually excessive. It makes sense to measure the rate of single-threaded and multi-threaded insertion of a large number of records (with and without partitioning), as well as the rate of execution of queries to join big tables, to update or delete a large number of records. It is important to measure memory usage, processor load, etc.
Publishing benchmarks seems to be far beyond what our documentation go for. Mostly, because benchmarks are tricky. You can prove anything with benchmarks.
Everyone is welcome to publish benchmark results in their blogs, but IMO docs have a very different job to do.
I’ll just publish one benchmark in this mailing list. With patch v39 applied on my MB Air M2 I get:
postgres=# create table table_for_uuidv4(id uuid primary key);
CREATE TABLE
Time: 9.479 ms
postgres=# insert into table_for_uuidv4 select uuidv4() from generate_series(1,3e7);
INSERT 0 30000000
Time: 2003918.770 ms (33:23.919)
postgres=# create table table_for_uuidv7(id uuid primary key);
CREATE TABLE
Time: 3.930 ms
postgres=# insert into table_for_uuidv7 select uuidv7() from generate_series(1,3e7);
INSERT 0 30000000
Time: 337001.315 ms (05:37.001)
Almost an order of magnitude better :)
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-28 19:46 Peter Eisentraut <[email protected]>
parent: Masahiko Sawada <[email protected]>
1 sibling, 1 reply; 61+ messages in thread
From: Peter Eisentraut @ 2024-11-28 19:46 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; Jelte Fennema-Nio <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Andrey M. Borodin <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On 27.11.24 00:11, Masahiko Sawada wrote:
> On Tue, Nov 26, 2024 at 1:55 PM Jelte Fennema-Nio <[email protected]> wrote:
>>
>> On Tue, 26 Nov 2024 at 21:48, Sergey Prokhorenko
>> <[email protected]> wrote:
>>> gen_uuidv7() is OK
>>
>> I'd very much prefer to not have a gen_ or get_ prefix as argued before[1][2].
>>
>> My vote is still for simply uuidv7() and uuidv4()
>>
>>> uuid-ossp is outdated, slow and not supported by the author. UUIDv7 is the renaissance of UUIDs. So we should not depend on legacy technology names
>>
>> agreed
>>
>
> It seems that we agreed to use 'uuidv7' instead of 'uuid_v7()'. There
> is discussion whether we should add 'gen_' or 'get_' but let's go back
> to the previously-agreed function name 'uuidv7()' for now. We can
> rename it later if we find a better name.
>
> I've attached the new version patch that incorporated all comments and
> renamed the functions. Also I avoided using 'if defined(__darwin__) ||
> defined(_MSC_VER)' twice.
* doc/src/sgml/func.sgml
The function variant uuidv7(interval) is not documented.
* src/backend/utils/adt/uuid.c
+/* Set the given UUID version and the variant bits */
+static inline void
+uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
This should be a block comment, like
/*
* Set the ...
*/
+/*
+ * Generate UUID version 7 per RFC 9562, with the given timestamp.
+ *
...
+static pg_attribute_always_inline pg_uuid_t *
+generate_uuidv7(int64 ns)
Is "ns" the timestamp argument? What format is it? Explain.
+ /*
+ * Shift the current timestamp by the given interval. To make correct
+ * calculating the time shift, we convert the UNIX epoch to TimestampTz
+ * and use timestamptz_pl_interval(). Since this calculation is done
with
+ * microsecond precision, we carry back the nanoseconds.
+ */
This needs a bit of grammar tweaking, I think: "To make correct calculating"
I don't know what the meaning of "carry back" is.
+ Interval *span = PG_GETARG_INTERVAL_P(0);
Not sure why this is named "span"? Maybe "shift" would be better?
* src/include/catalog/pg_proc.dat
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp
shifted on specific interval',
better "shifted by"?
* src/test/regress/expected/opr_sanity.out
+uuidv4()
+uuidv7()
+uuidv7(interval)
Functions without arguments don't need to be marked leakproof.
uuidv7(interval) internally calls timestamptz_pl_interval(), which is
not leakproof, so I don't think that classification is sound.
* src/test/regress/sql/uuid.sql
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
Make the whitespace of the comment consistent with the rest of the file.
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') =
'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis
test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') =
'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test
vector for v1
Here as well.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Отв.: Re: UUID v7
@ 2024-11-29 04:13 Sergey Prokhorenko <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 2 replies; 61+ messages in thread
From: Sergey Prokhorenko @ 2024-11-29 04:13 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Masahiko Sawada <[email protected]>; Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
I mean to add not benchmark results to the patch, but functions so that everyone can compare themselves on their equipment. The comparison with UUIDv4 is not very interesting, as the choice is usually between UUIDv7 and an integer key. And I have described many use cases, and in your benchmark there is only one, the simplest.
Отправлено из Yahoo Почты на iPhone
Пользователь четверг, ноября 28, 2024, 11:09 AM написал Andrey M. Borodin <[email protected]>:
> On 28 Nov 2024, at 04:07, Sergey Prokhorenko <[email protected]> wrote:
>
> It would be useful to add a standard comparative benchmark with several parameters and use cases to the patch, so that IT departments can compare UUIDv7, ULID, UUIDv4, Snowflake ID and BIGSERIAL for their hardware and conditions.
>
> I know for a fact that IT departments make such benchmarks of low quality. They usually measure the generation rate, which is meaningless because it is usually excessive. It makes sense to measure the rate of single-threaded and multi-threaded insertion of a large number of records (with and without partitioning), as well as the rate of execution of queries to join big tables, to update or delete a large number of records. It is important to measure memory usage, processor load, etc.
Publishing benchmarks seems to be far beyond what our documentation go for. Mostly, because benchmarks are tricky. You can prove anything with benchmarks.
Everyone is welcome to publish benchmark results in their blogs, but IMO docs have a very different job to do.
I’ll just publish one benchmark in this mailing list. With patch v39 applied on my MB Air M2 I get:
postgres=# create table table_for_uuidv4(id uuid primary key);
CREATE TABLE
Time: 9.479 ms
postgres=# insert into table_for_uuidv4 select uuidv4() from generate_series(1,3e7);
INSERT 0 30000000
Time: 2003918.770 ms (33:23.919)
postgres=# create table table_for_uuidv7(id uuid primary key);
CREATE TABLE
Time: 3.930 ms
postgres=# insert into table_for_uuidv7 select uuidv7() from generate_series(1,3e7);
INSERT 0 30000000
Time: 337001.315 ms (05:37.001)
Almost an order of magnitude better :)
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: Отв.: Re: UUID v7
@ 2024-11-29 04:22 Kirill Reshke <[email protected]>
parent: Sergey Prokhorenko <[email protected]>
1 sibling, 0 replies; 61+ messages in thread
From: Kirill Reshke @ 2024-11-29 04:22 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Masahiko Sawada <[email protected]>; Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Fri, 29 Nov 2024, 09:14 Sergey Prokhorenko, <
[email protected]> wrote:
> I mean to add not benchmark results to the patch, but functions so that
> everyone can compare themselves on their equipment. The comparison with
> UUIDv4 is not very interesting, as the choice is usually between UUIDv7 and
> an integer key. And I have described many use cases, and in your benchmark
> there is only one, the simplest.
>
>
> Отправлено из Yahoo Почты на iPhone
> <https://mail.onelink.me/107872968?pid=nativeplacement&c=Global_Acquisition_YMktg_315_Internal_Em...;
>
> Пользователь четверг, ноября 28, 2024, 11:09 AM написал Andrey M. Borodin <
> [email protected]>:
>
>
>
> > On 28 Nov 2024, at 04:07, Sergey Prokhorenko <
> [email protected]> wrote:
> >
> > It would be useful to add a standard comparative benchmark with several
> parameters and use cases to the patch, so that IT departments can compare
> UUIDv7, ULID, UUIDv4, Snowflake ID and BIGSERIAL for their hardware and
> conditions.
> >
> > I know for a fact that IT departments make such benchmarks of low
> quality. They usually measure the generation rate, which is meaningless
> because it is usually excessive. It makes sense to measure the rate of
> single-threaded and multi-threaded insertion of a large number of records
> (with and without partitioning), as well as the rate of execution of
> queries to join big tables, to update or delete a large number of records.
> It is important to measure memory usage, processor load, etc.
>
>
> Publishing benchmarks seems to be far beyond what our documentation go
> for. Mostly, because benchmarks are tricky. You can prove anything with
> benchmarks.
>
> Everyone is welcome to publish benchmark results in their blogs, but IMO
> docs have a very different job to do.
>
> I’ll just publish one benchmark in this mailing list. With patch v39
> applied on my MB Air M2 I get:
>
> postgres=# create table table_for_uuidv4(id uuid primary key);
> CREATE TABLE
> Time: 9.479 ms
> postgres=# insert into table_for_uuidv4 select uuidv4() from
> generate_series(1,3e7);
> INSERT 0 30000000
> Time: 2003918.770 ms (33:23.919)
> postgres=# create table table_for_uuidv7(id uuid primary key);
> CREATE TABLE
> Time: 3.930 ms
> postgres=# insert into table_for_uuidv7 select uuidv7() from
> generate_series(1,3e7);
> INSERT 0 30000000
> Time: 337001.315 ms (05:37.001)
>
> Almost an order of magnitude better :)
>
>
> Best regards, Andrey Borodin.
>
> Hi!
Do not top-post on this list
>
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: Отв.: Re: UUID v7
@ 2024-11-29 06:18 Masahiko Sawada <[email protected]>
parent: Sergey Prokhorenko <[email protected]>
1 sibling, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-29 06:18 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Thu, Nov 28, 2024 at 8:13 PM Sergey Prokhorenko
<[email protected]> wrote:
>
> I mean to add not benchmark results to the patch, but functions so that everyone can compare themselves on their equipment. The comparison with UUIDv4 is not very interesting, as the choice is usually between UUIDv7 and an integer key. And I have described many use cases, and in your benchmark there is only one, the simplest.
I don't think we should add such benchmark functions at least to this
patch. If there already is a well-established workload using UUIDv7
and UUIDv4 etc, users can use pgbench with custom scripts, or it might
make sense to add it to pgbench as a built-in workload. Which however
should be a separate patch. Having said that, I think users should use
benchmarks that fit their workloads, and it would not be easy to
establish workloads that are reasonable for most systems.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: Отв.: Re: UUID v7
@ 2024-11-29 13:57 Sergey Prokhorenko <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 2 replies; 61+ messages in thread
From: Sergey Prokhorenko @ 2024-11-29 13:57 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
Sergey Prokhorenko [email protected]
On Friday 29 November 2024 at 09:19:33 am GMT+3, Masahiko Sawada <[email protected]> wrote:
On Thu, Nov 28, 2024 at 8:13 PM Sergey Prokhorenko
<[email protected]> wrote:
>
> I mean to add not benchmark results to the patch, but functions so that everyone can compare themselves on their equipment. The comparison with UUIDv4 is not very interesting, as the choice is usually between UUIDv7 and an integer key. And I have described many use cases, and in your benchmark there is only one, the simplest.
I don't think we should add such benchmark functions at least to this
patch. If there already is a well-established workload using UUIDv7
and UUIDv4 etc, users can use pgbench with custom scripts, or it might
make sense to add it to pgbench as a built-in workload. Which however
should be a separate patch. Having said that, I think users should use
benchmarks that fit their workloads, and it would not be easy to
establish workloads that are reasonable for most systems.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Workloads can and must be added with parameters. Typically, companies use test tables of 10,000 and 1,000,000 records, etc. Different companies have mostly similar usage scenarios (for example, incremental loading). Each company has to duplicate the work of others, creating the same benchmarks. The worst thing is that this is entrusted to incompetent employees who are not very good at understanding typical key usage scenarios. As a rule, these are programmers, not system analysts. Accordingly, the solution in 99% of cases will be in favor of integer keys, as they take up less space and are generated faster. If we leave this problem until the next patch, it will take us a year and a half. This is completely wrong.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: Отв.: Re: UUID v7
@ 2024-11-29 16:40 Andrey M. Borodin <[email protected]>
parent: Sergey Prokhorenko <[email protected]>
1 sibling, 0 replies; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-29 16:40 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 29 Nov 2024, at 18:57, Sergey Prokhorenko <[email protected]> wrote:
>
> Workloads can and must be added with parameters. Typically, companies use test tables of 10,000 and 1,000,000 records, etc. Different companies have mostly similar usage scenarios (for example, incremental loading). Each company has to duplicate the work of others, creating the same benchmarks. The worst thing is that this is entrusted to incompetent employees who are not very good at understanding typical key usage scenarios. As a rule, these are programmers, not system analysts. Accordingly, the solution in 99% of cases will be in favor of integer keys, as they take up less space and are generated faster. If we leave this problem until the next patch, it will take us a year and a half. This is completely wrong.
I think we have pretty decent documentation in the patch. It only points to RFC and that's it.
There were patch versions with opinionated novels in docs. Giving advises, comparing possibilities and all that stuff. I'm so happy we passed through this stage and moved forward :)
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: Отв.: Re: UUID v7
@ 2024-11-29 17:54 Masahiko Sawada <[email protected]>
parent: Sergey Prokhorenko <[email protected]>
1 sibling, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-29 17:54 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Fri, Nov 29, 2024 at 5:59 AM Sergey Prokhorenko
<[email protected]> wrote:
>
>
>
> Sergey Prokhorenko [email protected]
>
>
> On Friday 29 November 2024 at 09:19:33 am GMT+3, Masahiko Sawada <[email protected]> wrote:
>
>
> On Thu, Nov 28, 2024 at 8:13 PM Sergey Prokhorenko
>
> <[email protected]> wrote:
> >
> > I mean to add not benchmark results to the patch, but functions so that everyone can compare themselves on their equipment. The comparison with UUIDv4 is not very interesting, as the choice is usually between UUIDv7 and an integer key. And I have described many use cases, and in your benchmark there is only one, the simplest.
>
>
> I don't think we should add such benchmark functions at least to this
> patch. If there already is a well-established workload using UUIDv7
> and UUIDv4 etc, users can use pgbench with custom scripts, or it might
> make sense to add it to pgbench as a built-in workload. Which however
> should be a separate patch. Having said that, I think users should use
> benchmarks that fit their workloads, and it would not be easy to
> establish workloads that are reasonable for most systems.
>
> Regards,
>
> --
> Masahiko Sawada
> Amazon Web Services: https://aws.amazon.com
>
>
>
>
>
>
> Workloads can and must be added with parameters. Typically, companies use test tables of 10,000 and 1,000,000 records, etc. Different companies have mostly similar usage scenarios (for example, incremental loading). Each company has to duplicate the work of others, creating the same benchmarks. The worst thing is that this is entrusted to incompetent employees who are not very good at understanding typical key usage scenarios. As a rule, these are programmers, not system analysts. Accordingly, the solution in 99% of cases will be in favor of integer keys, as they take up less space and are generated faster. If we leave this problem until the next patch, it will take us a year and a half. This is completely wrong.
There are still 4 months left until the feature freeze. We can discuss
this topic and might find solutions. I don't think it's a blocker of
this patch (UUIDv7 implementation patch).
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-29 18:39 Andrey M. Borodin <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Andrey M. Borodin @ 2024-11-29 18:39 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; +Cc: Masahiko Sawada <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 29 Nov 2024, at 00:46, Peter Eisentraut <[email protected]> wrote:
>
> Here as well.
Peter, many thanks for the next round of review. I agree with all corrections.
I'm sending amendments addressing your review as a separate step in patch set. Step 1 of this patch set is identical to v39.
Thanks!
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v40-0001-Add-UUID-version-7-generation-function.patch (21.2K, ../../[email protected]/2-v40-0001-Add-UUID-version-7-generation-function.patch)
download | inline diff:
From 6c80fe539be9ecc81e72216124a938235f6006f8 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Wed, 20 Mar 2024 22:30:14 +0500
Subject: [PATCH v40 1/2] Add UUID version 7 generation function.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit implements the uuidv7() SQL function for generating UUID
version 7, as defined in RFC 9652. UUID v7 comprises a Unix timestamp
in milliseconds and random bits, providing uniqueness and
sortability.
In our implementation, the 12-bit sub-millisecond timestamp fraction
is stored immediately after the timestamp, referred to as "rand_a" in
the RFC. This ensures additional monotonicity within a millisecond.
Additionally, an alias uuidv4() is added for the existing
gen_random_uuid() SQL function to maintain consistency.
Bump catalog version.
Author: Andrey Borodin
Reviewed-by: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewed-by: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewed-by: Peter Eisentraut, Chris Travers, Lukas Fittl
Reviewed-by: Michael Paquier, Masahiko Sawada, Stepan Neretin
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/datatype.sgml | 2 +-
doc/src/sgml/func.sgml | 21 +-
src/backend/utils/adt/uuid.c | 243 ++++++++++++++++++++++-
src/include/catalog/pg_proc.dat | 11 +-
src/test/regress/expected/opr_sanity.out | 3 +
src/test/regress/expected/uuid.out | 56 +++++-
src/test/regress/sql/uuid.sql | 28 ++-
7 files changed, 344 insertions(+), 20 deletions(-)
diff --git a/doc/src/sgml/datatype.sgml b/doc/src/sgml/datatype.sgml
index e0d33f12e1..3e6751d64c 100644
--- a/doc/src/sgml/datatype.sgml
+++ b/doc/src/sgml/datatype.sgml
@@ -4380,7 +4380,7 @@ SELECT to_tsvector( 'postgraduate' ), to_tsquery( 'postgres:*' );
<para>
The data type <type>uuid</type> stores Universally Unique Identifiers
- (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>,
+ (UUID) as defined by <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>,
ISO/IEC 9834-8:2005, and related standards.
(Some systems refer to this data type as a globally unique identifier, or
GUID,<indexterm><primary>GUID</primary></indexterm> instead.) This
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8b81106fa2..e9a2db2e93 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14255,6 +14255,14 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
<indexterm>
<primary>uuid_extract_timestamp</primary>
</indexterm>
@@ -14264,12 +14272,17 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</indexterm>
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function returns a version 7 UUID (UNIX timestamp with millisecond
+ precision + sub-millisecond timestamp + random).
</para>
<para>
@@ -14293,7 +14306,7 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuid_extract_version</function> (uuid) <returnvalue>smallint</returnvalue>
</synopsis>
This function extracts the version from a UUID of the variant described by
- <ulink url="https://datatracker.ietf.org/doc/html/rfc4122">RFC 4122</ulink>. For
+ <ulink url="https://datatracker.ietf.org/doc/html/rfc9562">RFC 9562</ulink>. For
other variants, this function returns null. For example, for a UUID
generated by <function>gen_random_uuid</function>, this function will
return 4.
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 5284d23dcc..bb890e9f60 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,6 +13,8 @@
#include "postgres.h"
+#include <time.h> /* for clock_gettime() */
+
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
@@ -23,6 +25,34 @@
#include "utils/timestamp.h"
#include "utils/uuid.h"
+/* helper macros */
+#define NS_PER_S INT64CONST(1000000000)
+#define NS_PER_MS INT64CONST(1000000)
+#define NS_PER_US INT64CONST(1000)
+
+/*
+ * UUID version 7 uses 12 bits in "rand_a" to store 1/4096 (or 2^12) fractions of
+ * sub-millisecond. While most Unix-like platforms provide nanosecond-precision
+ * timestamps, some systems only offer microsecond precision, limiting us to 10
+ * bits of sub-millisecond information. For example, on macOS, real time is
+ * truncated to microseconds. Additionally, MSVC uses the ported version of
+ * gettimeofday() that returns microsecond precision.
+ *
+ * On systems with only 10 bits of sub-millisecond precision, we still use
+ * 1/4096 parts of a millisecond, but fill lower 2 bits with random numbers
+ * (see generate_uuidv7() for details).
+ *
+ * SUBMS_MINIMAL_STEP defines the minimum number of nanoseconds that guarantees
+ * an increase in the UUID's clock precision.
+ */
+#if defined(__darwin__) || defined(_MSC_VER)
+#define SUBMS_MINIMAL_STEP_BITS 10
+#else
+#define SUBMS_MINIMAL_STEP_BITS 12
+#endif
+#define SUBMS_BITS 12
+#define SUBMS_MINIMAL_STEP_NS ((NS_PER_MS / (1 << SUBMS_MINIMAL_STEP_BITS)) + 1)
+
/* sortsupport for uuid */
typedef struct
{
@@ -37,6 +67,8 @@ static int uuid_internal_cmp(const pg_uuid_t *arg1, const pg_uuid_t *arg2);
static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
+static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
+static inline int64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -401,6 +433,23 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
+/* Set the given UUID version and the variant bits */
+static inline void
+uuid_set_version(pg_uuid_t *uuid, unsigned char version)
+{
+ /* set version field, top four bits */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | (version << 4);
+
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+}
+
+/*
+ * Generate UUID version 4.
+ *
+ * All UUID bytes are filled with strong random numbers except version and
+ * variant bits.
+ */
Datum
gen_random_uuid(PG_FUNCTION_ARGS)
{
@@ -412,21 +461,180 @@ gen_random_uuid(PG_FUNCTION_ARGS)
errmsg("could not generate random values")));
/*
- * Set magic numbers for a "version 4" (pseudorandom) UUID, see
- * http://tools.ietf.org/html/rfc4122#section-4.4
+ * Set magic numbers for a "version 4" (pseudorandom) UUID and variant,
+ * see https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-version-4
+ */
+ uuid_set_version(uuid, 4);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Get the current timestamp with nanosecond precision for UUID generation.
+ * The returned timestamp is ensured to be at least SUBMS_MINIMAL_STEP greater
+ * than the previous returned timestamp (on this backend).
+ */
+static inline int64
+get_real_time_ns_ascending()
+{
+ static int64 previous_ns = 0;
+ int64 ns;
+
+ /* Get the current real timestamp */
+
+#ifdef _MSC_VER
+ struct timeval tmp;
+
+ gettimeofday(&tmp, NULL);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_usec * NS_PER_US;
+#else
+ struct timespec tmp;
+
+ /*
+ * We don't use gettimeofday() where available, instead use
+ * clock_gettime() with CLOCK_REALTIME in order to get a high-precision
+ * (nanoseconds) real timestamp.
+ *
+ * Note that a timestamp returned by clock_gettime() with CLOCK_REALTIME
+ * is nanosecond-precision on most Unix-like platforms. On some platforms
+ * such as macOS, it's restricted to microsecond-precision.
+ */
+ clock_gettime(CLOCK_REALTIME, &tmp);
+ ns = tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+#endif
+
+ /* Guarantee the minimal step advancement of the timestamp */
+ if (previous_ns + SUBMS_MINIMAL_STEP_NS >= ns)
+ ns = previous_ns + SUBMS_MINIMAL_STEP_NS;
+ previous_ns = ns;
+
+ return ns;
+}
+
+/*
+ * Generate UUID version 7 per RFC 9562, with the given timestamp.
+ *
+ * UUID version 7 consists of a Unix timestamp in milliseconds (48 bits) and
+ * 74 random bits, excluding the required version and variant bits. To ensure
+ * monotonicity in scenarios of high-frequency UUID generation, we employ the
+ * method "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
+ * This method utilizes 12 bits from the "rand_a" bits to store a 1/4096
+ * (or 2^12) fraction of sub-millisecond precision.
+ */
+static pg_attribute_always_inline pg_uuid_t *
+generate_uuidv7(int64 ns)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ int64 unix_ts_ms;
+ int32 increased_clock_precision;
+
+ unix_ts_ms = ns / NS_PER_MS;
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
+ uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
+ uuid->data[2] = (unsigned char) (unix_ts_ms >> 24);
+ uuid->data[3] = (unsigned char) (unix_ts_ms >> 16);
+ uuid->data[4] = (unsigned char) (unix_ts_ms >> 8);
+ uuid->data[5] = (unsigned char) unix_ts_ms;
+
+ /*
+ * sub-millisecond timestamp fraction (SUBMS_BITS bits, not
+ * SUBMS_MINIMAL_STEP_BITS)
+ */
+ increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+
+ /* Fill the increased clock precision to "rand_a" bits */
+ uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
+ uuid->data[7] = (unsigned char) (increased_clock_precision);
+
+ /* fill everything after the increased clock precision with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+#if SUBMS_MINIMAL_STEP_BITS == 10
+
+ /*
+ * On systems that have only 10 bits of sub-ms precision, 2 least
+ * significant are dependent on other time-specific bits, and they do not
+ * contribute to uniqueness. To make these bit random we mix in two bits
+ * from CSPRNG. SUBMS_MINIMAL_STEP is chosen so that we still guarantee
+ * monotonicity despite altering these bits.
+ */
+ uuid->data[7] = uuid->data[7] ^ (uuid->data[8] >> 6);
+#endif
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID and variant,
+ * see https://www.rfc-editor.org/rfc/rfc9562#name-version-field
*/
- uuid->data[6] = (uuid->data[6] & 0x0f) | 0x40; /* time_hi_and_version */
- uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80; /* clock_seq_hi_and_reserved */
+ uuid_set_version(uuid, 7);
+
+ return uuid;
+}
+
+/*
+ * Generate UUID version 7 with the current timestamp.
+ */
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
PG_RETURN_UUID_P(uuid);
}
-#define UUIDV1_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
+/*
+ * Similar to uuidv7() but with the timestamp adjusted by the given interval.
+ */
+Datum
+uuidv7_interval(PG_FUNCTION_ARGS)
+{
+ Interval *span = PG_GETARG_INTERVAL_P(0);
+ TimestampTz ts;
+ pg_uuid_t *uuid;
+ int64 ns = get_real_time_ns_ascending();
+
+ /*
+ * Shift the current timestamp by the given interval. To make correct
+ * calculating the time shift, we convert the UNIX epoch to TimestampTz
+ * and use timestamptz_pl_interval(). Since this calculation is done with
+ * microsecond precision, we carry back the nanoseconds.
+ */
+
+ ts = (TimestampTz) (ns / NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ /* Compute time shift */
+ ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
+ TimestampTzGetDatum(ts),
+ IntervalPGetDatum(span)));
+
+ /*
+ * Convert a TimestampTz value back to an UNIX epoch and carry back
+ * nanoseconds.
+ */
+ ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
+ * NS_PER_US + ns % NS_PER_US;
+
+ /* Generate an UUID */
+ uuid = generate_uuidv7(ns);
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+/*
+ * Start of a Gregorian epoch == date2j(1582,10,15)
+ * We cast it to 64-bit because it's used in overflow-prone computations
+ */
+#define GREGORIAN_EPOCH_JDATE INT64CONST(2299161)
/*
* Extract timestamp from UUID.
*
- * Returns null if not RFC 4122 variant or not a version that has a timestamp.
+ * Returns null if not RFC 9562 variant or not a version that has a timestamp.
*/
Datum
uuid_extract_timestamp(PG_FUNCTION_ARGS)
@@ -436,7 +644,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
uint64 tms;
TimestampTz ts;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
@@ -455,7 +663,22 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/* convert 100-ns intervals to us, then adjust */
ts = (TimestampTz) (tms / 10) -
- ((uint64) POSTGRES_EPOCH_JDATE - UUIDV1_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ ((uint64) POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if (version == 7)
+ {
+ tms = (uuid->data[5])
+ + (((uint64) uuid->data[4]) << 8)
+ + (((uint64) uuid->data[3]) << 16)
+ + (((uint64) uuid->data[2]) << 24)
+ + (((uint64) uuid->data[1]) << 32)
+ + (((uint64) uuid->data[0]) << 40);
+
+ /* convert ms to us, then adjust */
+ ts = (TimestampTz) (tms * NS_PER_US) -
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
PG_RETURN_TIMESTAMPTZ(ts);
}
@@ -467,7 +690,7 @@ uuid_extract_timestamp(PG_FUNCTION_ARGS)
/*
* Extract UUID version.
*
- * Returns null if not RFC 4122 variant.
+ * Returns null if not RFC 9562 variant.
*/
Datum
uuid_extract_version(PG_FUNCTION_ARGS)
@@ -475,7 +698,7 @@ uuid_extract_version(PG_FUNCTION_ARGS)
pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
uint16 version;
- /* check if RFC 4122 variant */
+ /* check if RFC 9562 variant */
if ((uuid->data[8] & 0xc0) != 0x80)
PG_RETURN_NULL();
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cbbe8acd38..3353e9d6e3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9342,11 +9342,20 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate UUID version 4',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
prorettype => 'timestamptz', proargtypes => 'uuid',
prosrc => 'uuid_extract_timestamp' },
-{ oid => '6343', descr => 'extract version from RFC 4122 UUID',
+{ oid => '6343', descr => 'extract version from RFC 9562 UUID',
proname => 'uuid_extract_version', proleakproof => 't', prorettype => 'int2',
proargtypes => 'uuid', prosrc => 'uuid_extract_version' },
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 34a32bd11d..43e7180a16 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,6 +878,9 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
+uuidv4()
+uuidv7()
+uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8f4ef0d7a6..0059a8c716 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
INSERT INTO guid1(guid_field) VALUES('11111111-1111-1111-1111-111111111111F');
@@ -199,6 +204,35 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 3
+(1 row)
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
+ array_agg
+------------------------
+ {1,2,3,4,5,6,7,8,9,10}
+(1 row)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
@@ -219,8 +253,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+SELECT uuid_extract_version(uuidv4()); --4
+ uuid_extract_version
+----------------------
+ 4
+(1 row)
+
+SELECT uuid_extract_version(uuidv7()); --7
+ uuid_extract_version
+----------------------
+ 7
+(1 row)
+
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
@@ -239,4 +291,4 @@ SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 75ee966ded..6eb8efbd3d 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -10,6 +10,11 @@ CREATE TABLE guid2
guid_field UUID,
text_field TEXT DEFAULT(now())
);
+CREATE TABLE guid3
+(
+ id SERIAL,
+ guid_field UUID
+);
-- inserting invalid data tests
-- too long
@@ -97,6 +102,22 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(INTERVAL '1 day'));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- test sortability of v7
+INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
+SELECT array_agg(id ORDER BY guid_field) FROM guid3;
-- extract functions
@@ -104,12 +125,15 @@ SELECT count(DISTINCT guid_field) FROM guid1;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
+SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv7()); --7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 4122bis test vector
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
-- clean up
-DROP TABLE guid1, guid2 CASCADE;
+DROP TABLE guid1, guid2, guid3 CASCADE;
--
2.39.5 (Apple Git-154)
[application/octet-stream] v40-0002-Fixes-to-address-review-notes-by-Peter-Eisentrau.patch (8.4K, ../../[email protected]/3-v40-0002-Fixes-to-address-review-notes-by-Peter-Eisentrau.patch)
download | inline diff:
From a7720773eb27f821d603f8aba2cff6453d0797a3 Mon Sep 17 00:00:00 2001
From: Andrey Borodin <[email protected]>
Date: Fri, 29 Nov 2024 23:04:57 +0500
Subject: [PATCH v40 2/2] Fixes to address review notes by Peter Eisentraut
---
doc/src/sgml/func.sgml | 4 +++-
src/backend/utils/adt/uuid.c | 27 ++++++++++++++++++------
src/include/catalog/pg_proc.dat | 10 ++++-----
src/test/regress/expected/opr_sanity.out | 3 ---
src/test/regress/expected/uuid.out | 8 +++----
src/test/regress/sql/uuid.sql | 8 +++----
6 files changed, 37 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index e9a2db2e93..f992d5266c 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14282,7 +14282,9 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
This function returns a version 7 UUID (UNIX timestamp with millisecond
- precision + sub-millisecond timestamp + random).
+ precision + sub-millisecond timestamp + random). This function can accept
+ optional <parameter>shift</parameter> parameter of type <type>interval</type>
+ which shift internal timestamp by the given interval.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index bb890e9f60..e962f85838 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -433,7 +433,9 @@ uuid_hash_extended(PG_FUNCTION_ARGS)
return hash_any_extended(key->data, UUID_LEN, PG_GETARG_INT64(1));
}
-/* Set the given UUID version and the variant bits */
+/*
+ * Set the given UUID version and the variant bits
+ */
static inline void
uuid_set_version(pg_uuid_t *uuid, unsigned char version)
{
@@ -469,6 +471,15 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+/*
+ * Wrapper for gen_random_uuid()
+ */
+Datum
+uuidv4(PG_FUNCTION_ARGS)
+{
+ return gen_random_uuid(fcinfo);
+}
+
/*
* Get the current timestamp with nanosecond precision for UUID generation.
* The returned timestamp is ensured to be at least SUBMS_MINIMAL_STEP greater
@@ -520,6 +531,9 @@ get_real_time_ns_ascending()
* method "Replace Leftmost Random Bits with Increased Clock Precision (Method 3)".
* This method utilizes 12 bits from the "rand_a" bits to store a 1/4096
* (or 2^12) fraction of sub-millisecond precision.
+ *
+ * ns is a number of nanoseconds since start of the UNIX epoch. This value is
+ * used for time-dependent bits of UUID.
*/
static pg_attribute_always_inline pg_uuid_t *
generate_uuidv7(int64 ns)
@@ -592,16 +606,17 @@ uuidv7(PG_FUNCTION_ARGS)
Datum
uuidv7_interval(PG_FUNCTION_ARGS)
{
- Interval *span = PG_GETARG_INTERVAL_P(0);
+ Interval *shift = PG_GETARG_INTERVAL_P(0);
TimestampTz ts;
pg_uuid_t *uuid;
int64 ns = get_real_time_ns_ascending();
/*
- * Shift the current timestamp by the given interval. To make correct
- * calculating the time shift, we convert the UNIX epoch to TimestampTz
+ * Shift the current timestamp by the given interval. To calsulate time
+ * shift correctly, we convert the UNIX epoch to TimestampTz
* and use timestamptz_pl_interval(). Since this calculation is done with
- * microsecond precision, we carry back the nanoseconds.
+ * microsecond precision, we carry nanoseconds from original ns value to
+ * shifted ns value.
*/
ts = (TimestampTz) (ns / NS_PER_US) -
@@ -610,7 +625,7 @@ uuidv7_interval(PG_FUNCTION_ARGS)
/* Compute time shift */
ts = DatumGetTimestampTz(DirectFunctionCall2(timestamptz_pl_interval,
TimestampTzGetDatum(ts),
- IntervalPGetDatum(span)));
+ IntervalPGetDatum(shift)));
/*
* Convert a TimestampTz value back to an UNIX epoch and carry back
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3353e9d6e3..046ce3eb67 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9343,13 +9343,13 @@
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
{ oid => '9895', descr => 'generate UUID version 4',
- proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
- prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+ proname => 'uuidv4', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv4' },
{ oid => '9896', descr => 'generate UUID version 7',
- proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ proname => 'uuidv7', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
-{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted on specific interval',
- proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted by specific interval',
+ proname => 'uuidv7', provolatile => 'v', proargnames => '{shift}',
prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 43e7180a16..34a32bd11d 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -878,9 +878,6 @@ crc32(bytea)
crc32c(bytea)
bytea_larger(bytea,bytea)
bytea_smaller(bytea,bytea)
-uuidv4()
-uuidv7()
-uuidv7(interval)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 0059a8c716..798633ad51 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -253,26 +253,26 @@ SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
-SELECT uuid_extract_version(uuidv4()); --4
+SELECT uuid_extract_version(uuidv4()); -- 4
uuid_extract_version
----------------------
4
(1 row)
-SELECT uuid_extract_version(uuidv7()); --7
+SELECT uuid_extract_version(uuidv7()); -- 7
uuid_extract_version
----------------------
7
(1 row)
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
?column?
----------
t
(1 row)
-SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
?column?
----------
t
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 6eb8efbd3d..110188361d 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -125,12 +125,12 @@ SELECT array_agg(id ORDER BY guid_field) FROM guid3;
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
SELECT uuid_extract_version(gen_random_uuid()); -- 4
SELECT uuid_extract_version('11111111-1111-1111-1111-111111111111'); -- null
-SELECT uuid_extract_version(uuidv4()); --4
-SELECT uuid_extract_version(uuidv7()); --7
+SELECT uuid_extract_version(uuidv4()); -- 4
+SELECT uuid_extract_version(uuidv7()); -- 7
-- timestamp
-SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
-SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
+SELECT uuid_extract_timestamp('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v1
+SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00'; -- RFC 9562 test vector for v7
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-29 18:49 Masahiko Sawada <[email protected]>
parent: Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-29 18:49 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Fri, Nov 29, 2024 at 10:39 AM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 29 Nov 2024, at 00:46, Peter Eisentraut <[email protected]> wrote:
> >
> > Here as well.
>
> Peter, many thanks for the next round of review. I agree with all corrections.
> I'm sending amendments addressing your review as a separate step in patch set. Step 1 of this patch set is identical to v39.
>
Thank you for updating the patch! Here are two comments:
<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
This function returns a version 7 UUID (UNIX timestamp with millisecond
- precision + sub-millisecond timestamp + random).
+ precision + sub-millisecond timestamp + random). This function can accept
+ optional <parameter>shift</parameter> parameter of type
<type>interval</type>
+ which shift internal timestamp by the given interval.
</para>
There is no "shift" parameter in the function synopsis.
Also, while reviewing the changes for func.sgml, I find that now that
we have 5 UUID functions, it might make sense to create a table for
UUID functions instead of describing functions separately. Which seems
to be more readable and consistent with other functions in docs.
---
+/*
+ * Wrapper for gen_random_uuid()
+ */
+Datum
+uuidv4(PG_FUNCTION_ARGS)
+{
+ return gen_random_uuid(fcinfo);
+}
Why do we need this? IIUC we marked uuidv4() (and uuidv7()) leafproof
because gen_random_uuid() is marked too. Otherwise, the following test
in opr_sanity would fail:
-- Considering only built-in procs (prolang = 12), look for multiple uses
-- of the same internal function (ie, matching prosrc fields). It's OK to
-- have several entries with different pronames for the same internal function,
-- but conflicts in the number of arguments and other critical items should
-- be complained of. (We don't check data types here; see next query.)
-- Note: ignore aggregate functions here, since they all point to the same
-- dummy built-in function.
Given that these functions don't need to be marked leakproof, does it
make sense to remove the leakproof mark from gen_random_uuid() too?
That way, we don't need the wrapper function.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-29 19:46 Marcos Pegoraro <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Marcos Pegoraro @ 2024-11-29 19:46 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
Em sex., 29 de nov. de 2024 às 15:49, Masahiko Sawada <[email protected]>
escreveu:
> <function>uuidv7</function> () <returnvalue>uuid</returnvalue>
>
Wouldn't it be better to change this to
<function>uuidv7</function> ([interval]) <returnvalue>uuid</returnvalue
and explain what that param is ?
regards
Marcos
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-29 20:17 Masahiko Sawada <[email protected]>
parent: Marcos Pegoraro <[email protected]>
0 siblings, 1 reply; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-29 20:17 UTC (permalink / raw)
To: Marcos Pegoraro <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Fri, Nov 29, 2024 at 11:47 AM Marcos Pegoraro <[email protected]> wrote:
>
> Em sex., 29 de nov. de 2024 às 15:49, Masahiko Sawada <[email protected]> escreveu:
>>
>> <function>uuidv7</function> () <returnvalue>uuid</returnvalue>
>
>
> Wouldn't it be better to change this to
> <function>uuidv7</function> ([interval]) <returnvalue>uuid</returnvalue
> and explain what that param is ?
Yes, the function synopsis in the doc should be either:
uuidv7([interval]) -> uuid
or
uuidv7([shift interval]) -> uuid
Since this function has only one function argument it doesn't
necessarily need an argument name 'shift'. So the proposed description
might be okay but we need to change at least the function synopsis.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Re: UUID v7
@ 2024-11-30 06:50 Masahiko Sawada <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Masahiko Sawada @ 2024-11-30 06:50 UTC (permalink / raw)
To: Marcos Pegoraro <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Fri, Nov 29, 2024 at 12:17 PM Masahiko Sawada <[email protected]> wrote:
>
> On Fri, Nov 29, 2024 at 11:47 AM Marcos Pegoraro <[email protected]> wrote:
> >
> > Em sex., 29 de nov. de 2024 às 15:49, Masahiko Sawada <[email protected]> escreveu:
> >>
> >> <function>uuidv7</function> () <returnvalue>uuid</returnvalue>
> >
> >
> > Wouldn't it be better to change this to
> > <function>uuidv7</function> ([interval]) <returnvalue>uuid</returnvalue
> > and explain what that param is ?
>
> Yes, the function synopsis in the doc should be either:
>
> uuidv7([interval]) -> uuid
>
> or
>
> uuidv7([shift interval]) -> uuid
>
> Since this function has only one function argument it doesn't
> necessarily need an argument name 'shift'. So the proposed description
> might be okay but we need to change at least the function synopsis.
>
I realized that the description of uuid_extract_timestamp() needs to
be updated as well since it now supports version 7 too:
<synopsis>
<function>uuid_extract_timestamp</function> (uuid)
<returnvalue>timestamp with time zone</returnvalue>
</synopsis>
This function extracts a <type>timestamp with time zone</type> from UUID
version 1. For other versions, this function returns null. Note that the
extracted timestamp is not necessarily exactly equal to the time the UUID
was generated; this depends on the implementation that generated the UUID.
</para>
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 61+ messages in thread
* Benchmark function for uuidv7()
@ 2024-12-04 20:37 Sergey Prokhorenko <[email protected]>
parent: Masahiko Sawada <[email protected]>
0 siblings, 0 replies; 61+ messages in thread
From: Sergey Prokhorenko @ 2024-12-04 20:37 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Friday 29 November 2024 at 08:55:09 pm GMT+3, Masahiko Sawada <[email protected]> wrote:
On Fri, Nov 29, 2024 at 5:59 AM Sergey Prokhorenko
<[email protected]> wrote:
>
>
>
> Sergey Prokhorenko [email protected]
>
>
> On Friday 29 November 2024 at 09:19:33 am GMT+3, Masahiko Sawada <[email protected]> wrote:
>
>
> On Thu, Nov 28, 2024 at 8:13 PM Sergey Prokhorenko
>
> <[email protected]> wrote:
> >
> > I mean to add not benchmark results to the patch, but functions so that everyone can compare themselves on their equipment. The comparison with UUIDv4 is not very interesting, as the choice is usually between UUIDv7 and an integer key. And I have described many use cases, and in your benchmark there is only one, the simplest.
>
>
> I don't think we should add such benchmark functions at least to this
> patch. If there already is a well-established workload using UUIDv7
> and UUIDv4 etc, users can use pgbench with custom scripts, or it might
> make sense to add it to pgbench as a built-in workload. Which however
> should be a separate patch. Having said that, I think users should use
> benchmarks that fit their workloads, and it would not be easy to
> establish workloads that are reasonable for most systems.
>
> Regards,
>
> --
> Masahiko Sawada
> Amazon Web Services: https://aws.amazon.com
>
>
>
>
>
>
> Workloads can and must be added with parameters. Typically, companies use test tables of 10,000 and 1,000,000 records, etc. Different companies have mostly similar usage scenarios (for example, incremental loading). Each company has to duplicate the work of others, creating the same benchmarks. The worst thing is that this is entrusted to incompetent employees who are not very good at understanding typical key usage scenarios. As a rule, these are programmers, not system analysts. Accordingly, the solution in 99% of cases will be in favor of integer keys, as they take up less space and are generated faster. If we leave this problem until the next patch, it will take us a year and a half. This is completely wrong.
There are still 4 months left until the feature freeze. We can discuss
this topic and might find solutions. I don't think it's a blocker of
this patch (UUIDv7 implementation patch).
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
_________________________________________________________________________________________
_________________________________________________________________________________________
I am not a programmer, but a systems analyst. Therefore, I cannot develop a function for the benchmark myself, but I can describe the requirements for this function.
The function for the benchmark could be implemented as a separate patch (not UUIDv7 implementation patch) to avoid blocking the UUIDv7 implementation patch.
Requirements for the uuidv7() benchmark function
1. Benchmark function
The function for uuidv7() benchmark should be called uuidv7_benchmark().
This function should have parameters:• optional parameter mock_table_record_count. If the parameter is not passed, then the value of 1 million records is taken• optional parameter payload_size_b. If the parameter is not passed, then the value of 2048 bytes is taken
2. Benchmark results table
The result of each benchmark step for a certain surrogate key type should be dumped into a separate row of the uuidv7_benchmark_results table. This table should contain the following columns:• benchmark_start_datetime• mock_table_record_count• step_name• surrogate_key_type• rate_per_ms (nullable)• cpu_usage_ percent (nullable)• memory_usage_mb (nullable)• drive_usage_mb_per_s (nullable)
The table rows should be sequentially sorted by the columns benchmark_start_datetime, step_name, surrogate_key_type.
Sample benchmark results table:
| benchmark_start_datetime | mock_table_record_count | step_name | surrogate_key_type | rate_per_ms | cpu_usage \_ percent | memory_usage_mb | drive_usage_mb_per_s || ------------------------ | ----------------------- | ----------------- | ------------------ | ----------- | -------------------- | --------------- | -------------------- || | | 1_insert | UUIDv7 | | | | || | | 1_insert | BIGSERIAL | | | | || | | 1_insert | UUIDv4 | | | | || | | 2_parallel_insert | UUIDv7 | | | | || | | 2_parallel_insert | BIGSERIAL | | | | || | | 2_parallel_insert | UUIDv4 | | | | || | | 3_left_join | UUIDv7 | | | | || | | 3_left_join | BIGSERIAL | | | | || | | 3_left_join | UUIDv4 | | | | || | | … | … | | | | |
3. Mock tableEach record in the mock_table table must contain the following columns:• id with the UUID data type, PRIMARY KEY (indexed)• payload with the bytea data type
4. Compared surrogate key typesEach benchmark step is run sequentially with the following surrogate key types in the mock_table.id column:• UUIDv7• BIGSERIAL• The name of the function for generating surrogate keys (for example, one of the formats: UUIDv4, ULID or Snowflake ID), if the user specifies such a function and makes it available
5. Benchmark steps
Before running the benchmark steps, a mock table is created.Pseudocode:CREATE TABLE mock_table (id UUID PRIMARY KEY DEFAULT uuidv7(),payload BYTEA);
The benchmark must have the following steps:
step_name = '1_insert'Pseudocode:INSERT INTO mock_table (payload)SELECT filled_payload(payload_size_b)FROM generate_series(1, mock_table_record_count);
step_name = '2_parallel_insert'The algorithm is at the discretion of the developer.
step_name = '3_left_join'Pseudocode:SELECT COUNT(*)FROM mock_table aLEFT JOIN mock_table b ON b.id = a.idWHERE b.id IS NULL;
step_name = '4_inner_join'Pseudocode:SELECT COUNT(*)FROM mock_table aINNER JOIN mock_table b ON b.id = a.idWHERE b.id IS NULL;
step_name = '5_group_by'Pseudocode:SELECT id, COUNT(*)FROM mock_tableGROUP BY idHAVING COUNT(*) > 1;
step_name = '6_delete'Pseudocode:DELETE FROM mock_table aUSING mock_table bWHERE b.id = a.id;
Regards,
Sergey Prokhorenko
[email protected]
^ permalink raw reply [nested|flat] 61+ messages in thread
end of thread, other threads:[~2024-12-04 20:37 UTC | newest]
Thread overview: 61+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-09-11 20:13 Re: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c) Andrew Dunstan <[email protected]>
2018-09-11 20:27 ` Andres Freund <[email protected]>
2019-04-03 00:13 [PATCH v1] Cleanup/remove/update references to OID column... Justin Pryzby <[email protected]>
2019-04-03 00:13 [PATCH v3] Cleanup/remove/update references to OID column... Justin Pryzby <[email protected]>
2019-04-03 00:13 [PATCH v2] Cleanup/remove/update references to OID column... Justin Pryzby <[email protected]>
2019-04-03 00:13 [PATCH v3 12/12] Cleanup/remove/update references to OID column... Justin Pryzby <[email protected]>
2019-05-08 18:57 [PATCH v1] Cleanup/remove/update references to OID column... Justin Pryzby <[email protected]>
2022-11-23 07:52 drop postmaster symlink Peter Eisentraut <[email protected]>
2022-11-23 14:18 ` Re: drop postmaster symlink Joe Conway <[email protected]>
2022-11-23 14:28 ` Re: drop postmaster symlink Devrim Gündüz <[email protected]>
2022-11-23 15:07 ` Re: drop postmaster symlink Tom Lane <[email protected]>
2022-11-23 19:50 ` Re: drop postmaster symlink Andres Freund <[email protected]>
2022-11-23 20:10 ` Re: drop postmaster symlink Robert Haas <[email protected]>
2022-11-23 20:32 ` Re: drop postmaster symlink Joe Conway <[email protected]>
2022-11-24 00:15 ` Re: drop postmaster symlink Daniel Gustafsson <[email protected]>
2022-11-23 20:48 ` Re: drop postmaster symlink Tom Lane <[email protected]>
2022-11-23 21:08 ` Re: drop postmaster symlink Andres Freund <[email protected]>
2024-11-11 20:20 Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-15 01:44 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-16 19:06 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-17 18:39 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-18 21:16 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-19 09:31 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-19 17:44 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-19 19:06 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-20 03:48 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-20 23:24 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-21 21:22 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-22 22:37 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-23 07:58 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-23 08:20 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-25 06:08 ` Re: UUID v7 wenhui qiu <[email protected]>
2024-11-25 17:53 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-25 18:14 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-25 20:11 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-26 13:14 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-26 15:14 ` Re: UUID v7 Japin Li <[email protected]>
2024-11-26 19:11 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-11-26 19:30 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-26 19:35 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-11-26 20:47 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-11-26 21:55 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-11-26 23:11 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-27 03:11 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-11-27 18:24 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-27 23:07 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-11-27 23:28 ` Отв.: Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-11-28 08:09 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-29 04:13 ` Отв.: Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-11-29 04:22 ` Re: Отв.: Re: UUID v7 Kirill Reshke <[email protected]>
2024-11-29 06:18 ` Re: Отв.: Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-29 13:57 ` Re: Отв.: Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-11-29 16:40 ` Re: Отв.: Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-29 17:54 ` Re: Отв.: Re: UUID v7 Masahiko Sawada <[email protected]>
2024-12-04 20:37 ` Benchmark function for uuidv7() Sergey Prokhorenko <[email protected]>
2024-11-28 19:46 ` Re: UUID v7 Peter Eisentraut <[email protected]>
2024-11-29 18:39 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-11-29 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-29 19:46 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
2024-11-29 20:17 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2024-11-30 06:50 ` Re: UUID v7 Masahiko Sawada <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox