public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v1] Common Era for dates
68+ messages / 15 participants
[nested] [flat]
* [PATCH v1] Common Era for dates
@ 2021-06-09 06:44 David Fetter <[email protected]>
0 siblings, 0 replies; 68+ messages in thread
From: David Fetter @ 2021-06-09 06:44 UTC (permalink / raw)
---
doc/src/sgml/datatype.sgml | 25 ++++----
doc/src/sgml/datetime.sgml | 4 +-
doc/src/sgml/func.sgml | 16 +++---
src/backend/utils/adt/formatting.c | 80 +++++++++++++++++++++++---
src/test/regress/expected/date.out | 72 +++++++++++------------
src/test/regress/expected/horology.out | 4 +-
src/test/regress/sql/date.sql | 4 +-
7 files changed, 134 insertions(+), 71 deletions(-)
diff --git doc/src/sgml/datatype.sgml doc/src/sgml/datatype.sgml
index 8e30b82273..d0ca9e500f 100644
--- doc/src/sgml/datatype.sgml
+++ doc/src/sgml/datatype.sgml
@@ -1740,24 +1740,24 @@ SELECT 'abc \153\154\155 \052\251\124'::bytea;
<entry><type>timestamp [ (<replaceable>p</replaceable>) ] [ without time zone ]</type></entry>
<entry>8 bytes</entry>
<entry>both date and time (no time zone)</entry>
- <entry>4713 BC</entry>
- <entry>294276 AD</entry>
+ <entry>4713 BC[E]</entry>
+ <entry>294276 AD/CE</entry>
<entry>1 microsecond</entry>
</row>
<row>
<entry><type>timestamp [ (<replaceable>p</replaceable>) ] with time zone</type></entry>
<entry>8 bytes</entry>
<entry>both date and time, with time zone</entry>
- <entry>4713 BC</entry>
- <entry>294276 AD</entry>
+ <entry>4713 BC[E]</entry>
+ <entry>294276 AD/CE</entry>
<entry>1 microsecond</entry>
</row>
<row>
<entry><type>date</type></entry>
<entry>4 bytes</entry>
<entry>date (no time of day)</entry>
- <entry>4713 BC</entry>
- <entry>5874897 AD</entry>
+ <entry>4713 BC[E]</entry>
+ <entry>5874897 AD/CE</entry>
<entry>1 day</entry>
</row>
<row>
@@ -2168,23 +2168,24 @@ MINUTE TO SECOND
<para>
Valid input for the time stamp types consists of the concatenation
of a date and a time, followed by an optional time zone,
- followed by an optional <literal>AD</literal> or <literal>BC</literal>.
- (Alternatively, <literal>AD</literal>/<literal>BC</literal> can appear
- before the time zone, but this is not the preferred ordering.)
+ followed by an optional <literal>AD</literal>, <literal>CE</literal>,
+ <literal>BC</literal>, or <literal>BCE</literal>
+ (Alternatively, <literal>AD</literal>/<literal>CE</literal>/<literal>BC</literal>/<literal>BCE</literal>
+ can appear before the time zone, but this is not the preferred ordering.)
Thus:
<programlisting>
-1999-01-08 04:05:06
+2021-01-08 04:05:06
</programlisting>
and:
<programlisting>
-1999-01-08 04:05:06 -8:00
+2021-01-08 04:05:06 -8:00
</programlisting>
are valid values, which follow the <acronym>ISO</acronym> 8601
standard. In addition, the common format:
<programlisting>
-January 8 04:05:06 1999 PST
+January 8 04:05:06 2021 PST
</programlisting>
is supported.
</para>
diff --git doc/src/sgml/datetime.sgml doc/src/sgml/datetime.sgml
index ecc3245a46..6266328107 100644
--- doc/src/sgml/datetime.sgml
+++ doc/src/sgml/datetime.sgml
@@ -167,8 +167,8 @@
<tip>
<para>
- Gregorian years AD 1–99 can be entered by using 4 digits with leading
- zeros (e.g., <literal>0099</literal> is AD 99).
+ Gregorian years AD/CE 1–99 can be entered by using 4 digits with leading
+ zeros (e.g., <literal>0099</literal> is AD/CE 99).
</para>
</tip>
</para>
diff --git doc/src/sgml/func.sgml doc/src/sgml/func.sgml
index 7b652460a1..3fcb9793e7 100644
--- doc/src/sgml/func.sgml
+++ doc/src/sgml/func.sgml
@@ -7864,13 +7864,13 @@ SELECT regexp_match('abc01234xyz', '(?:(.*?)(\d+)(.*)){1,1}');
<entry>last digit of ISO 8601 week-numbering year</entry>
</row>
<row>
- <entry><literal>BC</literal>, <literal>bc</literal>,
- <literal>AD</literal> or <literal>ad</literal></entry>
+ <entry><literal>BC</literal>, <literal>bc</literal>, <literal>BCE</literal>, <literal>bce</literal>
+ <literal>AD</literal>, <literal>ad</literal>, <literal>CE</literal>, <literal>ce</literal></entry>
<entry>era indicator (without periods)</entry>
</row>
<row>
- <entry><literal>B.C.</literal>, <literal>b.c.</literal>,
- <literal>A.D.</literal> or <literal>a.d.</literal></entry>
+ <entry><literal>B.C.</literal>, <literal>b.c.</literal>, <literal>B.C.E.</literal>, <literal>b.c.e.</literal>
+ <literal>A.D.</literal>, <literal>a.d.</literal>, <literal>C.E.</literal>, or <literal>c.e.</literal></entry>
<entry>era indicator (with periods)</entry>
</row>
<row>
@@ -9849,13 +9849,15 @@ SELECT EXTRACT(CENTURY FROM TIMESTAMP '2001-02-16 20:38:40');
</screen>
<para>
- The first century starts at 0001-01-01 00:00:00 AD, although
- they did not know it at the time. This definition applies to all
+ The first century starts at 0001-01-01 00:00:00 AD/CE, although
+ the calendar did not exist at the time. This definition applies to all
Gregorian calendar countries. There is no century number 0,
you go from -1 century to 1 century.
If you disagree with this, please write your complaint to:
- Pope, Cathedral Saint-Peter of Roma, Vatican.
+ His Holiness, Pope Francis
+ Apostolic Palace
+ 00120 Vatican City
</para>
</listitem>
</varlistentry>
diff --git src/backend/utils/adt/formatting.c src/backend/utils/adt/formatting.c
index e909c1a200..11876dbd19 100644
--- src/backend/utils/adt/formatting.c
+++ src/backend/utils/adt/formatting.c
@@ -238,11 +238,19 @@ static const char *const days_short[] = {
#define a_d_STR "a.d."
#define AD_STR "AD"
#define ad_STR "ad"
+#define C_E_STR "C.E."
+#define c_e_STR "c.e."
+#define CE_STR "CE"
+#define ce_STR "ce"
#define B_C_STR "B.C."
#define b_c_STR "b.c."
#define BC_STR "BC"
#define bc_STR "bc"
+#define B_C_E_STR "B.C.E."
+#define b_c_e_STR "b.c.e."
+#define BCE_STR "BCE"
+#define bce_STR "bce"
/*
* AD / BC strings for seq_search.
@@ -254,8 +262,8 @@ static const char *const days_short[] = {
* matches for BC have an odd index. So the boolean value for BC is given by
* taking the array index of the match, modulo 2.
*/
-static const char *const adbc_strings[] = {ad_STR, bc_STR, AD_STR, BC_STR, NULL};
-static const char *const adbc_strings_long[] = {a_d_STR, b_c_STR, A_D_STR, B_C_STR, NULL};
+static const char *const adbc_strings[] = {ad_STR, bc_STR, AD_STR, BC_STR, CE_STR, BCE_STR, NULL};
+static const char *const adbc_strings_long[] = {a_d_STR, b_c_STR, A_D_STR, B_C_STR, c_e_STR, b_c_e_STR, C_E_STR, B_C_E_STR, NULL};
/* ----------
* AM / PM
@@ -643,9 +651,13 @@ typedef enum
DCH_A_M,
DCH_AD,
DCH_AM,
+ DCH_B_C_E,
DCH_B_C,
+ DCH_BCE,
DCH_BC,
DCH_CC,
+ DCH_CE,
+ DCH_C_E,
DCH_DAY,
DCH_DDD,
DCH_DD,
@@ -701,9 +713,13 @@ typedef enum
DCH_a_m,
DCH_ad,
DCH_am,
+ DCH_b_c_e,
DCH_b_c,
+ DCH_bce,
DCH_bc,
DCH_cc,
+ DCH_c_e,
+ DCH_ce,
DCH_day,
DCH_ddd,
DCH_dd,
@@ -809,9 +825,13 @@ static const KeyWord DCH_keywords[] = {
{"A.M.", 4, DCH_A_M, false, FROM_CHAR_DATE_NONE},
{"AD", 2, DCH_AD, false, FROM_CHAR_DATE_NONE},
{"AM", 2, DCH_AM, false, FROM_CHAR_DATE_NONE},
- {"B.C.", 4, DCH_B_C, false, FROM_CHAR_DATE_NONE}, /* B */
+ {"B.C.E.", 6, DCH_B_C_E, false, FROM_CHAR_DATE_NONE}, /* B */
+ {"B.C.", 4, DCH_B_C, false, FROM_CHAR_DATE_NONE},
+ {"BCE", 3, DCH_BCE, false, FROM_CHAR_DATE_NONE},
{"BC", 2, DCH_BC, false, FROM_CHAR_DATE_NONE},
- {"CC", 2, DCH_CC, true, FROM_CHAR_DATE_NONE}, /* C */
+ {"CC", 2, DCH_CC, true, FROM_CHAR_DATE_NONE}, /* C */
+ {"CE", 2, DCH_CE, false, FROM_CHAR_DATE_NONE},
+ {"C.E.", 4, DCH_CE, false, FROM_CHAR_DATE_NONE},
{"DAY", 3, DCH_DAY, false, FROM_CHAR_DATE_NONE}, /* D */
{"DDD", 3, DCH_DDD, true, FROM_CHAR_DATE_GREGORIAN},
{"DD", 2, DCH_DD, true, FROM_CHAR_DATE_GREGORIAN},
@@ -867,9 +887,13 @@ static const KeyWord DCH_keywords[] = {
{"a.m.", 4, DCH_a_m, false, FROM_CHAR_DATE_NONE},
{"ad", 2, DCH_ad, false, FROM_CHAR_DATE_NONE},
{"am", 2, DCH_am, false, FROM_CHAR_DATE_NONE},
- {"b.c.", 4, DCH_b_c, false, FROM_CHAR_DATE_NONE}, /* b */
+ {"b.c.e.", 6, DCH_b_c_e, false, FROM_CHAR_DATE_NONE}, /* b */
+ {"b.c.", 4, DCH_b_c, false, FROM_CHAR_DATE_NONE},
+ {"bce", 3, DCH_bce, false, FROM_CHAR_DATE_NONE},
{"bc", 2, DCH_bc, false, FROM_CHAR_DATE_NONE},
- {"cc", 2, DCH_CC, true, FROM_CHAR_DATE_NONE}, /* c */
+ {"cc", 2, DCH_cc, true, FROM_CHAR_DATE_NONE}, /* c */
+ {"c.e.", 4, DCH_c_e, true, FROM_CHAR_DATE_NONE},
+ {"ce", 2, DCH_ce, true, FROM_CHAR_DATE_NONE},
{"day", 3, DCH_day, false, FROM_CHAR_DATE_NONE}, /* d */
{"ddd", 3, DCH_DDD, true, FROM_CHAR_DATE_GREGORIAN},
{"dd", 2, DCH_DD, true, FROM_CHAR_DATE_GREGORIAN},
@@ -985,10 +1009,10 @@ static const int DCH_index[KeyWord_INDEX_SIZE] = {
-1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
- -1, -1, -1, -1, -1, DCH_A_D, DCH_B_C, DCH_CC, DCH_DAY, -1,
+ -1, -1, -1, -1, -1, DCH_A_D, DCH_B_C_E, DCH_CC, DCH_DAY, -1,
DCH_FF1, -1, DCH_HH24, DCH_IDDD, DCH_J, -1, -1, DCH_MI, -1, DCH_OF,
DCH_P_M, DCH_Q, DCH_RM, DCH_SSSSS, DCH_TZH, DCH_US, -1, DCH_WW, -1, DCH_Y_YYY,
- -1, -1, -1, -1, -1, -1, -1, DCH_a_d, DCH_b_c, DCH_cc,
+ -1, -1, -1, -1, -1, -1, -1, DCH_a_d, DCH_b_c_e, DCH_cc,
DCH_day, -1, DCH_ff1, -1, DCH_hh24, DCH_iddd, DCH_j, -1, -1, DCH_mi,
-1, DCH_of, DCH_p_m, DCH_q, DCH_rm, DCH_sssss, DCH_tzh, DCH_us, -1, DCH_ww,
-1, DCH_y_yyy, -1, -1, -1, -1
@@ -2865,6 +2889,30 @@ DCH_to_char(FormatNode *node, bool is_interval, TmToChar *in, char *out, Oid col
strcpy(s, (tm->tm_year <= 0 ? bc_STR : ad_STR));
s += strlen(s);
break;
+ case DCH_C_E:
+ case DCH_B_C_E:
+ INVALID_FOR_INTERVAL;
+ strcpy(s, (tm->tm_year <= 0 ? B_C_E_STR : C_E_STR));
+ s += strlen(s);
+ break;
+ case DCH_CE:
+ case DCH_BCE:
+ INVALID_FOR_INTERVAL;
+ strcpy(s, (tm->tm_year <= 0 ? BCE_STR : CE_STR));
+ s += strlen(s);
+ break;
+ case DCH_c_e:
+ case DCH_b_c_e:
+ INVALID_FOR_INTERVAL;
+ strcpy(s, (tm->tm_year <= 0 ? b_c_e_STR : c_e_STR));
+ s += strlen(s);
+ break;
+ case DCH_ce:
+ case DCH_bce:
+ INVALID_FOR_INTERVAL;
+ strcpy(s, (tm->tm_year <= 0 ? bce_STR : ce_STR));
+ s += strlen(s);
+ break;
case DCH_MONTH:
INVALID_FOR_INTERVAL;
if (!tm->tm_mon)
@@ -3570,8 +3618,12 @@ DCH_from_char(FormatNode *node, const char *in, TmFromChar *out,
break;
case DCH_A_D:
case DCH_B_C:
+ case DCH_B_C_E:
+ case DCH_C_E:
case DCH_a_d:
case DCH_b_c:
+ case DCH_b_c_e:
+ case DCH_c_e:
from_char_seq_search(&value, &s, adbc_strings_long,
NULL, InvalidOid,
n, have_error);
@@ -3581,8 +3633,12 @@ DCH_from_char(FormatNode *node, const char *in, TmFromChar *out,
break;
case DCH_AD:
case DCH_BC:
+ case DCH_BCE:
+ case DCH_CE:
case DCH_ad:
case DCH_bc:
+ case DCH_bce:
+ case DCH_ce:
from_char_seq_search(&value, &s, adbc_strings,
NULL, InvalidOid,
n, have_error);
@@ -3886,12 +3942,20 @@ DCH_datetime_type(FormatNode *node, bool *have_error)
break;
case DCH_A_D:
case DCH_B_C:
+ case DCH_B_C_E:
+ case DCH_C_E:
case DCH_a_d:
case DCH_b_c:
+ case DCH_b_c_e:
+ case DCH_c_e:
case DCH_AD:
case DCH_BC:
+ case DCH_BCE:
+ case DCH_CE:
case DCH_ad:
case DCH_bc:
+ case DCH_bce:
+ case DCH_ce:
case DCH_MONTH:
case DCH_Month:
case DCH_month:
diff --git src/test/regress/expected/date.out src/test/regress/expected/date.out
index 75ff659378..d7529f1783 100644
--- src/test/regress/expected/date.out
+++ src/test/regress/expected/date.out
@@ -21,10 +21,13 @@ INSERT INTO DATE_TBL VALUES ('2000-04-03');
INSERT INTO DATE_TBL VALUES ('2038-04-08');
INSERT INTO DATE_TBL VALUES ('2039-04-09');
INSERT INTO DATE_TBL VALUES ('2040-04-10');
-INSERT INTO DATE_TBL VALUES ('2040-04-10 BC');
+INSERT INTO DATE_TBL VALUES ('2040-04-10 BCE');
+ERROR: invalid input syntax for type date: "2040-04-10 BCE"
+LINE 1: INSERT INTO DATE_TBL VALUES ('2040-04-10 BCE');
+ ^
SELECT f1 FROM DATE_TBL;
- f1
----------------
+ f1
+------------
04-09-1957
06-13-1957
02-28-1996
@@ -40,12 +43,11 @@ SELECT f1 FROM DATE_TBL;
04-08-2038
04-09-2039
04-10-2040
- 04-10-2040 BC
-(16 rows)
+(15 rows)
SELECT f1 FROM DATE_TBL WHERE f1 < '2000-01-01';
- f1
----------------
+ f1
+------------
04-09-1957
06-13-1957
02-28-1996
@@ -55,8 +57,7 @@ SELECT f1 FROM DATE_TBL WHERE f1 < '2000-01-01';
02-28-1997
03-01-1997
03-02-1997
- 04-10-2040 BC
-(10 rows)
+(9 rows)
SELECT f1 FROM DATE_TBL
WHERE f1 BETWEEN '2000-01-01' AND '2001-01-01';
@@ -863,8 +864,7 @@ SELECT f1 - date '2000-01-01' AS "Days From 2K" FROM DATE_TBL;
13977
14343
14710
- -1475115
-(16 rows)
+(15 rows)
SELECT f1 - date 'epoch' AS "Days From Epoch" FROM DATE_TBL;
Days From Epoch
@@ -884,8 +884,7 @@ SELECT f1 - date 'epoch' AS "Days From Epoch" FROM DATE_TBL;
24934
25300
25667
- -1464158
-(16 rows)
+(15 rows)
SELECT date 'yesterday' - date 'today' AS "One day";
One day
@@ -942,25 +941,24 @@ SELECT f1 as "date",
date_part('julian', f1) AS julian,
date_part('epoch', f1) AS epoch
FROM date_tbl;
- date | year | month | day | quarter | decade | century | millennium | isoyear | week | dow | isodow | doy | julian | epoch
----------------+-------+-------+-----+---------+--------+---------+------------+---------+------+-----+--------+-----+---------+---------------
- 04-09-1957 | 1957 | 4 | 9 | 2 | 195 | 20 | 2 | 1957 | 15 | 2 | 2 | 99 | 2435938 | -401760000
- 06-13-1957 | 1957 | 6 | 13 | 2 | 195 | 20 | 2 | 1957 | 24 | 4 | 4 | 164 | 2436003 | -396144000
- 02-28-1996 | 1996 | 2 | 28 | 1 | 199 | 20 | 2 | 1996 | 9 | 3 | 3 | 59 | 2450142 | 825465600
- 02-29-1996 | 1996 | 2 | 29 | 1 | 199 | 20 | 2 | 1996 | 9 | 4 | 4 | 60 | 2450143 | 825552000
- 03-01-1996 | 1996 | 3 | 1 | 1 | 199 | 20 | 2 | 1996 | 9 | 5 | 5 | 61 | 2450144 | 825638400
- 03-02-1996 | 1996 | 3 | 2 | 1 | 199 | 20 | 2 | 1996 | 9 | 6 | 6 | 62 | 2450145 | 825724800
- 02-28-1997 | 1997 | 2 | 28 | 1 | 199 | 20 | 2 | 1997 | 9 | 5 | 5 | 59 | 2450508 | 857088000
- 03-01-1997 | 1997 | 3 | 1 | 1 | 199 | 20 | 2 | 1997 | 9 | 6 | 6 | 60 | 2450509 | 857174400
- 03-02-1997 | 1997 | 3 | 2 | 1 | 199 | 20 | 2 | 1997 | 9 | 0 | 7 | 61 | 2450510 | 857260800
- 04-01-2000 | 2000 | 4 | 1 | 2 | 200 | 20 | 2 | 2000 | 13 | 6 | 6 | 92 | 2451636 | 954547200
- 04-02-2000 | 2000 | 4 | 2 | 2 | 200 | 20 | 2 | 2000 | 13 | 0 | 7 | 93 | 2451637 | 954633600
- 04-03-2000 | 2000 | 4 | 3 | 2 | 200 | 20 | 2 | 2000 | 14 | 1 | 1 | 94 | 2451638 | 954720000
- 04-08-2038 | 2038 | 4 | 8 | 2 | 203 | 21 | 3 | 2038 | 14 | 4 | 4 | 98 | 2465522 | 2154297600
- 04-09-2039 | 2039 | 4 | 9 | 2 | 203 | 21 | 3 | 2039 | 14 | 6 | 6 | 99 | 2465888 | 2185920000
- 04-10-2040 | 2040 | 4 | 10 | 2 | 204 | 21 | 3 | 2040 | 15 | 2 | 2 | 101 | 2466255 | 2217628800
- 04-10-2040 BC | -2040 | 4 | 10 | 2 | -204 | -21 | -3 | -2040 | 15 | 1 | 1 | 100 | 976430 | -126503251200
-(16 rows)
+ date | year | month | day | quarter | decade | century | millennium | isoyear | week | dow | isodow | doy | julian | epoch
+------------+------+-------+-----+---------+--------+---------+------------+---------+------+-----+--------+-----+---------+------------
+ 04-09-1957 | 1957 | 4 | 9 | 2 | 195 | 20 | 2 | 1957 | 15 | 2 | 2 | 99 | 2435938 | -401760000
+ 06-13-1957 | 1957 | 6 | 13 | 2 | 195 | 20 | 2 | 1957 | 24 | 4 | 4 | 164 | 2436003 | -396144000
+ 02-28-1996 | 1996 | 2 | 28 | 1 | 199 | 20 | 2 | 1996 | 9 | 3 | 3 | 59 | 2450142 | 825465600
+ 02-29-1996 | 1996 | 2 | 29 | 1 | 199 | 20 | 2 | 1996 | 9 | 4 | 4 | 60 | 2450143 | 825552000
+ 03-01-1996 | 1996 | 3 | 1 | 1 | 199 | 20 | 2 | 1996 | 9 | 5 | 5 | 61 | 2450144 | 825638400
+ 03-02-1996 | 1996 | 3 | 2 | 1 | 199 | 20 | 2 | 1996 | 9 | 6 | 6 | 62 | 2450145 | 825724800
+ 02-28-1997 | 1997 | 2 | 28 | 1 | 199 | 20 | 2 | 1997 | 9 | 5 | 5 | 59 | 2450508 | 857088000
+ 03-01-1997 | 1997 | 3 | 1 | 1 | 199 | 20 | 2 | 1997 | 9 | 6 | 6 | 60 | 2450509 | 857174400
+ 03-02-1997 | 1997 | 3 | 2 | 1 | 199 | 20 | 2 | 1997 | 9 | 0 | 7 | 61 | 2450510 | 857260800
+ 04-01-2000 | 2000 | 4 | 1 | 2 | 200 | 20 | 2 | 2000 | 13 | 6 | 6 | 92 | 2451636 | 954547200
+ 04-02-2000 | 2000 | 4 | 2 | 2 | 200 | 20 | 2 | 2000 | 13 | 0 | 7 | 93 | 2451637 | 954633600
+ 04-03-2000 | 2000 | 4 | 3 | 2 | 200 | 20 | 2 | 2000 | 14 | 1 | 1 | 94 | 2451638 | 954720000
+ 04-08-2038 | 2038 | 4 | 8 | 2 | 203 | 21 | 3 | 2038 | 14 | 4 | 4 | 98 | 2465522 | 2154297600
+ 04-09-2039 | 2039 | 4 | 9 | 2 | 203 | 21 | 3 | 2039 | 14 | 6 | 6 | 99 | 2465888 | 2185920000
+ 04-10-2040 | 2040 | 4 | 10 | 2 | 204 | 21 | 3 | 2040 | 15 | 2 | 2 | 101 | 2466255 | 2217628800
+(15 rows)
--
-- epoch
@@ -1043,12 +1041,10 @@ SELECT EXTRACT(MILLENNIUM FROM DATE '0001-12-31 BC'); -- -1
-1
(1 row)
-SELECT EXTRACT(MILLENNIUM FROM DATE '0001-01-01 AD'); -- 1
- extract
----------
- 1
-(1 row)
-
+SELECT EXTRACT(MILLENNIUM FROM DATE '0001-01-01 CE'); -- 1
+ERROR: invalid input syntax for type date: "0001-01-01 CE"
+LINE 1: SELECT EXTRACT(MILLENNIUM FROM DATE '0001-01-01 CE');
+ ^
SELECT EXTRACT(MILLENNIUM FROM DATE '1000-12-31'); -- 1
extract
---------
diff --git src/test/regress/expected/horology.out src/test/regress/expected/horology.out
index de73683690..75847c52d4 100644
--- src/test/regress/expected/horology.out
+++ src/test/regress/expected/horology.out
@@ -2181,7 +2181,7 @@ select count(*) from date_tbl
where f1 not between '1997-01-01' and '1998-01-01';
count
-------
- 13
+ 12
(1 row)
explain (costs off)
@@ -2215,7 +2215,7 @@ select count(*) from date_tbl
where f1 not between symmetric '1997-01-01' and '1998-01-01';
count
-------
- 13
+ 12
(1 row)
--
diff --git src/test/regress/sql/date.sql src/test/regress/sql/date.sql
index 8f7435b767..c7c3ba4e0c 100644
--- src/test/regress/sql/date.sql
+++ src/test/regress/sql/date.sql
@@ -20,7 +20,7 @@ INSERT INTO DATE_TBL VALUES ('2000-04-03');
INSERT INTO DATE_TBL VALUES ('2038-04-08');
INSERT INTO DATE_TBL VALUES ('2039-04-09');
INSERT INTO DATE_TBL VALUES ('2040-04-10');
-INSERT INTO DATE_TBL VALUES ('2040-04-10 BC');
+INSERT INTO DATE_TBL VALUES ('2040-04-10 BCE');
SELECT f1 FROM DATE_TBL;
@@ -256,7 +256,7 @@ SELECT EXTRACT(CENTURY FROM CURRENT_DATE)>=21 AS True; -- true
-- millennium
--
SELECT EXTRACT(MILLENNIUM FROM DATE '0001-12-31 BC'); -- -1
-SELECT EXTRACT(MILLENNIUM FROM DATE '0001-01-01 AD'); -- 1
+SELECT EXTRACT(MILLENNIUM FROM DATE '0001-01-01 CE'); -- 1
SELECT EXTRACT(MILLENNIUM FROM DATE '1000-12-31'); -- 1
SELECT EXTRACT(MILLENNIUM FROM DATE '1001-01-01'); -- 2
SELECT EXTRACT(MILLENNIUM FROM DATE '2000-12-31'); -- 2
--
2.36.1
--k+w/mQv8wyuph6w0--
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
@ 2024-01-18 13:17 ` Andrey Borodin <[email protected]>
2024-01-18 20:26 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-19 02:58 ` Re: UUID v7 Lukas Fittl <[email protected]>
1 sibling, 2 replies; 68+ messages in thread
From: Andrey Borodin @ 2024-01-18 13:17 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 17 Jan 2024, at 02:19, Jelte Fennema-Nio <[email protected]> wrote:
I want to ask Kyzer or Brad, I hope they will see this message. I'm working on the patch for time extraction for v1, v6 and v7.
Do I understand correctly, that UUIDs contain local time, not UTC time? For examples in [0] I see that "A.6. Example of a UUIDv7 Value" I see that February 22, 2022 2:22:22.00 PM GMT-05:00 results in unix_ts_ms = 0x017F22E279B0, which is not UTC, but local time.
Is it intentional? Section "5.1. UUID Version 1" states otherwise.
If so, I should swap signatures of functions from TimestampTz to Timestamp.
I'm hard-coding examples from this standard to tests, so I want to be precise...
If I follow the standard I see this in tests:
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') at time zone 'GMT-05';
+ timezone
+--------------------------
+ Wed Feb 23 00:22:22 2022
+(1 row)
Current patch version attached. I've addressed all other requests: function renames, aliases, multiple functions instead of optional params, cleaner catalog definitions, not throwing error when [var,ver,time] value is unknown.
What is left: deal with timezones, improve documentation.
Best regards, Andrey Borodin.
[0] https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis#name-example-of-a-uuidv1-value
Attachments:
[application/octet-stream] v10-0001-Implement-UUID-v7-as-per-IETF-draft.patch (15.1K, ../../[email protected]/2-v10-0001-Implement-UUID-v7-as-per-IETF-draft.patch)
download | inline diff:
From f5e527d49a792b9b140562c7d42e0b1b15f5b315 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 20 Aug 2023 23:55:31 +0300
Subject: [PATCH v10] Implement UUID v7 as per IETF draft
This commit addes function to generate UUID v7.
This function optionally accepts datetime used to generate
next UUID.
Also we add a function to extract timestamp from UUID v7.
Authors: Andrey Borodin, Sergey Prokhorenko
---
doc/src/sgml/func.sgml | 18 ++-
src/backend/utils/adt/uuid.c | 192 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 19 +++
src/test/regress/expected/opr_sanity.out | 13 +-
src/test/regress/expected/uuid.out | 88 +++++++++++
src/test/regress/sql/uuid.sql | 35 +++++
6 files changed, 361 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 210c7c0b02..417e0c7f19 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14130,13 +14130,29 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_time</primary>
+ </indexterm>
+
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes two functions to generate a UUID:
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
</synopsis>
This function returns a version 4 (random) UUID. This is the most commonly
used type of UUID and is appropriate for most applications.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 (time-ordered + random) UUID.
+<synopsis>
+<function>uuid_extract_time</function> (uuid) <returnvalue>timestamptz</returnvalue>
+</synopsis>
+ This function extracts a timestamptz from UUID version 7.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 73dfd711c7..3538fcba6d 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,13 +13,18 @@
#include "postgres.h"
+#include <sys/time.h>
+
+#include "access/xlog.h"
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
+#include "utils/datetime.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
+#include "utils/timestamp.h"
#include "utils/uuid.h"
/* sortsupport for uuid */
@@ -421,3 +426,190 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+
+static uint32_t sequence_counter;
+static uint64_t previous_timestamp = 0;
+
+
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ TimestampTz ts;
+ uint64_t tms;
+ struct timeval tp;
+ bool increment_counter;
+
+ if (PG_NARGS() == 0 || PG_ARGISNULL(0))
+ {
+ gettimeofday(&tp, NULL);
+ tms = ((uint64_t)tp.tv_sec) * 1000 + (tp.tv_usec) / 1000;
+ /* time from clock is protected from backward leaps */
+ increment_counter = tms <= previous_timestamp;
+ }
+ else
+ {
+ ts = PG_GETARG_TIMESTAMPTZ(0);
+ tms = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC) / 1000;
+ /*
+ * The time can leap backwards when provided by the user, so we use
+ * counter only when called with exactly same unix_ts_ms argument.
+ */
+ increment_counter = (tms == previous_timestamp);
+ }
+
+ if (increment_counter)
+ {
+ /* Time did not increment from the previous generation, we must increment counter */
+ ++sequence_counter;
+ if (sequence_counter > 0x3ffff)
+ {
+ /* We only have 18-bit counter */
+ sequence_counter = 0;
+ previous_timestamp++;
+ }
+
+ /* protection from leap backward */
+ tms = previous_timestamp;
+
+ /* fill everything after the timestamp and counter with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /* most significant 4 bits of 18-bit counter */
+ uuid->data[6] = (unsigned char)(sequence_counter >> 14);
+ /* next 8 bits */
+ uuid->data[7] = (unsigned char)(sequence_counter >> 6);
+ /* least significant 6 bits */
+ uuid->data[8] = (unsigned char)(sequence_counter);
+ }
+ else
+ {
+ /* fill everything after the timestamp with random bytes */
+ if (!pg_strong_random(&uuid->data[6], UUID_LEN - 6))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Left-most counter bits are initialized as zero for the sole purpose
+ * of guarding against counter rollovers.
+ * See section "Fixed-Length Dedicated Counter Seeding"
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-09#monotonicity_counters
+ */
+ uuid->data[6] = (uuid->data[6] & 0xf7);
+
+ sequence_counter = ((uint32_t)uuid->data[8] & 0x3f) +
+ (((uint32_t)uuid->data[7]) << 6) +
+ (((uint32_t)uuid->data[6] & 0x0f) << 14);
+
+ previous_timestamp = tms;
+ }
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char)(tms >> 40);
+ uuid->data[1] = (unsigned char)(tms >> 32);
+ uuid->data[2] = (unsigned char)(tms >> 24);
+ uuid->data[3] = (unsigned char)(tms >> 16);
+ uuid->data[4] = (unsigned char)(tms >> 8);
+ uuid->data[5] = (unsigned char)tms;
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * http://tools.ietf.org/html/rfc ???
+ * https://datatracker.ietf.org/doc/html/draft-peabody-dispatch-new-uuid-format#name-creating-a-uuidv7-value
+ */
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+Datum
+uuid_extract_time(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ TimestampTz ts;
+ uint64_t tms;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+
+ if ((uuid->data[6] & 0xf0) == 0x70)
+ {
+ tms = uuid->data[5];
+ tms += ((uint64_t)uuid->data[4]) << 8;
+ tms += ((uint64_t)uuid->data[3]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 32;
+ tms += ((uint64_t)uuid->data[0]) << 40;
+
+ ts = (TimestampTz) (tms * 1000) - /* convert ms to us, than adjust */
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x10)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 8;
+ tms += ((uint64_t)uuid->data[3]);
+ tms += ((uint64_t)uuid->data[4]) << 40;
+ tms += ((uint64_t)uuid->data[5]) << 32;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 56;
+ tms += ((uint64_t)uuid->data[7]) << 48;
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - date2j(1582,10,15)) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x60)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 52;
+ tms += ((uint64_t)uuid->data[1]) << 44;
+ tms += ((uint64_t)uuid->data[2]) << 36;
+ tms += ((uint64_t)uuid->data[3]) << 28;
+ tms += ((uint64_t)uuid->data[4]) << 20;
+ tms += ((uint64_t)uuid->data[5]) << 12;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 8;
+ tms += ((uint64_t)uuid->data[7]);
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - date2j(1582,10,15)) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ PG_RETURN_NULL();
+}
+
+Datum
+uuid_extract_ver(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+ result = uuid->data[6] >> 4;
+
+ PG_RETURN_UINT16(result);
+}
+
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 58811a6530..956fb08ce9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9174,6 +9174,25 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate random UUID',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7', proisstrict => 'f',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'timestamptz', prosrc => 'uuidv7',
+ proargnames => '{unix_ts_ms}', proargmodes => '{i}' },
+{ oid => '9898', descr => 'extract timestamp from UUID version 7',
+ proname => 'uuid_extract_time', proleakproof => 't',
+ prorettype => 'timestamptz', proargtypes => 'uuid', prosrc => 'uuid_extract_time' },
+{ oid => '9899', descr => 'extract version from RFC 4122 UUID',
+ proname => 'uuid_extract_ver', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_ver' },
+{ oid => '9900', descr => 'extract variant from UUID',
+ proname => 'uuid_extract_var', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_var' },
# pg_lsn
{ oid => '3229', descr => 'I/O',
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7610b011d6..1c37533975 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -126,9 +126,10 @@ WHERE p1.oid < p2.oid AND
p1.proretset != p2.proretset OR
p1.provolatile != p2.provolatile OR
p1.pronargs != p2.pronargs);
- oid | proname | oid | proname
------+---------+-----+---------
-(0 rows)
+ oid | proname | oid | proname
+------+---------+------+---------
+ 9896 | uuidv7 | 9897 | uuidv7
+(1 row)
-- Look for uses of different type OIDs in the argument/result type fields
-- for different aliases of the same built-in function.
@@ -872,6 +873,12 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+uuidv4()
+uuidv7()
+uuidv7(timestamp with time zone)
+uuid_extract_time(uuid)
+uuid_extract_ver(uuid)
+uuid_extract_var(uuid)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8e7f21910d..b61f7a64ff 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,5 +168,93 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+ ?column?
+----------
+ @ 0
+(1 row)
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+ uuid_extract_ver
+------------------
+ 7
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+ uuid_extract_ver
+------------------
+ 5
+(1 row)
+
+SELECT uuid_extract_var(uuidv7());
+ uuid_extract_var
+------------------
+ 2
+(1 row)
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+ uuid_extract_time
+-------------------
+
+(1 row)
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') at time zone 'GMT-05';
+ timezone
+--------------------------
+ Wed Feb 23 00:22:22 2022
+(1 row)
+
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') at time zone 'GMT-05';
+ timezone
+--------------------------
+ Wed Feb 23 00:22:22 2022
+(1 row)
+
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') at time zone 'GMT-05';
+ timezone
+--------------------------
+ Wed Feb 23 00:22:22 2022
+(1 row)
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 9a8f437c7d..d7185759b1 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,5 +85,40 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+SELECT uuid_extract_var(uuidv7());
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') at time zone 'GMT-05';
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') at time zone 'GMT-05';
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') at time zone 'GMT-05';
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 13:17 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-18 20:26 ` Przemysław Sztoch <[email protected]>
1 sibling, 0 replies; 68+ messages in thread
From: Przemysław Sztoch @ 2024-01-18 20:26 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Using localtime would be absurd. Especially since time goes back during
summer time change.
I believe our implementation should use UTC. No one forbids us from
assuming that our local time for generating uuid is UTC.
Andrey Borodin wrote on 1/18/2024 2:17 PM:
>
>> On 17 Jan 2024, at 02:19, Jelte Fennema-Nio <[email protected]> wrote:
> I want to ask Kyzer or Brad, I hope they will see this message. I'm working on the patch for time extraction for v1, v6 and v7.
>
> Do I understand correctly, that UUIDs contain local time, not UTC time? For examples in [0] I see that "A.6. Example of a UUIDv7 Value" I see that February 22, 2022 2:22:22.00 PM GMT-05:00 results in unix_ts_ms = 0x017F22E279B0, which is not UTC, but local time.
> Is it intentional? Section "5.1. UUID Version 1" states otherwise.
>
> If so, I should swap signatures of functions from TimestampTz to Timestamp.
> I'm hard-coding examples from this standard to tests, so I want to be precise...
>
> If I follow the standard I see this in tests:
> +-- extract UUID v1, v6 and v7 timestamp
> +SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') at time zone 'GMT-05';
> + timezone
> +--------------------------
> + Wed Feb 23 00:22:22 2022
> +(1 row)
>
> Current patch version attached. I've addressed all other requests: function renames, aliases, multiple functions instead of optional params, cleaner catalog definitions, not throwing error when [var,ver,time] value is unknown.
> What is left: deal with timezones, improve documentation.
>
>
> Best regards, Andrey Borodin.
>
> [0] https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis#name-example-of-a-uuidv1-value
--
Przemysław Sztoch | Mobile +48 509 99 00 66
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 13:17 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-19 02:58 ` Lukas Fittl <[email protected]>
1 sibling, 0 replies; 68+ messages in thread
From: Lukas Fittl @ 2024-01-19 02:58 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
On Thu, Jan 18, 2024 at 5:18 AM Andrey Borodin <[email protected]> wrote:
> Current patch version attached. I've addressed all other requests:
> function renames, aliases, multiple functions instead of optional params,
> cleaner catalog definitions, not throwing error when [var,ver,time] value
> is unknown.
> What is left: deal with timezones, improve documentation.
>
I've done a test of the v10 patch, and ran into an interesting behavior
when passing in a timestamp to the function (which, as a side note, is
actually very useful to have as a feature, to support creating time-based
range partitions on UUIDv7 fields):
postgres=# SELECT uuid_extract_time(uuidv7());
uuid_extract_time
---------------------------
2024-01-18 18:49:00.01-08
(1 row)
postgres=# SELECT uuid_extract_time(uuidv7('2024-04-01'));
uuid_extract_time
------------------------
2024-04-01 00:00:00-07
(1 row)
postgres=# SELECT uuid_extract_time(uuidv7());
uuid_extract_time
------------------------
2024-04-01 00:00:00-07
(1 row)
Note how calling the uuidv7 function again after having called it with a
fixed future timestamp, returns the future timestamp, even though it should
return the current time.
I believe this is caused by incorrectly re-using the cached
previous_timestamp. In the second call here (with a fixed future
timestamp), we end up setting ts and tms to 2024-04-01, with
increment_counter = false, which leads us to set previous_timestamp to the
passed in timestamp (else branch of the second if in uuidv7). When we then
call the function again without an argument, we end up getting a new
timestamp from gettimeofday, but because we try to detect backwards leaps,
we set increment_counter to true, and thus end up reusing the previous
(future) timestamp here:
/* protection from leap backward */
tms = previous_timestamp;
Not sure how to fix this, but clearly something is amiss here.
Thanks,
Lukas
--
Lukas Fittl
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
@ 2024-01-18 14:20 ` Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 20:39 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
1 sibling, 2 replies; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-18 14:20 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Przemysław Sztoch <[email protected]>; Andrey Borodin <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> Another question: how did you choose between using TimestampTz and
> Timestamp types? I realize that internally it's all the same. Maybe
> Timestamp will be slightly better since the way it is displayed
> doesn't depend on the session settings. Many people I talked to find
> this part of TimestampTz confusing.
>
> timstamptz internally always store UTC.
> I believe that in SQL, when operating with time in UTC, you should always use timestamptz.
> timestamp is theoretically the same thing. But internally it does not convert time to UTC and will lead to incorrect use.
No.
Timestamp and TimestampTz are absolutely the same thing. The only
difference is how they are shown to the user. TimestampTz uses session
context in order to be displayed in the TZ chosen by the user. Thus
typically it is somewhat more confusing to the users and thus I asked
whether there was a good reason to choose TimestampTz over Timestamp.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-18 15:39 ` Andrey Borodin <[email protected]>
2024-01-18 16:21 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 20:49 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
1 sibling, 3 replies; 68+ messages in thread
From: Andrey Borodin @ 2024-01-18 15:39 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; +Cc: pgsql-hackers; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 18 Jan 2024, at 19:20, Aleksander Alekseev <[email protected]> wrote:
>
> Timestamp and TimestampTz are absolutely the same thing.
My question is not about Postgres data types. I'm asking about examples in the standard.
There's an example 017F22E2-79B0-7CC3-98C4-DC0C0C07398F. It is expected to be generated on "Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00".
It's exaplained to be 164555774200000ns after 1582-10-15 00:00:00 UTC.
But 164555774200000ns after 1582-10-15 00:00:00 UTC was 2022-02-22 19:22:22 UTC. And that was 2022-02-23 00:22:22 in UTC-05.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-18 16:21 ` Aleksander Alekseev <[email protected]>
2024-01-18 18:28 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2 siblings, 1 reply; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-18 16:21 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey Borodin <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi Andrey,
> > Timestamp and TimestampTz are absolutely the same thing.
> My question is not about Postgres data types. I'm asking about examples in the standard.
>
> There's an example 017F22E2-79B0-7CC3-98C4-DC0C0C07398F. It is expected to be generated on "Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00".
> It's exaplained to be 164555774200000ns after 1582-10-15 00:00:00 UTC.
>
> But 164555774200000ns after 1582-10-15 00:00:00 UTC was 2022-02-22 19:22:22 UTC. And that was 2022-02-23 00:22:22 in UTC-05.
Not 100% sure which text you are referring to exactly, but I'm
guessing it's section B.2 of [1]
"""
This example UUIDv7 test vector utilizes a well-known 32 bit Unix
epoch with additional millisecond precision to fill the first 48 bits
[...]
The timestamp is Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00
represented as 0x17F22E279B0 or 1645557742000
"""
If this is the case, I think the example is indeed wrong:
```
=# select extract(epoch from 'Tuesday, February 22, 2022 2:22:22.00 PM
GMT-05:00' :: timestamptz)*1000;
?column?
----------------------
1645521742000.000000
(1 row)
```
And the difference between the value in the text and the actual value
is 10 hours as you pointed out.
Also you named the date 1582-10-15 00:00:00 UTC. Maybe you actually
meant 1970-01-01 00:00:00 UTC?
[1]: https://www.ietf.org/archive/id/draft-peabody-dispatch-new-uuid-format-04.html
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 16:21 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-18 18:28 ` Sergey Prokhorenko <[email protected]>
0 siblings, 0 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-18 18:28 UTC (permalink / raw)
To: pgsql-hackers; Aleksander Alekseev <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Przemysław Sztoch <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi Andrey,
Aleksander Alekseev wrote: "If this is the case, I think the example is indeed wrong".
This is one of the reasons why I was categorically against any examples of implementation in the new RFC. The examples have been very poorly studied and discussed, and therefore it is better not to use them at all. But the text of the RFC itself clearly refers to UTC, and not at all about local time: "UUID version 7 features a time-ordered value field derived from the widely implemented and well known Unix Epoch timestamp source, the number of milliseconds since midnight 1 Jan 1970 UTC, leap seconds excluded". The main reason for using UTC is so that UUIDv7's, generated approximately simultaneously in different time zones, are correctly ordered in time when they get into one database.
Sergey Prokhorenko
[email protected]
On Thursday, 18 January 2024 at 07:22:05 pm GMT+3, Aleksander Alekseev <[email protected]> wrote:
Hi Andrey,
> > Timestamp and TimestampTz are absolutely the same thing.
> My question is not about Postgres data types. I'm asking about examples in the standard.
>
> There's an example 017F22E2-79B0-7CC3-98C4-DC0C0C07398F. It is expected to be generated on "Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00".
> It's exaplained to be 164555774200000ns after 1582-10-15 00:00:00 UTC.
>
> But 164555774200000ns after 1582-10-15 00:00:00 UTC was 2022-02-22 19:22:22 UTC. And that was 2022-02-23 00:22:22 in UTC-05.
Not 100% sure which text you are referring to exactly, but I'm
guessing it's section B.2 of [1]
"""
This example UUIDv7 test vector utilizes a well-known 32 bit Unix
epoch with additional millisecond precision to fill the first 48 bits
[...]
The timestamp is Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00
represented as 0x17F22E279B0 or 1645557742000
"""
If this is the case, I think the example is indeed wrong:
```
=# select extract(epoch from 'Tuesday, February 22, 2022 2:22:22.00 PM
GMT-05:00' :: timestamptz)*1000;
?column?
----------------------
1645521742000.000000
(1 row)
```
And the difference between the value in the text and the actual value
is 10 hours as you pointed out.
Also you named the date 1582-10-15 00:00:00 UTC. Maybe you actually
meant 1970-01-01 00:00:00 UTC?
[1]: https://www.ietf.org/archive/id/draft-peabody-dispatch-new-uuid-format-04.html
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-18 18:31 ` Andrey Borodin <[email protected]>
2024-01-18 18:59 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2 siblings, 2 replies; 68+ messages in thread
From: Andrey Borodin @ 2024-01-18 18:31 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; +Cc: pgsql-hackers; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 18 Jan 2024, at 20:39, Andrey Borodin <[email protected]> wrote:
>
> But 164555774200000ns after 1582-10-15 00:00:00 UTC was 2022-02-22 19:22:22 UTC. And that was 2022-02-23 00:22:22 in UTC-05.
'2022-02-22 19:22:22 UTC' is exactly that moment which was encoded into example UUIDs. It's not '2022-02-23 00:22:22 in UTC-05' as I thought.
I got confused by "at timezone" changes which in fact removes timezone information. And that's per SQL standard...
Now I'm completely lost in time... I've set local time to NY (UTC-5).
postgres=# select TIMESTAMP WITH TIME ZONE '2022-02-22 14:22:22-05' - TIMESTAMP WITH TIME ZONE 'Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00';
?column?
----------
10:00:00
(1 row)
postgres=# select TIMESTAMP WITH TIME ZONE 'Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00';
timestamptz
------------------------
2022-02-22 04:22:22-05
(1 row)
I cannot wrap my mind around it... Any pointers would be appreciated.
I'm certain that code extracted UTC time correctly, I just want a reliable test that verifies timestamp constant (+ I understand what is going on).
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-18 18:59 ` Sergey Prokhorenko <[email protected]>
1 sibling, 0 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-18 18:59 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; Andrey Borodin <[email protected]>; +Cc: pgsql-hackers; Przemysław Sztoch <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi Andrey,
You'd better generate a test UUIDv7 for midnight 1 Jan 1970 UTC. In this case, the timestamp in UUIDv7 according to the new RFC must be filled with zeros. By extracting the timestamp from this test UUIDv7, you should get exactly midnight 1 Jan 1970 UTC.
I also recommend this article: https://habr.com/ru/articles/772954/
Sergey Prokhorenko
[email protected]
On Thursday, 18 January 2024 at 09:31:16 pm GMT+3, Andrey Borodin <[email protected]> wrote:
> On 18 Jan 2024, at 20:39, Andrey Borodin <[email protected]> wrote:
>
> But 164555774200000ns after 1582-10-15 00:00:00 UTC was 2022-02-22 19:22:22 UTC. And that was 2022-02-23 00:22:22 in UTC-05.
'2022-02-22 19:22:22 UTC' is exactly that moment which was encoded into example UUIDs. It's not '2022-02-23 00:22:22 in UTC-05' as I thought.
I got confused by "at timezone" changes which in fact removes timezone information. And that's per SQL standard...
Now I'm completely lost in time... I've set local time to NY (UTC-5).
postgres=# select TIMESTAMP WITH TIME ZONE '2022-02-22 14:22:22-05' - TIMESTAMP WITH TIME ZONE 'Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00';
?column?
----------
10:00:00
(1 row)
postgres=# select TIMESTAMP WITH TIME ZONE 'Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00';
timestamptz
------------------------
2022-02-22 04:22:22-05
(1 row)
I cannot wrap my mind around it... Any pointers would be appreciated.
I'm certain that code extracted UTC time correctly, I just want a reliable test that verifies timestamp constant (+ I understand what is going on).
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-19 03:24 ` David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
1 sibling, 1 reply; 68+ messages in thread
From: David G. Johnston @ 2024-01-19 03:24 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
On Thu, Jan 18, 2024 at 11:31 AM Andrey Borodin <[email protected]>
wrote:
>
> Now I'm completely lost in time... I've set local time to NY (UTC-5).
>
> postgres=# select TIMESTAMP WITH TIME ZONE '2022-02-22 14:22:22-05' -
> TIMESTAMP WITH TIME ZONE 'Tuesday, February 22, 2022 2:22:22.00 PM
> GMT-05:00';
> ?column?
> ----------
> 10:00:00
> (1 row)
>
>
You are mixing POSIX and ISO-8601 conventions and, as noted in our
appendix, they disagree on the direction that is positive.
https://www.postgresql.org/docs/current/datetime-posix-timezone-specs.html
The offset fields specify the hours, and optionally minutes and seconds,
difference from UTC. They have the format hh[:mm[:ss]] optionally with a
leading sign (+ or -). The positive sign is used for zones west of
Greenwich. (Note that this is the opposite of the ISO-8601 sign convention
used elsewhere in PostgreSQL.)
David J.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
@ 2024-01-19 08:25 ` Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey Borodin @ 2024-01-19 08:25 UTC (permalink / raw)
To: David G. Johnston <[email protected]>; +Cc: Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 19 Jan 2024, at 08:24, David G. Johnston <[email protected]> wrote:
>
>
> You are mixing POSIX and ISO-8601 conventions and, as noted in our appendix, they disagree on the direction that is positive.
Thanks! Now everything seems on its place.
I want to include in the patch following tests:
-- extract UUID v1, v6 and v7 timestamp
SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
How do you think, will it be stable all across buildfarm? Or should we change anything to avoid false positives inferred from different timestamp parsing?
> On 19 Jan 2024, at 07:58, Lukas Fittl <[email protected]> wrote:
>
> Note how calling the uuidv7 function again after having called it with a fixed future timestamp, returns the future timestamp, even though it should return the current time.
Thanks for the review.
Well, that was intentional. But now I see it's kind of confusing behaviour. I've changed it to more expected version.
Also, I've added some documentation on all functions.
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v11-0001-Implement-UUID-v7-as-per-IETF-draft.patch (16.2K, ../../[email protected]/2-v11-0001-Implement-UUID-v7-as-per-IETF-draft.patch)
download | inline diff:
From 98726fc9b75d9abaec8bb4b305531e79d16bf6b0 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 20 Aug 2023 23:55:31 +0300
Subject: [PATCH v11] Implement UUID v7 as per IETF draft
This commit adds function to generate UUID v7.
This function optionally accepts datetime used to generate
next UUID.
Also we add a function to extract timestamp from UUID v7.
Authors: Andrey Borodin, Sergey Prokhorenko
---
doc/src/sgml/func.sgml | 36 ++++-
src/backend/utils/adt/uuid.c | 195 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 19 +++
src/test/regress/expected/opr_sanity.out | 13 +-
src/test/regress/expected/uuid.out | 88 ++++++++++
src/test/regress/sql/uuid.sql | 35 ++++
6 files changed, 380 insertions(+), 6 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 210c7c0b02..1d4d48d7cb 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14130,13 +14130,43 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_time</primary>
+ </indexterm>
+
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID:
+ <function>gen_random_uuid</function>, <function>uuidv4</function>, and <function>uuidv7</function>.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ Both functions return a version 4 (random) UUID. This is the most commonly
+ used type of UUID and is appropriate when random distribution of keys does
+ not affect performance of an application.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 (time-ordered + random) UUID. This UUID
+ version should be used when application prefers locality of identifiers.
+<synopsis>
+<function>uuid_extract_time</function> (uuid) <returnvalue>timestamptz</returnvalue>
+</synopsis>
+ This function extracts a timestamptz from UUID versions 1,6 and 7. For other
+ versions and variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_ver</function> (uuid) <returnvalue>int2</returnvalue>
+</synopsis>
+ This function extracts a version bits from UUID of variants described by
+ IETF standard (b10xx variant). For other variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_var</function> (uuid) <returnvalue>int2</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function extracts a vartiant bits from UUID.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 73dfd711c7..6125061b35 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,13 +13,18 @@
#include "postgres.h"
+#include <sys/time.h>
+
+#include "access/xlog.h"
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
+#include "utils/datetime.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
+#include "utils/timestamp.h"
#include "utils/uuid.h"
/* sortsupport for uuid */
@@ -421,3 +426,193 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+
+static uint32_t sequence_counter;
+static uint64_t previous_timestamp = 0;
+static bool external_times_used = false;
+
+
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ TimestampTz ts;
+ uint64_t tms;
+ struct timeval tp;
+ bool increment_counter;
+
+ if (PG_NARGS() == 0 || PG_ARGISNULL(0))
+ {
+ gettimeofday(&tp, NULL);
+ tms = ((uint64_t)tp.tv_sec) * 1000 + (tp.tv_usec) / 1000;
+ /* time from clock is protected from backward leaps */
+ increment_counter = (tms <= previous_timestamp) && !external_times_used;
+ external_times_used = false;
+ }
+ else
+ {
+ ts = PG_GETARG_TIMESTAMPTZ(0);
+ tms = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC) / 1000;
+ /*
+ * The time can leap backwards when provided by the user, so we use
+ * counter only when called with exactly same unix_ts_ms argument.
+ */
+ increment_counter = (tms == previous_timestamp);
+ external_times_used = true;
+ }
+
+ if (increment_counter)
+ {
+ /* Time did not increment from the previous generation, we must increment counter */
+ ++sequence_counter;
+ if (sequence_counter > 0x3ffff)
+ {
+ /* We only have 18-bit counter */
+ sequence_counter = 0;
+ previous_timestamp++;
+ }
+
+ /* protection from leap backward */
+ tms = previous_timestamp;
+
+ /* fill everything after the timestamp and counter with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /* most significant 4 bits of 18-bit counter */
+ uuid->data[6] = (unsigned char)(sequence_counter >> 14);
+ /* next 8 bits */
+ uuid->data[7] = (unsigned char)(sequence_counter >> 6);
+ /* least significant 6 bits */
+ uuid->data[8] = (unsigned char)(sequence_counter);
+ }
+ else
+ {
+ /* fill everything after the timestamp with random bytes */
+ if (!pg_strong_random(&uuid->data[6], UUID_LEN - 6))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Left-most counter bits are initialized as zero for the sole purpose
+ * of guarding against counter rollovers.
+ * See section "Fixed-Length Dedicated Counter Seeding"
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-09#monotonicity_counters
+ */
+ uuid->data[6] = (uuid->data[6] & 0xf7);
+
+ sequence_counter = ((uint32_t)uuid->data[8] & 0x3f) +
+ (((uint32_t)uuid->data[7]) << 6) +
+ (((uint32_t)uuid->data[6] & 0x0f) << 14);
+
+ previous_timestamp = tms;
+ }
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char)(tms >> 40);
+ uuid->data[1] = (unsigned char)(tms >> 32);
+ uuid->data[2] = (unsigned char)(tms >> 24);
+ uuid->data[3] = (unsigned char)(tms >> 16);
+ uuid->data[4] = (unsigned char)(tms >> 8);
+ uuid->data[5] = (unsigned char)tms;
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * http://tools.ietf.org/html/rfc ???
+ * https://datatracker.ietf.org/doc/html/draft-peabody-dispatch-new-uuid-format#name-creating-a-uuidv7-value
+ */
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+Datum
+uuid_extract_time(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ TimestampTz ts;
+ uint64_t tms;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+
+ if ((uuid->data[6] & 0xf0) == 0x70)
+ {
+ tms = uuid->data[5];
+ tms += ((uint64_t)uuid->data[4]) << 8;
+ tms += ((uint64_t)uuid->data[3]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 32;
+ tms += ((uint64_t)uuid->data[0]) << 40;
+
+ ts = (TimestampTz) (tms * 1000) - /* convert ms to us, than adjust */
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x10)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 8;
+ tms += ((uint64_t)uuid->data[3]);
+ tms += ((uint64_t)uuid->data[4]) << 40;
+ tms += ((uint64_t)uuid->data[5]) << 32;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 56;
+ tms += ((uint64_t)uuid->data[7]) << 48;
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - date2j(1582,10,15)) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x60)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 52;
+ tms += ((uint64_t)uuid->data[1]) << 44;
+ tms += ((uint64_t)uuid->data[2]) << 36;
+ tms += ((uint64_t)uuid->data[3]) << 28;
+ tms += ((uint64_t)uuid->data[4]) << 20;
+ tms += ((uint64_t)uuid->data[5]) << 12;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 8;
+ tms += ((uint64_t)uuid->data[7]);
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - date2j(1582,10,15)) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ PG_RETURN_NULL();
+}
+
+Datum
+uuid_extract_ver(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+ result = uuid->data[6] >> 4;
+
+ PG_RETURN_UINT16(result);
+}
+
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 58811a6530..956fb08ce9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9174,6 +9174,25 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate random UUID',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7', proisstrict => 'f',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'timestamptz', prosrc => 'uuidv7',
+ proargnames => '{unix_ts_ms}', proargmodes => '{i}' },
+{ oid => '9898', descr => 'extract timestamp from UUID version 7',
+ proname => 'uuid_extract_time', proleakproof => 't',
+ prorettype => 'timestamptz', proargtypes => 'uuid', prosrc => 'uuid_extract_time' },
+{ oid => '9899', descr => 'extract version from RFC 4122 UUID',
+ proname => 'uuid_extract_ver', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_ver' },
+{ oid => '9900', descr => 'extract variant from UUID',
+ proname => 'uuid_extract_var', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_var' },
# pg_lsn
{ oid => '3229', descr => 'I/O',
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7610b011d6..1c37533975 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -126,9 +126,10 @@ WHERE p1.oid < p2.oid AND
p1.proretset != p2.proretset OR
p1.provolatile != p2.provolatile OR
p1.pronargs != p2.pronargs);
- oid | proname | oid | proname
------+---------+-----+---------
-(0 rows)
+ oid | proname | oid | proname
+------+---------+------+---------
+ 9896 | uuidv7 | 9897 | uuidv7
+(1 row)
-- Look for uses of different type OIDs in the argument/result type fields
-- for different aliases of the same built-in function.
@@ -872,6 +873,12 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+uuidv4()
+uuidv7()
+uuidv7(timestamp with time zone)
+uuid_extract_time(uuid)
+uuid_extract_ver(uuid)
+uuid_extract_var(uuid)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8e7f21910d..df78fd0385 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,5 +168,93 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+ ?column?
+----------
+ @ 0
+(1 row)
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+ uuid_extract_ver
+------------------
+ 7
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+ uuid_extract_ver
+------------------
+ 5
+(1 row)
+
+SELECT uuid_extract_var(uuidv7());
+ uuid_extract_var
+------------------
+ 2
+(1 row)
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+ uuid_extract_time
+-------------------
+
+(1 row)
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 9a8f437c7d..c7a09dd21d 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,5 +85,40 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+SELECT uuid_extract_var(uuidv7());
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-19 18:07 ` Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey Borodin @ 2024-01-19 18:07 UTC (permalink / raw)
To: David G. Johnston <[email protected]>; +Cc: Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 19 Jan 2024, at 13:25, Andrey Borodin <[email protected]> wrote:
>
> Also, I've added some documentation on all functions.
Here's v12. Changes:
1. Documentation improvements
2. Code comments
3. Better commit message and reviews list
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v12-0001-Implement-UUID-v7.patch (17.8K, ../../[email protected]/2-v12-0001-Implement-UUID-v7.patch)
download | inline diff:
From 10401a50691e7d8c8416895b9862beb1444d2bfc Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 20 Aug 2023 23:55:31 +0300
Subject: [PATCH v12] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation.
Most important function here is uuidv7() which generates
new UUID according to new standard. This function can optionally
accept a timestamp used instead of current time. This allows
implementation of k-way sotable identifiers.
For code readability this commit adds alias uuidv4() to function
gen_random_uuid().
Also we add a function to extract timestamp from UUID v1, v6 and v7.
To allow user to distinguish various UUID versions and variants
we add functions uuid_extract_ver() and uuid_extract_var().
Author: Andrey Borodin
Reviewers: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewers: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewers: Peter Eisentraut, Chris Travers, Lukas Fittl
---
doc/src/sgml/func.sgml | 49 +++++-
src/backend/utils/adt/uuid.c | 195 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 19 +++
src/include/datatype/timestamp.h | 3 +-
src/test/regress/expected/opr_sanity.out | 13 +-
src/test/regress/expected/uuid.out | 88 ++++++++++
src/test/regress/sql/uuid.sql | 35 ++++
7 files changed, 395 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 210c7c0b02..ce6715721f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14130,13 +14130,56 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_time</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_ver</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_var</primary>
+ </indexterm>
+
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID:
+ <function>gen_random_uuid</function>, <function>uuidv4</function>, and <function>uuidv7</function>.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ Both functions return a version 4 (random) UUID. This is the most commonly
+ used type of UUID and is appropriate when random distribution of keys does
+ not affect performance of an application.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 (time-ordered + random) UUID. This UUID
+ version should be used when application prefers locality of identifiers.
+<synopsis>
+<function>uuid_extract_time</function> (uuid) <returnvalue>timestamptz</returnvalue>
+</synopsis>
+ This function extracts a timestamptz from UUID versions 1, 6 and 7. For other
+ versions and variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_ver</function> (uuid) <returnvalue>int2</returnvalue>
+</synopsis>
+ This function extracts a version bits from UUID of variant described by
+ <ulink url="https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis">IETF standard</ulink>
+ (b10xx variant). For other variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_var</function> (uuid) <returnvalue>int2</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function extracts a vartiant bits from UUID.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 73dfd711c7..386cdb7a73 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,13 +13,18 @@
#include "postgres.h"
+#include <sys/time.h>
+
+#include "access/xlog.h"
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
+#include "utils/datetime.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
+#include "utils/timestamp.h"
#include "utils/uuid.h"
/* sortsupport for uuid */
@@ -421,3 +426,193 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+
+static uint32_t sequence_counter;
+static uint64_t previous_timestamp = 0;
+static bool external_times_used = false;
+
+
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ TimestampTz ts;
+ uint64_t tms;
+ struct timeval tp;
+ bool increment_counter;
+
+ if (PG_NARGS() == 0 || PG_ARGISNULL(0))
+ {
+ gettimeofday(&tp, NULL);
+ tms = ((uint64_t)tp.tv_sec) * 1000 + (tp.tv_usec) / 1000;
+ /* time from clock is protected from backward leaps */
+ increment_counter = (tms <= previous_timestamp) && !external_times_used;
+ external_times_used = false;
+ }
+ else
+ {
+ ts = PG_GETARG_TIMESTAMPTZ(0);
+ tms = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC) / 1000;
+ /*
+ * The time can leap backwards when provided by the user, so we use
+ * counter only when called with exactly same unix_ts_ms argument.
+ */
+ increment_counter = (tms == previous_timestamp);
+ external_times_used = true;
+ }
+
+ if (increment_counter)
+ {
+ /* Time did not advance from the previous generation, we must increment counter */
+ ++sequence_counter;
+ if (sequence_counter > 0x3ffff)
+ {
+ /* We only have 18-bit counter */
+ sequence_counter = 0;
+ previous_timestamp++;
+ }
+
+ /* protection from leap backward */
+ tms = previous_timestamp;
+
+ /* fill everything after the timestamp and counter with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /* most significant 4 bits of 18-bit counter */
+ uuid->data[6] = (unsigned char)(sequence_counter >> 14);
+ /* next 8 bits */
+ uuid->data[7] = (unsigned char)(sequence_counter >> 6);
+ /* least significant 6 bits */
+ uuid->data[8] = (unsigned char)(sequence_counter);
+ }
+ else
+ {
+ /* fill everything after the timestamp with random bytes */
+ if (!pg_strong_random(&uuid->data[6], UUID_LEN - 6))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Left-most counter bits are initialized as zero for the sole purpose
+ * of guarding against counter rollovers.
+ * See section "Fixed-Length Dedicated Counter Seeding"
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-09#monotonicity_counters
+ */
+ uuid->data[6] = (uuid->data[6] & 0xf7);
+
+ /* read randomly initialized bits of counter */
+ sequence_counter = ((uint32_t)uuid->data[8] & 0x3f) +
+ (((uint32_t)uuid->data[7]) << 6) +
+ (((uint32_t)uuid->data[6] & 0x0f) << 14);
+
+ previous_timestamp = tms;
+ }
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char)(tms >> 40);
+ uuid->data[1] = (unsigned char)(tms >> 32);
+ uuid->data[2] = (unsigned char)(tms >> 24);
+ uuid->data[3] = (unsigned char)(tms >> 16);
+ uuid->data[4] = (unsigned char)(tms >> 8);
+ uuid->data[5] = (unsigned char)tms;
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis
+ */
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+Datum
+uuid_extract_time(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ TimestampTz ts;
+ uint64_t tms;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+
+ if ((uuid->data[6] & 0xf0) == 0x70)
+ {
+ tms = uuid->data[5];
+ tms += ((uint64_t)uuid->data[4]) << 8;
+ tms += ((uint64_t)uuid->data[3]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 32;
+ tms += ((uint64_t)uuid->data[0]) << 40;
+
+ ts = (TimestampTz) (tms * 1000) - /* convert ms to us, than adjust */
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x10)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 8;
+ tms += ((uint64_t)uuid->data[3]);
+ tms += ((uint64_t)uuid->data[4]) << 40;
+ tms += ((uint64_t)uuid->data[5]) << 32;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 56;
+ tms += ((uint64_t)uuid->data[7]) << 48;
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x60)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 52;
+ tms += ((uint64_t)uuid->data[1]) << 44;
+ tms += ((uint64_t)uuid->data[2]) << 36;
+ tms += ((uint64_t)uuid->data[3]) << 28;
+ tms += ((uint64_t)uuid->data[4]) << 20;
+ tms += ((uint64_t)uuid->data[5]) << 12;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 8;
+ tms += ((uint64_t)uuid->data[7]);
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ PG_RETURN_NULL();
+}
+
+Datum
+uuid_extract_ver(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+ result = uuid->data[6] >> 4;
+
+ PG_RETURN_UINT16(result);
+}
+
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 58811a6530..956fb08ce9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9174,6 +9174,25 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate random UUID',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7', proisstrict => 'f',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'timestamptz', prosrc => 'uuidv7',
+ proargnames => '{unix_ts_ms}', proargmodes => '{i}' },
+{ oid => '9898', descr => 'extract timestamp from UUID version 7',
+ proname => 'uuid_extract_time', proleakproof => 't',
+ prorettype => 'timestamptz', proargtypes => 'uuid', prosrc => 'uuid_extract_time' },
+{ oid => '9899', descr => 'extract version from RFC 4122 UUID',
+ proname => 'uuid_extract_ver', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_ver' },
+{ oid => '9900', descr => 'extract variant from UUID',
+ proname => 'uuid_extract_var', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_var' },
# pg_lsn
{ oid => '3229', descr => 'I/O',
diff --git a/src/include/datatype/timestamp.h b/src/include/datatype/timestamp.h
index 3a37cb661e..652aeb428e 100644
--- a/src/include/datatype/timestamp.h
+++ b/src/include/datatype/timestamp.h
@@ -230,9 +230,10 @@ struct pg_itm_in
((y) < JULIAN_MAXYEAR || \
((y) == JULIAN_MAXYEAR && ((m) < JULIAN_MAXMONTH))))
-/* Julian-date equivalents of Day 0 in Unix and Postgres reckoning */
+/* Julian-date equivalents of Day 0 in Unix, Postgres and Gregorian epochs */
#define UNIX_EPOCH_JDATE 2440588 /* == date2j(1970, 1, 1) */
#define POSTGRES_EPOCH_JDATE 2451545 /* == date2j(2000, 1, 1) */
+#define GREGORIAN_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
/*
* Range limits for dates and timestamps.
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7610b011d6..1c37533975 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -126,9 +126,10 @@ WHERE p1.oid < p2.oid AND
p1.proretset != p2.proretset OR
p1.provolatile != p2.provolatile OR
p1.pronargs != p2.pronargs);
- oid | proname | oid | proname
------+---------+-----+---------
-(0 rows)
+ oid | proname | oid | proname
+------+---------+------+---------
+ 9896 | uuidv7 | 9897 | uuidv7
+(1 row)
-- Look for uses of different type OIDs in the argument/result type fields
-- for different aliases of the same built-in function.
@@ -872,6 +873,12 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+uuidv4()
+uuidv7()
+uuidv7(timestamp with time zone)
+uuid_extract_time(uuid)
+uuid_extract_ver(uuid)
+uuid_extract_var(uuid)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8e7f21910d..df78fd0385 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,5 +168,93 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+ ?column?
+----------
+ @ 0
+(1 row)
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+ uuid_extract_ver
+------------------
+ 7
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+ uuid_extract_ver
+------------------
+ 5
+(1 row)
+
+SELECT uuid_extract_var(uuidv7());
+ uuid_extract_var
+------------------
+ 2
+(1 row)
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+ uuid_extract_time
+-------------------
+
+(1 row)
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 9a8f437c7d..c7a09dd21d 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,5 +85,40 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+SELECT uuid_extract_var(uuidv7());
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-22 04:22 ` Nikolay Samokhvalov <[email protected]>
2024-01-22 04:24 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
0 siblings, 3 replies; 68+ messages in thread
From: Nikolay Samokhvalov @ 2024-01-22 04:22 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: David G. Johnston <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
On Fri, Jan 19, 2024 at 10:07 AM Andrey Borodin <[email protected]>
wrote:
>
>
> > On 19 Jan 2024, at 13:25, Andrey Borodin <[email protected]> wrote:
> >
> > Also, I've added some documentation on all functions.
>
> Here's v12. Changes:
> 1. Documentation improvements
> 2. Code comments
> 3. Better commit message and reviews list
>
Thank you, Andrey! I have just checked v12 – cleanly applied to HEAD, and
functions work well. I especially like that fact that we keep
uuid_extract_time(..) here – this is a great thing to have for time-based
partitioning, and in many cases we will be able to decide not to have a
creation column timestamp (e.g., "created_at") at all, saving 8 bytes.
The docs and comments look great too.
Overall, the patch looks mature enough. It would be great to have it in
pg17. Yes, the RFC is not fully finalized yet, but it's very close. And
many libraries are already including implementation of UUIDv7 – here are
some examples:
- https://www.npmjs.com/package/uuidv7
- https://crates.io/crates/uuidv7
- https://github.com/google/uuid/pull/139
Nik
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
@ 2024-01-22 04:24 ` Nikolay Samokhvalov <[email protected]>
2 siblings, 0 replies; 68+ messages in thread
From: Nikolay Samokhvalov @ 2024-01-22 04:24 UTC (permalink / raw)
To: [email protected]; +Cc: Andrey Borodin <[email protected]>; Kirk Wolak <[email protected]>
The following review has been posted through the commitfest application:
make installcheck-world: not tested
Implements feature: tested, passed
Spec compliant: not tested
Documentation: tested, passed
Manually tested uuidv7(), uuid_extract_time() – they work as expected. The basic docs provided look clear.
I haven't checked the tests though and possible edge cases, so leaving it as "needs review" waiting for more reviewers
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
@ 2024-01-22 15:02 ` Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2 siblings, 1 reply; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-22 15:02 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> But now (after big timeseries project with multiple time zones and DST problems) I think differently.
> Even though timestamp and timestamptz are practically the same, timestamptz should be used to store the time in UTC.
> Choosing timestamp is more likely to lead to problems and misunderstandings than timestamptz.
As somebody who contributed TZ support to TimescaleDB I'm more or less
aware about the pros and cons of Timestamp and TimestampTz :)
Engineering is all about compromises. I can imagine a project where it
makes sense to use only TimestampTz for the entire database, and the
opposite - when it's better to use only UTC and Timestamp. In this
particular case I was merely concerned that the particular choice
could be confusing for the users but I think I changed my mind by now,
see below.
>> Here's v12. Changes:
>> 1. Documentation improvements
>> 2. Code comments
>> 3. Better commit message and reviews list
>
>
> Thank you, Andrey! I have just checked v12 – cleanly applied to HEAD, and functions work well. I especially like that fact that we keep uuid_extract_time(..) here – this is a great thing to have for time-based partitioning, and in many cases we will be able to decide not to have a creation column timestamp (e.g., "created_at") at all, saving 8 bytes.
>
> The docs and comments look great too.
>
> Overall, the patch looks mature enough. It would be great to have it in pg17. Yes, the RFC is not fully finalized yet, but it's very close. And many libraries are already including implementation of UUIDv7 – here are some examples:
>
> - https://www.npmjs.com/package/uuidv7
> - https://crates.io/crates/uuidv7
> - https://github.com/google/uuid/pull/139
Thanks!
After playing with v12 I'm inclined to agree that it's RfC.
I only have a couple of silly nitpicks:
- It could make sense to decompose the C implementation of uuidv7() in
two functions, for readability.
- It could make sense to get rid of curly braces in SQL tests when
calling uuid_extract_ver() and uuid_extract_ver(), for consistency.
I'm not going to insist on these changes though and prefer leaving it
to the author and the committer to decide.
Also I take back what I said above about using Timestamp instead of
TimestampTz. I forgot that Timestamps are implicitly casted to
TimestampTz's, so users preferring Timestamps can do this:
```
=# select uuidv7('2024-01-22 12:34:56' :: timestamp);
uuidv7
--------------------------------------
018d3085-de00-77c1-9e7b-7b04ddb9ebb9
```
Cfbot also seems to be happy with the patch so I'm changing the CF
entry status to RfC.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-24 12:31 ` Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-24 12:31 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> Cfbot also seems to be happy with the patch so I'm changing the CF
> entry status to RfC.
I've found a bug:
```
=# select now() - interval '5000 years';
?column?
----------------------------------------
2977-01-24 15:29:01.779462+02:30:17 BC
Time: 0.957 ms
=# select uuidv7(now() - interval '5000 years');
uuidv7
--------------------------------------
720c1868-0764-7677-99cd-265b84ea08b9
=# select uuid_extract_time('720c1868-0764-7677-99cd-265b84ea08b9');
uuid_extract_time
----------------------------
5943-08-26 21:30:44.836+03
```
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-24 12:40 ` Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey M. Borodin @ 2024-01-24 12:40 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; +Cc: pgsql-hackers; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 24 Jan 2024, at 17:31, Aleksander Alekseev <[email protected]> wrote:
>
> Hi,
>
>> Cfbot also seems to be happy with the patch so I'm changing the CF
>> entry status to RfC.
>
> I've found a bug:
>
> ```
> =# select now() - interval '5000 years';
> ?column?
> ----------------------------------------
> 2977-01-24 15:29:01.779462+02:30:17 BC
>
> Time: 0.957 ms
>
> =# select uuidv7(now() - interval '5000 years');
> uuidv7
> --------------------------------------
> 720c1868-0764-7677-99cd-265b84ea08b9
>
> =# select uuid_extract_time('720c1868-0764-7677-99cd-265b84ea08b9');
> uuid_extract_time
> ----------------------------
> 5943-08-26 21:30:44.836+03
> ```
UUIDv7 range does not correspond to timestamp range. But it’s purpose is not in storing timestamp, but in being unique identifier. So I don’t think it worth throwing an error when overflowing value is given. BTW if you will subtract some nanoseconds - you will not get back timestamp you put into UUID too.
UUID does not store timpestamp, it only uses it to generate an identifier. Some value can be extracted back, but with limited precision, limited range and only if UUID was generated precisely by the specification in standard (and standard allows deviation! Most of implementation try to tradeoff something).
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-24 13:02 ` Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
0 siblings, 2 replies; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-24 13:02 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey M. Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> UUIDv7 range does not correspond to timestamp range. But it’s purpose is not in storing timestamp, but in being unique identifier. So I don’t think it worth throwing an error when overflowing value is given. BTW if you will subtract some nanoseconds - you will not get back timestamp you put into UUID too.
> UUID does not store timpestamp, it only uses it to generate an identifier. Some value can be extracted back, but with limited precision, limited range and only if UUID was generated precisely by the specification in standard (and standard allows deviation! Most of implementation try to tradeoff something).
I don't claim that UUIDv7 purpose is storing timestamps, but I think
the invariant:
```
uuid_extract_time(uidv7(X)) == X
```
and (!) even more importantly:
```
if X > Y then uuidv7(X) > uuidv7(Y)
```
... should hold. Otherwise you can calculate crc64(X) or sha256(X)
internally in order to generate an unique ID and claim that it's fine.
Values that violate named invariants should be rejected with an error.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-24 13:16 ` Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
1 sibling, 1 reply; 68+ messages in thread
From: Andrey M. Borodin @ 2024-01-24 13:16 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; +Cc: pgsql-hackers; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 24 Jan 2024, at 18:02, Aleksander Alekseev <[email protected]> wrote:
>
> Hi,
>
>> UUIDv7 range does not correspond to timestamp range. But it’s purpose is not in storing timestamp, but in being unique identifier. So I don’t think it worth throwing an error when overflowing value is given. BTW if you will subtract some nanoseconds - you will not get back timestamp you put into UUID too.
>> UUID does not store timpestamp, it only uses it to generate an identifier. Some value can be extracted back, but with limited precision, limited range and only if UUID was generated precisely by the specification in standard (and standard allows deviation! Most of implementation try to tradeoff something).
>
> I don't claim that UUIDv7 purpose is storing timestamps, but I think
> the invariant:
>
> ```
> uuid_extract_time(uidv7(X)) == X
> ```
>
> and (!) even more importantly:
>
> ```
> if X > Y then uuidv7(X) > uuidv7(Y)
> ```
>
> ... should hold.
Function to extract timestamp does not provide any guarantees at all. Standard states this, see Kyzer answers upthread.
Moreover, standard urges against relying on that if uuidX was generated before uuidY, then uuidX<uuid. The standard is doing a lot to make this happen, but does not guaranty that.
All what is guaranteed is the uniqueness at certain conditions.
> Otherwise you can calculate crc64(X) or sha256(X)
> internally in order to generate an unique ID and claim that it's fine.
>
> Values that violate named invariants should be rejected with an error.
Think about the value that you pass to uuid generation function as an entropy. It’s there to ensure uniqueness and promote ordering (but not guarantee).
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-24 13:29 ` Aleksander Alekseev <[email protected]>
2024-01-24 15:29 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 21:30 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
0 siblings, 2 replies; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-24 13:29 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey M. Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> Function to extract timestamp does not provide any guarantees at all. Standard states this, see Kyzer answers upthread.
> Moreover, standard urges against relying on that if uuidX was generated before uuidY, then uuidX<uuid. The standard is doing a lot to make this happen, but does not guaranty that.
> All what is guaranteed is the uniqueness at certain conditions.
>
> > Otherwise you can calculate crc64(X) or sha256(X)
> > internally in order to generate an unique ID and claim that it's fine.
> >
> > Values that violate named invariants should be rejected with an error.
>
> Think about the value that you pass to uuid generation function as an entropy. It’s there to ensure uniqueness and promote ordering (but not guarantee).
If the standard doesn't guarantee something it doesn't mean it forbids
us to give stronger guarantees. I'm convinced that these guarantees
will be useful in real-world applications, at least the ones acting
exclusively within Postgres.
This being said, I understand your point of view too. Let's see what
other people think.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-24 15:29 ` Andrey M. Borodin <[email protected]>
2024-01-24 15:46 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
1 sibling, 1 reply; 68+ messages in thread
From: Andrey M. Borodin @ 2024-01-24 15:29 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; +Cc: pgsql-hackers; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 24 Jan 2024, at 18:29, Aleksander Alekseev <[email protected]> wrote:
>
> Hi,
>
>> Function to extract timestamp does not provide any guarantees at all. Standard states this, see Kyzer answers upthread.
>> Moreover, standard urges against relying on that if uuidX was generated before uuidY, then uuidX<uuid. The standard is doing a lot to make this happen, but does not guaranty that.
>> All what is guaranteed is the uniqueness at certain conditions.
>>
>>> Otherwise you can calculate crc64(X) or sha256(X)
>>> internally in order to generate an unique ID and claim that it's fine.
>>>
>>> Values that violate named invariants should be rejected with an error.
>>
>> Think about the value that you pass to uuid generation function as an entropy. It’s there to ensure uniqueness and promote ordering (but not guarantee).
>
> If the standard doesn't guarantee something it doesn't mean it forbids
> us to give stronger guarantees.
No, the standard makes these guarantees impossible.
If we insist that uuid_extract_time(uuidv7(time))==time, we won't be able to generate uuidv7 most of the time. uuidv7(now()) will always ERROR-out.
Standard implies more coarse-grained timestamp that we have.
Also, please not that uuidv7(time+1us) and uuidv7(time) will have the same internal timestamp, so despite time+1us > time, still second uuid will be greater.
Both invariants you proposed cannot be reasonably guaranteed. Upholding any of them greatly reduces usability of UUID v7.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 15:29 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-24 15:46 ` Aleksander Alekseev <[email protected]>
2024-01-24 16:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-24 15:46 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey M. Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> Also, please not that uuidv7(time+1us) and uuidv7(time) will have the same internal timestamp, so despite time+1us > time, still second uuid will be greater.
>
> Both invariants you proposed cannot be reasonably guaranteed. Upholding any of them greatly reduces usability of UUID v7.
Again, personally I don't insist on the 1us precision [1]. Only the
fact that timestamp from the far past generates UUID from the future
bothers me.
[1]: https://postgr.es/m/CAJ7c6TPCSprWwVNdOB%3D%3DpgKZPqO5q%3DHRgmU7zmYqz9Dz5ffVYw%40mail.gmail.com
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 15:29 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 15:46 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-24 16:54 ` Andrey M. Borodin <[email protected]>
2024-01-24 17:00 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey M. Borodin @ 2024-01-24 16:54 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; +Cc: pgsql-hackers; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 24 Jan 2024, at 20:46, Aleksander Alekseev <[email protected]> wrote:
>
> Only the
> fact that timestamp from the far past generates UUID from the future
> bothers me.
PFA implementation of guard checks, but I'm afraid that this can cause failures in ID generation unexpected to the user...
See tests
+-- errors in edge cases of UUID v7
+SELECT 1 FROM uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '0ms');
+SELECT uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '1ms'); -- ERROR expected
+SELECT 1 FROM uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000'));
+SELECT uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000')+'1ms'); -- ERROR expected
Range is from 1970-01-01 00:00:00 to 10889-08-02 05:31:50.655. I'm not sure we should give this information in error message...
Thanks!
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v13-0001-Implement-UUID-v7.patch (18.9K, ../../[email protected]/2-v13-0001-Implement-UUID-v7.patch)
download | inline diff:
From 5ae983c668ed5e17c0b34faefbc62b8200704bb7 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 20 Aug 2023 23:55:31 +0300
Subject: [PATCH v13] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation.
Most important function here is uuidv7() which generates
new UUID according to new standard. This function can optionally
accept a timestamp used instead of current time. This allows
implementation of k-way sotable identifiers.
For code readability this commit adds alias uuidv4() to function
gen_random_uuid().
Also we add a function to extract timestamp from UUID v1, v6 and v7.
To allow user to distinguish various UUID versions and variants
we add functions uuid_extract_ver() and uuid_extract_var().
Author: Andrey Borodin
Reviewers: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewers: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewers: Peter Eisentraut, Chris Travers, Lukas Fittl
---
doc/src/sgml/func.sgml | 49 +++++-
src/backend/utils/adt/uuid.c | 200 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 19 +++
src/include/datatype/timestamp.h | 3 +-
src/test/regress/expected/opr_sanity.out | 13 +-
src/test/regress/expected/uuid.out | 105 ++++++++++++
src/test/regress/sql/uuid.sql | 41 +++++
7 files changed, 423 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5030a1045f..ec09d06bd8 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14130,13 +14130,56 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_time</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_ver</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_var</primary>
+ </indexterm>
+
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID:
+ <function>gen_random_uuid</function>, <function>uuidv4</function>, and <function>uuidv7</function>.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ Both functions return a version 4 (random) UUID. This is the most commonly
+ used type of UUID and is appropriate when random distribution of keys does
+ not affect performance of an application.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 (time-ordered + random) UUID. This UUID
+ version should be used when application prefers locality of identifiers.
+<synopsis>
+<function>uuid_extract_time</function> (uuid) <returnvalue>timestamptz</returnvalue>
+</synopsis>
+ This function extracts a timestamptz from UUID versions 1, 6 and 7. For other
+ versions and variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_ver</function> (uuid) <returnvalue>int2</returnvalue>
+</synopsis>
+ This function extracts a version bits from UUID of variant described by
+ <ulink url="https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis">IETF standard</ulink>
+ (b10xx variant). For other variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_var</function> (uuid) <returnvalue>int2</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function extracts a vartiant bits from UUID.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 73dfd711c7..f88ae95710 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,13 +13,18 @@
#include "postgres.h"
+#include <sys/time.h>
+
+#include "access/xlog.h"
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
+#include "utils/datetime.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
+#include "utils/timestamp.h"
#include "utils/uuid.h"
/* sortsupport for uuid */
@@ -421,3 +426,198 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+
+static uint32_t sequence_counter;
+static uint64_t previous_timestamp = 0;
+static bool external_times_used = false;
+
+
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ TimestampTz ts;
+ uint64_t tms;
+ struct timeval tp;
+ bool increment_counter;
+
+ if (PG_NARGS() == 0 || PG_ARGISNULL(0))
+ {
+ gettimeofday(&tp, NULL);
+ tms = ((uint64_t)tp.tv_sec) * 1000 + (tp.tv_usec) / 1000;
+ /* time from clock is protected from backward leaps */
+ increment_counter = (tms <= previous_timestamp) && !external_times_used;
+ external_times_used = false;
+ }
+ else
+ {
+ ts = PG_GETARG_TIMESTAMPTZ(0);
+ tms = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC) / 1000;
+ /*
+ * The time can leap backwards when provided by the user, so we use
+ * counter only when called with exactly same unix_ts_ms argument.
+ */
+ increment_counter = (tms == previous_timestamp);
+ external_times_used = true;
+ if (tms & ~0xFFFFFFFFFFFF)
+ {
+ /* The standard allows only 6bytes of tms */
+ elog(ERROR, "Time argument of UUID v7 cannot exceed 6 bytes");
+ }
+ }
+
+ if (increment_counter)
+ {
+ /* Time did not advance from the previous generation, we must increment counter */
+ ++sequence_counter;
+ if (sequence_counter > 0x3ffff)
+ {
+ /* We only have 18-bit counter */
+ sequence_counter = 0;
+ previous_timestamp++;
+ }
+
+ /* protection from leap backward */
+ tms = previous_timestamp;
+
+ /* fill everything after the timestamp and counter with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /* most significant 4 bits of 18-bit counter */
+ uuid->data[6] = (unsigned char)(sequence_counter >> 14);
+ /* next 8 bits */
+ uuid->data[7] = (unsigned char)(sequence_counter >> 6);
+ /* least significant 6 bits */
+ uuid->data[8] = (unsigned char)(sequence_counter);
+ }
+ else
+ {
+ /* fill everything after the timestamp with random bytes */
+ if (!pg_strong_random(&uuid->data[6], UUID_LEN - 6))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Left-most counter bits are initialized as zero for the sole purpose
+ * of guarding against counter rollovers.
+ * See section "Fixed-Length Dedicated Counter Seeding"
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-09#monotonicity_counters
+ */
+ uuid->data[6] = (uuid->data[6] & 0xf7);
+
+ /* read randomly initialized bits of counter */
+ sequence_counter = ((uint32_t)uuid->data[8] & 0x3f) +
+ (((uint32_t)uuid->data[7]) << 6) +
+ (((uint32_t)uuid->data[6] & 0x0f) << 14);
+
+ previous_timestamp = tms;
+ }
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char)(tms >> 40);
+ uuid->data[1] = (unsigned char)(tms >> 32);
+ uuid->data[2] = (unsigned char)(tms >> 24);
+ uuid->data[3] = (unsigned char)(tms >> 16);
+ uuid->data[4] = (unsigned char)(tms >> 8);
+ uuid->data[5] = (unsigned char)tms;
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis
+ */
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+Datum
+uuid_extract_time(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ TimestampTz ts;
+ uint64_t tms;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+
+ if ((uuid->data[6] & 0xf0) == 0x70)
+ {
+ tms = uuid->data[5];
+ tms += ((uint64_t)uuid->data[4]) << 8;
+ tms += ((uint64_t)uuid->data[3]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 32;
+ tms += ((uint64_t)uuid->data[0]) << 40;
+
+ ts = (TimestampTz) (tms * 1000) - /* convert ms to us, than adjust */
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x10)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 8;
+ tms += ((uint64_t)uuid->data[3]);
+ tms += ((uint64_t)uuid->data[4]) << 40;
+ tms += ((uint64_t)uuid->data[5]) << 32;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 56;
+ tms += ((uint64_t)uuid->data[7]) << 48;
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x60)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 52;
+ tms += ((uint64_t)uuid->data[1]) << 44;
+ tms += ((uint64_t)uuid->data[2]) << 36;
+ tms += ((uint64_t)uuid->data[3]) << 28;
+ tms += ((uint64_t)uuid->data[4]) << 20;
+ tms += ((uint64_t)uuid->data[5]) << 12;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 8;
+ tms += ((uint64_t)uuid->data[7]);
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ PG_RETURN_NULL();
+}
+
+Datum
+uuid_extract_ver(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+ result = uuid->data[6] >> 4;
+
+ PG_RETURN_UINT16(result);
+}
+
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e4115cd084..16eabfbcbd 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9174,6 +9174,25 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate random UUID',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7', proisstrict => 'f',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'timestamptz', prosrc => 'uuidv7',
+ proargnames => '{unix_ts_ms}', proargmodes => '{i}' },
+{ oid => '9898', descr => 'extract timestamp from UUID version 7',
+ proname => 'uuid_extract_time', proleakproof => 't',
+ prorettype => 'timestamptz', proargtypes => 'uuid', prosrc => 'uuid_extract_time' },
+{ oid => '9899', descr => 'extract version from RFC 4122 UUID',
+ proname => 'uuid_extract_ver', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_ver' },
+{ oid => '9900', descr => 'extract variant from UUID',
+ proname => 'uuid_extract_var', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_var' },
# pg_lsn
{ oid => '3229', descr => 'I/O',
diff --git a/src/include/datatype/timestamp.h b/src/include/datatype/timestamp.h
index 3a37cb661e..652aeb428e 100644
--- a/src/include/datatype/timestamp.h
+++ b/src/include/datatype/timestamp.h
@@ -230,9 +230,10 @@ struct pg_itm_in
((y) < JULIAN_MAXYEAR || \
((y) == JULIAN_MAXYEAR && ((m) < JULIAN_MAXMONTH))))
-/* Julian-date equivalents of Day 0 in Unix and Postgres reckoning */
+/* Julian-date equivalents of Day 0 in Unix, Postgres and Gregorian epochs */
#define UNIX_EPOCH_JDATE 2440588 /* == date2j(1970, 1, 1) */
#define POSTGRES_EPOCH_JDATE 2451545 /* == date2j(2000, 1, 1) */
+#define GREGORIAN_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
/*
* Range limits for dates and timestamps.
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7610b011d6..1c37533975 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -126,9 +126,10 @@ WHERE p1.oid < p2.oid AND
p1.proretset != p2.proretset OR
p1.provolatile != p2.provolatile OR
p1.pronargs != p2.pronargs);
- oid | proname | oid | proname
------+---------+-----+---------
-(0 rows)
+ oid | proname | oid | proname
+------+---------+------+---------
+ 9896 | uuidv7 | 9897 | uuidv7
+(1 row)
-- Look for uses of different type OIDs in the argument/result type fields
-- for different aliases of the same built-in function.
@@ -872,6 +873,12 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+uuidv4()
+uuidv7()
+uuidv7(timestamp with time zone)
+uuid_extract_time(uuid)
+uuid_extract_ver(uuid)
+uuid_extract_var(uuid)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8e7f21910d..22488d5990 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,5 +168,110 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+ ?column?
+----------
+ @ 0
+(1 row)
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+ uuid_extract_ver
+------------------
+ 7
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+ uuid_extract_ver
+------------------
+ 5
+(1 row)
+
+SELECT uuid_extract_var(uuidv7());
+ uuid_extract_var
+------------------
+ 2
+(1 row)
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+ uuid_extract_time
+-------------------
+
+(1 row)
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+-- errors in edge cases of UUID v7
+SELECT 1 FROM uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '0ms');
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '1ms'); -- ERROR expected
+ERROR: Time argument of UUID v7 cannot exceed 6 bytes
+SELECT 1 FROM uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000'));
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000')+'1ms'); -- ERROR expected
+ERROR: Time argument of UUID v7 cannot exceed 6 bytes
-- clean up
DROP TABLE guid1, guid2 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 9a8f437c7d..40c3152697 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,5 +85,46 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+SELECT uuid_extract_var(uuidv7());
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+
+-- errors in edge cases of UUID v7
+SELECT 1 FROM uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '0ms');
+SELECT uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '1ms'); -- ERROR expected
+SELECT 1 FROM uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000'));
+SELECT uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000')+'1ms'); -- ERROR expected
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
--
2.37.1 (Apple Git-137.1)
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 15:29 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 15:46 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 16:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-24 17:00 ` Marcos Pegoraro <[email protected]>
2024-01-24 17:51 ` Re: UUID v7 Andrey Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Marcos Pegoraro @ 2024-01-24 17:00 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Aleksander Alekseev <[email protected]>; pgsql-hackers; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Is enough from 1970 ?
How about if user wants to have an UUID of his birth date ?
regards
Marcos
Em qua., 24 de jan. de 2024 às 13:54, Andrey M. Borodin <
[email protected]> escreveu:
>
>
> > On 24 Jan 2024, at 20:46, Aleksander Alekseev <[email protected]>
> wrote:
> >
> > Only the
> > fact that timestamp from the far past generates UUID from the future
> > bothers me.
>
> PFA implementation of guard checks, but I'm afraid that this can cause
> failures in ID generation unexpected to the user...
> See tests
>
> +-- errors in edge cases of UUID v7
> +SELECT 1 FROM uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval
> '0ms');
> +SELECT uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '1ms'); --
> ERROR expected
> +SELECT 1 FROM
> uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000'));
> +SELECT
> uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000')+'1ms'); --
> ERROR expected
>
> Range is from 1970-01-01 00:00:00 to 10889-08-02 05:31:50.655. I'm not
> sure we should give this information in error message...
> Thanks!
>
>
> Best regards, Andrey Borodin.
>
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 15:29 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 15:46 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 16:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 17:00 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
@ 2024-01-24 17:51 ` Andrey Borodin <[email protected]>
2024-01-24 20:47 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey Borodin @ 2024-01-24 17:51 UTC (permalink / raw)
To: Marcos Pegoraro <[email protected]>; +Cc: Aleksander Alekseev <[email protected]>; pgsql-hackers; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 24 Jan 2024, at 22:00, Marcos Pegoraro <[email protected]> wrote:
>
> Is enough from 1970 ?
Per standard unix_ts_ms field is a number of milliseconds from UNIX start date 1970-01-01.
> How about if user wants to have an UUID of his birth date ?
I've claimed my
0078c135-bd00-70b1-865a-63c3741922a5
But again, UUIDs are not designed to store timestamp. They are unique and v7 promote data locality via time-ordering.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 15:29 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 15:46 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 16:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 17:00 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
2024-01-24 17:51 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-24 20:47 ` Marcos Pegoraro <[email protected]>
2024-01-24 21:15 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Marcos Pegoraro @ 2024-01-24 20:47 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Aleksander Alekseev <[email protected]>; pgsql-hackers; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
I understand your point, but
'2000-01-01' :: timestamp and '1900-01-01' :: timestamp are both valid
timestamps.
So looks strange if user can do
select uuidv7(TIMESTAMP '2000-01-01')
but cannot do
select uuidv7(TIMESTAMP '1900-01-01')
Regards
Marcos
Em qua., 24 de jan. de 2024 às 14:51, Andrey Borodin <[email protected]>
escreveu:
>
>
> > On 24 Jan 2024, at 22:00, Marcos Pegoraro <[email protected]> wrote:
> >
> > Is enough from 1970 ?
> Per standard unix_ts_ms field is a number of milliseconds from UNIX start
> date 1970-01-01.
>
> > How about if user wants to have an UUID of his birth date ?
>
> I've claimed my
> 0078c135-bd00-70b1-865a-63c3741922a5
>
> But again, UUIDs are not designed to store timestamp. They are unique and
> v7 promote data locality via time-ordering.
>
>
> Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 15:29 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 15:46 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 16:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 17:00 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
2024-01-24 17:51 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-24 20:47 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
@ 2024-01-24 21:15 ` Jelte Fennema-Nio <[email protected]>
0 siblings, 0 replies; 68+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-24 21:15 UTC (permalink / raw)
To: Marcos Pegoraro <[email protected]>; +Cc: Andrey Borodin <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
On Wed, 24 Jan 2024 at 21:47, Marcos Pegoraro <[email protected]> wrote:
>
> I understand your point, but
> '2000-01-01' :: timestamp and '1900-01-01' :: timestamp are both valid timestamps.
>
> So looks strange if user can do
> select uuidv7(TIMESTAMP '2000-01-01')
> but cannot do
> select uuidv7(TIMESTAMP '1900-01-01')
I think that would be okay honestly. I don't think there's any
reasonable value for the uuid when a timestamp is given outside of the
date range that the uuid7 "algorithm" supports.
So +1 for erroring when you provide a timestamp outside of that range
(either too far in the past or too far in the future).
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-24 21:30 ` Sergey Prokhorenko <[email protected]>
1 sibling, 0 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-24 21:30 UTC (permalink / raw)
To: pgsql-hackers; Aleksander Alekseev <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
"Other people" think that extracting the timestamp from UUIDv7 in violation of the new RFC, and generating UUIDv7 from the timestamp were both terrible and poorly thought out ideas. The authors of the new RFC had very good reasons to prohibit this. And the problems you face are the best confirmation of the correctness of the new RFC. It’s better to throw all this gag out of the official patch. Don't tempt developers to break the new RFC with these error-producing functions.
Sergey [email protected]
On Wednesday, 24 January 2024 at 04:30:02 pm GMT+3, Aleksander Alekseev <[email protected]> wrote:
Hi,
> Function to extract timestamp does not provide any guarantees at all. Standard states this, see Kyzer answers upthread.
> Moreover, standard urges against relying on that if uuidX was generated before uuidY, then uuidX<uuid. The standard is doing a lot to make this happen, but does not guaranty that.
> All what is guaranteed is the uniqueness at certain conditions.
>
> > Otherwise you can calculate crc64(X) or sha256(X)
> > internally in order to generate an unique ID and claim that it's fine.
> >
> > Values that violate named invariants should be rejected with an error.
>
> Think about the value that you pass to uuid generation function as an entropy. It’s there to ensure uniqueness and promote ordering (but not guarantee).
If the standard doesn't guarantee something it doesn't mean it forbids
us to give stronger guarantees. I'm convinced that these guarantees
will be useful in real-world applications, at least the ones acting
exclusively within Postgres.
This being said, I understand your point of view too. Let's see what
other people think.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-24 13:16 ` Aleksander Alekseev <[email protected]>
1 sibling, 0 replies; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-24 13:16 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey M. Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Przemysław Sztoch <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> Values that violate named invariants should be rejected with an error.
To clarify, I don't think we should bother about the precision part.
"Equals" in the example above means "equal within UUIDv7 precision",
same for "more" and "less". However, years 2977 BC and 5943 AC are
clearly not equal, thus 2977 BC should be rejected as an invalid value
for UUIDv7.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
@ 2024-01-24 21:49 ` Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-28 12:42 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2 siblings, 2 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-24 21:49 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; +Cc: David G. Johnston <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
That's right! There is no point in waiting for the official approval of the new RFC, which obviously will not change anything. I have been a contributor to this RFC for several years, and I can testify that every aspect imaginable has been thoroughly researched and agreed upon. Nothing new will definitely appear in the new RFC.
Sergey [email protected]
On Monday, 22 January 2024 at 07:22:32 am GMT+3, Nikolay Samokhvalov <[email protected]> wrote:
On Fri, Jan 19, 2024 at 10:07 AM Andrey Borodin <[email protected]> wrote:
> On 19 Jan 2024, at 13:25, Andrey Borodin <[email protected]> wrote:
>
> Also, I've added some documentation on all functions.
Here's v12. Changes:
1. Documentation improvements
2. Code comments
3. Better commit message and reviews list
Thank you, Andrey! I have just checked v12 – cleanly applied to HEAD, and functions work well. I especially like that fact that we keep uuid_extract_time(..) here – this is a great thing to have for time-based partitioning, and in many cases we will be able to decide not to have a creation column timestamp (e.g., "created_at") at all, saving 8 bytes.
The docs and comments look great too.
Overall, the patch looks mature enough. It would be great to have it in pg17. Yes, the RFC is not fully finalized yet, but it's very close. And many libraries are already including implementation of UUIDv7 – here are some examples:
- https://www.npmjs.com/package/uuidv7
- https://crates.io/crates/uuidv7
- https://github.com/google/uuid/pull/139
Nik
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
@ 2024-01-25 04:40 ` Nikolay Samokhvalov <[email protected]>
2024-01-25 04:41 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 08:09 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
1 sibling, 3 replies; 68+ messages in thread
From: Nikolay Samokhvalov @ 2024-01-25 04:40 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Andrey Borodin <[email protected]>; David G. Johnston <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
--0000000000002792f3060fbdc8f7
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On Wed, Jan 24, 2024 at 1:52=E2=80=AFPM Sergey Prokhorenko <
[email protected]> wrote:
> That's right! There is no point in waiting for the official approval of
> the new RFC, which obviously will not change anything. I have been a
> contributor to this RFC
> <https://www.ietf.org/archive/id/draft-ietf-uuidrev-rfc4122bis-14.html#na=
me-acknowledgements>
> for several years, and I can testify that every aspect imaginable has bee=
n
> thoroughly researched and agreed upon. Nothing new will definitely appear
> in the new RFC.
>
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
@ 2024-01-25 04:41 ` Nikolay Samokhvalov <[email protected]>
2 siblings, 0 replies; 68+ messages in thread
From: Nikolay Samokhvalov @ 2024-01-25 04:41 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Andrey Borodin <[email protected]>; David G. Johnston <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
On Wed, Jan 24, 2024 at 8:40 PM Nikolay Samokhvalov <[email protected]> wrote:
> On Wed, Jan 24, 2024 at 1:52 PM Sergey Prokhorenko <
> [email protected]> wrote:
>
>> That's right! There is no point in waiting for the official approval of
>> the new RFC, which obviously will not change anything. I have been a
>> contributor to this RFC
>> <https://www.ietf.org/archive/id/draft-ietf-uuidrev-rfc4122bis-14.html#name-acknowledgements;
>> for several years, and I can testify that every aspect imaginable has been
>> thoroughly researched and agreed upon. Nothing new will definitely
>> appear in the new RFC.
>>
>
> From a practical point of view, these two things are extremely important
> to have to support partitioning. It is better to implement limitations than
> throw them away.
>
> Without them, this functionality will be of a very limited use in
> databases. We need to think about large tables – which means partitioning.
>
apologies -- this was a response to another email from you:
> "Other people" think that extracting the timestamp from UUIDv7 in
violation of the new RFC, and generating UUIDv7 from the timestamp were
both terrible and poorly thought out ideas. The authors of the new RFC had
very good reasons to prohibit this. And the problems you face are the best
confirmation of the correctness of the new RFC. It’s better to throw all
this gag out of the official patch. Don't tempt developers to break the new
RFC with these error-producing functions.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
@ 2024-01-25 08:09 ` Sergey Prokhorenko <[email protected]>
2 siblings, 0 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-25 08:09 UTC (permalink / raw)
To: Nikolay Samokhvalov <[email protected]>; Andrey M. Borodin <[email protected]>; +Cc: David G. Johnston <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
I am against turning the DBMS into another C++, in which they do not so much design something new as fix bugs in production after a crash.
As for partitioning, I already wrote to Andrey Borodin that we need a special function to generate a partition id using the UUIDv7 timestamp or even simultaneously with the generation of the timestamp. For example, every month (or so, since precision is not needed here) a new partition is created. Here's a good example: https://elixirforum.com/t/partitioning-postgres-tables-by-timestamp-based-uuids/60916
But without a separate function for extracting the entire timestamp from the UUID! Let's solve this specific problem, and not give the developers a grenade with the safety removed. Many developers have already decided to store the timestamp in UUIDv7, so as not to create a separate created_at field. Then they will delete table records with the old timestamp, etc. Horrible mistakes are simply guaranteed.
Sergey Prokhorenko [email protected]
On Thursday, 25 January 2024 at 09:51:58 am GMT+3, Andrey M. Borodin <[email protected]> wrote:
> On 25 Jan 2024, at 09:40, Nikolay Samokhvalov <[email protected]> wrote:
>
> From a practical point of view, these two things are extremely important to have to support partitioning. It is better to implement limitations than throw them away.
Postgres always was a bit hackerish, allowing slightly more then is safe. I.e. you can define immutable function that is not really immutable, turn off autovacuum or fsync. Why bother with safety guards here?
My opinion is that we should have this function to extract timestamp. Even if it can return strange values for imprecise RFC implementation.
> On 25 Jan 2024, at 02:15, Jelte Fennema-Nio <[email protected]> wrote:
>
> So +1 for erroring when you provide a timestamp outside of that range
> (either too far in the past or too far in the future).
OK, it seems like we have some consensus on ERRORing..
Do we have any other open items? Does v13 address all open items? Maybe let’s compose better error message?
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
@ 2024-01-25 11:14 ` Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2 siblings, 1 reply; 68+ messages in thread
From: Przemysław Sztoch @ 2024-01-25 11:14 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Nikolay Samokhvalov <[email protected]>; Sergey Prokhorenko <[email protected]>; David G. Johnston <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Andrey M. Borodin wrote on 25.01.2024 07:51:
>
>> On 25 Jan 2024, at 09:40, Nikolay Samokhvalov <[email protected]> wrote:
>>
>> From a practical point of view, these two things are extremely important to have to support partitioning. It is better to implement limitations than throw them away.
> Postgres always was a bit hackerish, allowing slightly more then is safe. I.e. you can define immutable function that is not really immutable, turn off autovacuum or fsync. Why bother with safety guards here?
> My opinion is that we should have this function to extract timestamp. Even if it can return strange values for imprecise RFC implementation.
>
>
>> On 25 Jan 2024, at 02:15, Jelte Fennema-Nio <[email protected]> wrote:
>>
>> So +1 for erroring when you provide a timestamp outside of that range
>> (either too far in the past or too far in the future).
>
> OK, it seems like we have some consensus on ERRORing..
>
> Do we have any other open items? Does v13 address all open items? Maybe let’s compose better error message?
+1 for erroring when ts is outside range.
v13 looks good for me. I think we have reached a optimal compromise.
--
Przemysław Sztoch | Mobile +48 509 99 00 66
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
@ 2024-01-25 12:06 ` Aleksander Alekseev <[email protected]>
2024-01-25 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
0 siblings, 2 replies; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-25 12:06 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey M. Borodin <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; Sergey Prokhorenko <[email protected]>; David G. Johnston <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> Postgres always was a bit hackerish, allowing slightly more then is safe. I.e. you can define immutable function that is not really immutable, turn off autovacuum or fsync. Why bother with safety guards here?
> My opinion is that we should have this function to extract timestamp. Even if it can return strange values for imprecise RFC implementation.
Completely agree.
Users that don't like or don't need it can pretend there are no
uuid_extract_time() and uuidv7(T) in Postgres. If we don't provide
them however, users that need them will end up writing their own
probably buggy and not compatible implementations. That would be much
worse.
> So +1 for erroring when you provide a timestamp outside of that range
> (either too far in the past or too far in the future).
>
> OK, it seems like we have some consensus on ERRORing..
>
> Do we have any other open items? Does v13 address all open items? Maybe let’s compose better error message?
>
> +1 for erroring when ts is outside range.
>
> v13 looks good for me. I think we have reached a optimal compromise.
Andrey, many thanks for the updated patch.
LGTM, cfbot is happy and I don't think we have any open items left. So
changing CF entry status back to RfC.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-25 12:31 ` Aleksander Alekseev <[email protected]>
2024-01-29 12:01 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
1 sibling, 1 reply; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-25 12:31 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Andrey M. Borodin <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; Sergey Prokhorenko <[email protected]>; David G. Johnston <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>; Alexander Lakhin <[email protected]>
Hi,
> Andrey, many thanks for the updated patch.
>
> LGTM, cfbot is happy and I don't think we have any open items left. So
> changing CF entry status back to RfC.
PFA v14. I changed:
```
elog(ERROR, "Time argument of UUID v7 cannot exceed 6 bytes");
```
... to:
```
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("Time argument of UUID v7 is outside of the valid range")));
```
Which IMO tells a bit more to the average user and is translatable.
> At a quick glance, the patch needs improving English, IMO.
Agree. We could use some help from a native English speaker for this.
--
Best regards,
Aleksander Alekseev
Attachments:
[application/octet-stream] v14-0001-Implement-UUID-v7.patch (19.1K, ../../CAJ7c6TPi8o7+Kn=WtGArDnEfnm=cnrZRfbM-ZDBhyAomuBhr3A@mail.gmail.com/2-v14-0001-Implement-UUID-v7.patch)
download | inline diff:
From 874d7653f0345a93db3b6b8d954061d073d37915 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 20 Aug 2023 23:55:31 +0300
Subject: [PATCH v14] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
This function can optionally accept a timestamp used instead of current time.
This allows implementation of k-way sotable identifiers. For code readability
this commit adds alias uuidv4() to function gen_random_uuid().
Also we add a function to extract timestamp from UUID v1, v6 and v7.
To allow user to distinguish various UUID versions and variants
we add functions uuid_extract_ver() and uuid_extract_var().
Author: Andrey Borodin
Reviewers: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewers: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewers: Peter Eisentraut, Chris Travers, Lukas Fittl
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/func.sgml | 49 +++++-
src/backend/utils/adt/uuid.c | 202 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 19 +++
src/include/datatype/timestamp.h | 3 +-
src/test/regress/expected/opr_sanity.out | 13 +-
src/test/regress/expected/uuid.out | 105 ++++++++++++
src/test/regress/sql/uuid.sql | 41 +++++
7 files changed, 425 insertions(+), 7 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 968e8d59fb..74fbb982ab 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14130,13 +14130,56 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_time</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_ver</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_var</primary>
+ </indexterm>
+
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID:
+ <function>gen_random_uuid</function>, <function>uuidv4</function>, and <function>uuidv7</function>.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ Both functions return a version 4 (random) UUID. This is the most commonly
+ used type of UUID and is appropriate when random distribution of keys does
+ not affect performance of an application.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 (time-ordered + random) UUID. This UUID
+ version should be used when application prefers locality of identifiers.
+<synopsis>
+<function>uuid_extract_time</function> (uuid) <returnvalue>timestamptz</returnvalue>
+</synopsis>
+ This function extracts a timestamptz from UUID versions 1, 6 and 7. For other
+ versions and variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_ver</function> (uuid) <returnvalue>int2</returnvalue>
+</synopsis>
+ This function extracts a version bits from UUID of variant described by
+ <ulink url="https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis">IETF standard</ulink>
+ (b10xx variant). For other variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_var</function> (uuid) <returnvalue>int2</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function extracts a vartiant bits from UUID.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 73dfd711c7..665e27f498 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,13 +13,18 @@
#include "postgres.h"
+#include <sys/time.h>
+
+#include "access/xlog.h"
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
+#include "utils/datetime.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
+#include "utils/timestamp.h"
#include "utils/uuid.h"
/* sortsupport for uuid */
@@ -421,3 +426,200 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+
+static uint32_t sequence_counter;
+static uint64_t previous_timestamp = 0;
+static bool external_times_used = false;
+
+
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ TimestampTz ts;
+ uint64_t tms;
+ struct timeval tp;
+ bool increment_counter;
+
+ if (PG_NARGS() == 0 || PG_ARGISNULL(0))
+ {
+ gettimeofday(&tp, NULL);
+ tms = ((uint64_t)tp.tv_sec) * 1000 + (tp.tv_usec) / 1000;
+ /* time from clock is protected from backward leaps */
+ increment_counter = (tms <= previous_timestamp) && !external_times_used;
+ external_times_used = false;
+ }
+ else
+ {
+ ts = PG_GETARG_TIMESTAMPTZ(0);
+ tms = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC) / 1000;
+ /*
+ * The time can leap backwards when provided by the user, so we use
+ * counter only when called with exactly same unix_ts_ms argument.
+ */
+ increment_counter = (tms == previous_timestamp);
+ external_times_used = true;
+ if (tms & ~0xFFFFFFFFFFFF)
+ {
+ /* The standard allows only 6bytes of tms */
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("Time argument of UUID v7 is outside of the valid range")));
+ }
+ }
+
+ if (increment_counter)
+ {
+ /* Time did not advance from the previous generation, we must increment counter */
+ ++sequence_counter;
+ if (sequence_counter > 0x3ffff)
+ {
+ /* We only have 18-bit counter */
+ sequence_counter = 0;
+ previous_timestamp++;
+ }
+
+ /* protection from leap backward */
+ tms = previous_timestamp;
+
+ /* fill everything after the timestamp and counter with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /* most significant 4 bits of 18-bit counter */
+ uuid->data[6] = (unsigned char)(sequence_counter >> 14);
+ /* next 8 bits */
+ uuid->data[7] = (unsigned char)(sequence_counter >> 6);
+ /* least significant 6 bits */
+ uuid->data[8] = (unsigned char)(sequence_counter);
+ }
+ else
+ {
+ /* fill everything after the timestamp with random bytes */
+ if (!pg_strong_random(&uuid->data[6], UUID_LEN - 6))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Left-most counter bits are initialized as zero for the sole purpose
+ * of guarding against counter rollovers.
+ * See section "Fixed-Length Dedicated Counter Seeding"
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-09#monotonicity_counters
+ */
+ uuid->data[6] = (uuid->data[6] & 0xf7);
+
+ /* read randomly initialized bits of counter */
+ sequence_counter = ((uint32_t)uuid->data[8] & 0x3f) +
+ (((uint32_t)uuid->data[7]) << 6) +
+ (((uint32_t)uuid->data[6] & 0x0f) << 14);
+
+ previous_timestamp = tms;
+ }
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char)(tms >> 40);
+ uuid->data[1] = (unsigned char)(tms >> 32);
+ uuid->data[2] = (unsigned char)(tms >> 24);
+ uuid->data[3] = (unsigned char)(tms >> 16);
+ uuid->data[4] = (unsigned char)(tms >> 8);
+ uuid->data[5] = (unsigned char)tms;
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis
+ */
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+Datum
+uuid_extract_time(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ TimestampTz ts;
+ uint64_t tms;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+
+ if ((uuid->data[6] & 0xf0) == 0x70)
+ {
+ tms = uuid->data[5];
+ tms += ((uint64_t)uuid->data[4]) << 8;
+ tms += ((uint64_t)uuid->data[3]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 32;
+ tms += ((uint64_t)uuid->data[0]) << 40;
+
+ ts = (TimestampTz) (tms * 1000) - /* convert ms to us, than adjust */
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x10)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 8;
+ tms += ((uint64_t)uuid->data[3]);
+ tms += ((uint64_t)uuid->data[4]) << 40;
+ tms += ((uint64_t)uuid->data[5]) << 32;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 56;
+ tms += ((uint64_t)uuid->data[7]) << 48;
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x60)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 52;
+ tms += ((uint64_t)uuid->data[1]) << 44;
+ tms += ((uint64_t)uuid->data[2]) << 36;
+ tms += ((uint64_t)uuid->data[3]) << 28;
+ tms += ((uint64_t)uuid->data[4]) << 20;
+ tms += ((uint64_t)uuid->data[5]) << 12;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 8;
+ tms += ((uint64_t)uuid->data[7]);
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ PG_RETURN_NULL();
+}
+
+Datum
+uuid_extract_ver(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+ result = uuid->data[6] >> 4;
+
+ PG_RETURN_UINT16(result);
+}
+
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 29af4ce65d..414a9a417f 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9174,6 +9174,25 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate random UUID',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'generate UUID version 7', proisstrict => 'f',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => 'timestamptz', prosrc => 'uuidv7',
+ proargnames => '{unix_ts_ms}', proargmodes => '{i}' },
+{ oid => '9898', descr => 'extract timestamp from UUID version 7',
+ proname => 'uuid_extract_time', proleakproof => 't',
+ prorettype => 'timestamptz', proargtypes => 'uuid', prosrc => 'uuid_extract_time' },
+{ oid => '9899', descr => 'extract version from RFC 4122 UUID',
+ proname => 'uuid_extract_ver', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_ver' },
+{ oid => '9900', descr => 'extract variant from UUID',
+ proname => 'uuid_extract_var', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_var' },
# pg_lsn
{ oid => '3229', descr => 'I/O',
diff --git a/src/include/datatype/timestamp.h b/src/include/datatype/timestamp.h
index 3a37cb661e..652aeb428e 100644
--- a/src/include/datatype/timestamp.h
+++ b/src/include/datatype/timestamp.h
@@ -230,9 +230,10 @@ struct pg_itm_in
((y) < JULIAN_MAXYEAR || \
((y) == JULIAN_MAXYEAR && ((m) < JULIAN_MAXMONTH))))
-/* Julian-date equivalents of Day 0 in Unix and Postgres reckoning */
+/* Julian-date equivalents of Day 0 in Unix, Postgres and Gregorian epochs */
#define UNIX_EPOCH_JDATE 2440588 /* == date2j(1970, 1, 1) */
#define POSTGRES_EPOCH_JDATE 2451545 /* == date2j(2000, 1, 1) */
+#define GREGORIAN_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
/*
* Range limits for dates and timestamps.
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7610b011d6..1c37533975 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -126,9 +126,10 @@ WHERE p1.oid < p2.oid AND
p1.proretset != p2.proretset OR
p1.provolatile != p2.provolatile OR
p1.pronargs != p2.pronargs);
- oid | proname | oid | proname
------+---------+-----+---------
-(0 rows)
+ oid | proname | oid | proname
+------+---------+------+---------
+ 9896 | uuidv7 | 9897 | uuidv7
+(1 row)
-- Look for uses of different type OIDs in the argument/result type fields
-- for different aliases of the same built-in function.
@@ -872,6 +873,12 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+uuidv4()
+uuidv7()
+uuidv7(timestamp with time zone)
+uuid_extract_time(uuid)
+uuid_extract_ver(uuid)
+uuid_extract_var(uuid)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8e7f21910d..e85174ae82 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,5 +168,110 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+ ?column?
+----------
+ @ 0
+(1 row)
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+ uuid_extract_ver
+------------------
+ 7
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+ uuid_extract_ver
+------------------
+ 5
+(1 row)
+
+SELECT uuid_extract_var(uuidv7());
+ uuid_extract_var
+------------------
+ 2
+(1 row)
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+ uuid_extract_time
+-------------------
+
+(1 row)
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+-- errors in edge cases of UUID v7
+SELECT 1 FROM uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '0ms');
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '1ms'); -- ERROR expected
+ERROR: Time argument of UUID v7 is outside of the valid range
+SELECT 1 FROM uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000'));
+ ?column?
+----------
+ 1
+(1 row)
+
+SELECT uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000')+'1ms'); -- ERROR expected
+ERROR: Time argument of UUID v7 is outside of the valid range
-- clean up
DROP TABLE guid1, guid2 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 9a8f437c7d..40c3152697 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,5 +85,46 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7 with same unix_ts_ms
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+INSERT INTO guid1 (guid_field) VALUES (uuidv7(now()));
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- check that timestamp is extracted correctly
+SELECT uuid_extract_time(uuidv7(TIMESTAMP '2024-01-16 13:37:00')) - TIMESTAMP '2024-01-16 13:37:00';
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+SELECT uuid_extract_var(uuidv7());
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+
+-- errors in edge cases of UUID v7
+SELECT 1 FROM uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '0ms');
+SELECT uuidv7('1970-01-01 00:00:00+00'::timestamptz - interval '1ms'); -- ERROR expected
+SELECT 1 FROM uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000'));
+SELECT uuidv7(uuid_extract_time('FFFFFFFF-FFFF-7FFF-B000-000000000000')+'1ms'); -- ERROR expected
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
--
2.43.0
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-29 12:01 ` Jelte Fennema-Nio <[email protected]>
0 siblings, 0 replies; 68+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-29 12:01 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; +Cc: pgsql-hackers; Andrey M. Borodin <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; Sergey Prokhorenko <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>; Alexander Lakhin <[email protected]>
On Thu, 25 Jan 2024 at 13:31, Aleksander Alekseev
<[email protected]> wrote:
> PFA v14.
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ Both functions return a version 4 (random) UUID. This is the most commonly
+ used type of UUID and is appropriate when random distribution of keys does
+ not affect performance of an application.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 (time-ordered + random) UUID. This UUID
+ version should be used when application prefers locality of identifiers.
+<synopsis>
I think it would be good to explain the tradeoffs between uuidv4 and
uuidv7 a bit better. How about changing the docs to something like
this:
<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
</synopsis>
Both functions return a version 4 (random) UUID. UUIDv4 is one of the
most commonly used types of UUID. It is appropriate when random
distribution of keys does not affect performance of an application or
when exposing the generation time of a UUID has unacceptable security
or business intelligence implications.
<synopsis>
<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
</synopsis>
This function returns a version 7 (time-ordered + random) UUID. It
provides much better data locality than UUIDv4, which can greatly
improve performance when UUID is used in a BTREE index (the default
index type in PostgreSQL). To achieve this data locality, UUIDv7
embeds its own generation time into the UUID. If exposing such a
timestamp has unacceptable security or business intelligence
implications, then uuidv4() should be used instead.
<synopsis>
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-25 17:04 ` Sergey Prokhorenko <[email protected]>
2024-01-29 11:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
1 sibling, 2 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-25 17:04 UTC (permalink / raw)
To: pgsql-hackers; Aleksander Alekseev <[email protected]>; +Cc: Andrey M. Borodin <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Aleksander,
In this case the documentation must state that the functions uuid_extract_time() and uuidv7(T) are against the RFC requirements, and that developers may use these functions with caution at their own risk, and these functions are not recommended for production environment.
The function uuidv7(T) is not better than uuid_extract_time(). Careless developers may well pass any business date into this function: document date, registration date, payment date, reporting date, start date of the current month, data download date, and even a constant. This would be a profanation of UUIDv7 with very negative consequences.
Sergey [email protected]
On Thursday, 25 January 2024 at 03:06:50 pm GMT+3, Aleksander Alekseev <[email protected]> wrote:
Hi,
> Postgres always was a bit hackerish, allowing slightly more then is safe. I.e. you can define immutable function that is not really immutable, turn off autovacuum or fsync. Why bother with safety guards here?
> My opinion is that we should have this function to extract timestamp. Even if it can return strange values for imprecise RFC implementation.
Completely agree.
Users that don't like or don't need it can pretend there are no
uuid_extract_time() and uuidv7(T) in Postgres. If we don't provide
them however, users that need them will end up writing their own
probably buggy and not compatible implementations. That would be much
worse.
> So +1 for erroring when you provide a timestamp outside of that range
> (either too far in the past or too far in the future).
>
> OK, it seems like we have some consensus on ERRORing..
>
> Do we have any other open items? Does v13 address all open items? Maybe let’s compose better error message?
>
> +1 for erroring when ts is outside range.
>
> v13 looks good for me. I think we have reached a optimal compromise.
Andrey, many thanks for the updated patch.
LGTM, cfbot is happy and I don't think we have any open items left. So
changing CF entry status back to RfC.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
@ 2024-01-29 11:38 ` Jelte Fennema-Nio <[email protected]>
2024-01-29 13:58 ` Re: UUID v7 Junwang Zhao <[email protected]>
1 sibling, 1 reply; 68+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-29 11:38 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: pgsql-hackers; Aleksander Alekseev <[email protected]>; Andrey M. Borodin <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
tl;dr I believe we should remove the uuidv7(timestamp) function from
this patchset.
On Thu, 25 Jan 2024 at 18:04, Sergey Prokhorenko
<[email protected]> wrote:
> In this case the documentation must state that the functions uuid_extract_time() and uuidv7(T) are against the RFC requirements, and that developers may use these functions with caution at their own risk, and these functions are not recommended for production environment.
>
> The function uuidv7(T) is not better than uuid_extract_time(). Careless developers may well pass any business date into this function: document date, registration date, payment date, reporting date, start date of the current month, data download date, and even a constant. This would be a profanation of UUIDv7 with very negative consequences.
After re-reading the RFC more diligently, I'm inclined to agree with
Sergey that uuidv7(timestamp) is quite problematic. And I would even
say that we should not provide uuidv7(timestamp) at all, and instead
should only provide uuidv7(). Providing an explicit timestamp for
UUIDv7 is explicitly against the spec (in my reading):
> Implementations acquire the current timestamp from a reliable
> source to provide values that are time-ordered and continually
> increasing. Care must be taken to ensure that timestamp changes
> from the environment or operating system are handled in a way that
> is consistent with implementation requirements. For example, if
> it is possible for the system clock to move backward due to either
> manual adjustment or corrections from a time synchronization
> protocol, implementations need to determine how to handle such
> cases. (See Altering, Fuzzing, or Smearing below.)
>
> ...
>
> UUID version 1 and 6 both utilize a Gregorian epoch timestamp
> while UUIDv7 utilizes a Unix Epoch timestamp. If other timestamp
> sources or a custom timestamp epoch are required, UUIDv8 MUST be
> used.
>
> ...
>
> Monotonicity (each subsequent value being greater than the last) is
> the backbone of time-based sortable UUIDs.
By allowing users to provide a timestamp we're not using a continually
increasing timestamp for our UUIDv7 generation, and thus it would not
be a valid UUIDv7 implementation.
I do agree with others however, that being able to pass in an
arbitrary timestamp for UUID generation would be very useful. For
example to be able to partition by the timestamp in the UUID and then
being able to later load data for an older timestamp and have it be
added to to the older partition. But it's possible to do that while
still following the spec, by using a UUIDv8 instead of UUIDv7. So for
this usecase we could make a helper function that generates a UUIDv8
using the same format as a UUIDv7, but allows storing arbitrary
timestamps. You might say, why not sligthly change UUIDv7 then? Well
mainly because of this critical sentence in the RFC:
> UUIDv8's uniqueness will be implementation-specific and MUST NOT be assumed.
That would allow us to say that using this UUIDv8 helper requires
careful usage and checks if uniqueness is required.
So I believe we should remove the uuidv7(timestamp) function from this patchset.
I don't see a problem with including uuid_extract_time though. Afaict
the only thing the RFC says about extracting timestamps is that the
RFC does not give a requirement or guarantee about how close the
stored timestamp is to the actual time:
> Implementations MAY alter the actual timestamp. Some examples
> include security considerations around providing a real clock
> value within a UUID, to correct inaccurate clocks, to handle leap
> seconds, or instead of dividing a number of microseconds by 1000
> to obtain a millisecond value; dividing by 1024 (or some other
> value) for performance reasons. This specification makes no
> requirement or guarantee about how close the clock value needs to
> be to the actual time.
I see no reason why we cannot make stronger guarantees about the
timestamps that we use to generate UUIDs with our uuidv7() function.
And then we can update the documentation for
uuid_extract_time to something like this:
> This function extracts a timestamptz from UUID versions 1, 6 and 7. For other
> versions and variants this function returns NULL. The extracted timestamp
> does not necessarily equate to the time of UUID generation. How close it is
> to the actual time depends on the implementation that generated to UUID.
> The uuidv7() function provided PostgreSQL will normally store the actual time of
> generation to in the UUID, but if large batches of UUIDs are generated at the
> same time it's possible that some UUIDs will store a time that is slightly later
> than their actual generation time.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 11:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
@ 2024-01-29 13:58 ` Junwang Zhao <[email protected]>
0 siblings, 0 replies; 68+ messages in thread
From: Junwang Zhao @ 2024-01-29 13:58 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; pgsql-hackers; Aleksander Alekseev <[email protected]>; Andrey M. Borodin <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
On Mon, Jan 29, 2024 at 7:38 PM Jelte Fennema-Nio <[email protected]> wrote:
>
> tl;dr I believe we should remove the uuidv7(timestamp) function from
> this patchset.
>
> On Thu, 25 Jan 2024 at 18:04, Sergey Prokhorenko
> <[email protected]> wrote:
> > In this case the documentation must state that the functions uuid_extract_time() and uuidv7(T) are against the RFC requirements, and that developers may use these functions with caution at their own risk, and these functions are not recommended for production environment.
> >
> > The function uuidv7(T) is not better than uuid_extract_time(). Careless developers may well pass any business date into this function: document date, registration date, payment date, reporting date, start date of the current month, data download date, and even a constant. This would be a profanation of UUIDv7 with very negative consequences.
>
> After re-reading the RFC more diligently, I'm inclined to agree with
> Sergey that uuidv7(timestamp) is quite problematic. And I would even
> say that we should not provide uuidv7(timestamp) at all, and instead
> should only provide uuidv7(). Providing an explicit timestamp for
> UUIDv7 is explicitly against the spec (in my reading):
>
> > Implementations acquire the current timestamp from a reliable
> > source to provide values that are time-ordered and continually
> > increasing. Care must be taken to ensure that timestamp changes
> > from the environment or operating system are handled in a way that
> > is consistent with implementation requirements. For example, if
> > it is possible for the system clock to move backward due to either
> > manual adjustment or corrections from a time synchronization
> > protocol, implementations need to determine how to handle such
> > cases. (See Altering, Fuzzing, or Smearing below.)
> >
> > ...
> >
> > UUID version 1 and 6 both utilize a Gregorian epoch timestamp
> > while UUIDv7 utilizes a Unix Epoch timestamp. If other timestamp
> > sources or a custom timestamp epoch are required, UUIDv8 MUST be
> > used.
> >
> > ...
> >
> > Monotonicity (each subsequent value being greater than the last) is
> > the backbone of time-based sortable UUIDs.
>
> By allowing users to provide a timestamp we're not using a continually
> increasing timestamp for our UUIDv7 generation, and thus it would not
> be a valid UUIDv7 implementation.
>
> I do agree with others however, that being able to pass in an
> arbitrary timestamp for UUID generation would be very useful. For
> example to be able to partition by the timestamp in the UUID and then
> being able to later load data for an older timestamp and have it be
> added to to the older partition. But it's possible to do that while
> still following the spec, by using a UUIDv8 instead of UUIDv7. So for
> this usecase we could make a helper function that generates a UUIDv8
> using the same format as a UUIDv7, but allows storing arbitrary
> timestamps. You might say, why not sligthly change UUIDv7 then? Well
> mainly because of this critical sentence in the RFC:
>
> > UUIDv8's uniqueness will be implementation-specific and MUST NOT be assumed.
>
> That would allow us to say that using this UUIDv8 helper requires
> careful usage and checks if uniqueness is required.
>
> So I believe we should remove the uuidv7(timestamp) function from this patchset.
Agreed, the RFC section 6.1[1] has the following statements:
```
UUID version 1 and 6 both utilize a Gregorian epoch timestamp while
UUIDv7 utilizes a Unix Epoch timestamp. If other timestamp sources or
a custom timestamp epoch are required, UUIDv8 MUST be used.
```
In contrib/uuid-ossp, uuidv1 does not allow the user to supply a
custom timestamp,
so I think it should be the same for uuidv6 and uuidv7.
And I have the same feeling that we should not consider v6 and v8 in
this patch.
[1]: https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-14#section-6.1-2.4.1
>
> I don't see a problem with including uuid_extract_time though. Afaict
> the only thing the RFC says about extracting timestamps is that the
> RFC does not give a requirement or guarantee about how close the
> stored timestamp is to the actual time:
>
> > Implementations MAY alter the actual timestamp. Some examples
> > include security considerations around providing a real clock
> > value within a UUID, to correct inaccurate clocks, to handle leap
> > seconds, or instead of dividing a number of microseconds by 1000
> > to obtain a millisecond value; dividing by 1024 (or some other
> > value) for performance reasons. This specification makes no
> > requirement or guarantee about how close the clock value needs to
> > be to the actual time.
>
> I see no reason why we cannot make stronger guarantees about the
> timestamps that we use to generate UUIDs with our uuidv7() function.
> And then we can update the documentation for
> uuid_extract_time to something like this:
>
> > This function extracts a timestamptz from UUID versions 1, 6 and 7. For other
> > versions and variants this function returns NULL. The extracted timestamp
> > does not necessarily equate to the time of UUID generation. How close it is
> > to the actual time depends on the implementation that generated to UUID.
> > The uuidv7() function provided PostgreSQL will normally store the actual time of
> > generation to in the UUID, but if large batches of UUIDs are generated at the
> > same time it's possible that some UUIDs will store a time that is slightly later
> > than their actual generation time.
>
>
--
Regards
Junwang Zhao
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
@ 2024-01-29 18:32 ` Andrey M. Borodin <[email protected]>
2024-01-29 20:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-30 00:27 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
1 sibling, 2 replies; 68+ messages in thread
From: Andrey M. Borodin @ 2024-01-29 18:32 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 25 Jan 2024, at 22:04, Sergey Prokhorenko <[email protected]> wrote:
>
> Aleksander,
>
> In this case the documentation must state that the functions uuid_extract_time() and uuidv7(T) are against the RFC requirements, and that developers may use these functions with caution at their own risk, and these functions are not recommended for production environment.
Refining documentation is good. However, saying that these functions are not recommended for production must be based on some real threats.
>
> The function uuidv7(T) is not better than uuid_extract_time(). Careless developers may well pass any business date into this function: document date, registration date, payment date, reporting date, start date of the current month, data download date, and even a constant. This would be a profanation of UUIDv7 with very negative consequences.
Even if the developer pass constant time to uuidv7(T) they will get what they asked for - unique identifier. Moreover - it still will be keeping locality. There will be no negative consequences at all.
On the contrary, experienced developer can leverage parameter when data locality should be reduced. If you have serveral streams of data, you might want to introduce some shift in reduce contention.
For example, you can generate uuidv7(now() + '1 day' * random(0,10)). This will split 1 contention point to 10 and increase ingestion performance 10x-fold.
> On 29 Jan 2024, at 18:58, Junwang Zhao <[email protected]> wrote:
>
> If other timestamp sources or
> a custom timestamp epoch are required, UUIDv8 MUST be used.
Well, yeah. RFC says this... in 4 capital letters :) I believe it's kind of a big deficiency that k-way sortable identifiers are not implementable on top of UUIDv7. Well, let's go without this function. UUIDv7 is still an improvement over previous versions.
Jelte, your documentation corrections looks good to me, I'll include them in next version.
Thanks!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-29 20:38 ` Jelte Fennema-Nio <[email protected]>
2024-01-30 06:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
1 sibling, 1 reply; 68+ messages in thread
From: Jelte Fennema-Nio @ 2024-01-29 20:38 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
On Mon, 29 Jan 2024 at 19:32, Andrey M. Borodin <[email protected]> wrote:
> Even if the developer pass constant time to uuidv7(T) they will get what they asked for - unique identifier. Moreover - it still will be keeping locality. There will be no negative consequences at all.
It will be significantly "less unique" than if they wouldn't pass a
constant time. Basically it would become a UUIDv4, but with 74 bits of
random data instead of 122. That might not be enough anymore to
"guarantee" uniqueness. I guess that's why it is required to use
UUIDv8 in these cases, because correct usage is now a requirement for
assuming uniqueness. And for UUIDv8 the spec says this:
> UUIDv8's uniqueness will be implementation-specific and MUST NOT be assumed.
> > On 29 Jan 2024, at 18:58, Junwang Zhao <[email protected]> wrote:
> >
> > If other timestamp sources or
> > a custom timestamp epoch are required, UUIDv8 MUST be used.
>
> Well, yeah. RFC says this... in 4 capital letters :)
As an FYI, there is an RFC that defines these keywords that's why they
are capital letters: https://www.ietf.org/rfc/rfc2119.txt
> I believe it's kind of a big deficiency that k-way sortable identifiers are not implementable on top of UUIDv7. Well, let's go without this function. UUIDv7 is still an improvement over previous versions.
Yeah, I liked the feature to generate UUIDv7 based on timestamp too.
But following the spec seems more important than a nice feature to me.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-29 20:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
@ 2024-01-30 06:54 ` Andrey M. Borodin <[email protected]>
2024-01-30 07:28 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey M. Borodin @ 2024-01-30 06:54 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 30 Jan 2024, at 01:38, Jelte Fennema-Nio <[email protected]> wrote:
>
> Yeah, I liked the feature to generate UUIDv7 based on timestamp too.
> But following the spec seems more important than a nice feature to me.
PFA v15. Changes: removed timestamp argument, incorporated Jelte’s documentation addons.
Thanks!
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v15-0001-Implement-UUID-v7.patch (16.7K, ../../[email protected]/2-v15-0001-Implement-UUID-v7.patch)
download | inline diff:
From 9dde2582821354d4638c822e8d77eee6ff60cfa5 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 20 Aug 2023 23:55:31 +0300
Subject: [PATCH v15] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
This function can optionally accept a timestamp used instead of current time.
This allows implementation of k-way sotable identifiers. For code readability
this commit adds alias uuidv4() to function gen_random_uuid().
Also we add a function to extract timestamp from UUID v1, v6 and v7.
To allow user to distinguish various UUID versions and variants
we add functions uuid_extract_ver() and uuid_extract_var().
Author: Andrey Borodin
Reviewers: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewers: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewers: Peter Eisentraut, Chris Travers, Lukas Fittl
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/func.sgml | 62 +++++++-
src/backend/utils/adt/uuid.c | 179 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 15 ++
src/include/datatype/timestamp.h | 3 +-
src/test/regress/expected/opr_sanity.out | 5 +
src/test/regress/expected/uuid.out | 71 +++++++++
src/test/regress/sql/uuid.sql | 26 ++++
7 files changed, 357 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 6788ba8ef4a..588dd1ffd34 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14128,13 +14128,69 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_time</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_ver</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_var</primary>
+ </indexterm>
+
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID:
+ <function>gen_random_uuid</function>, <function>uuidv4</function>, and <function>uuidv7</function>.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ Both functions return a version 4 (random) UUID. UUIDv4 is one of the
+ most commonly used types of UUID. It is appropriate when random
+ distribution of keys does not affect performance of an application or
+ when exposing the generation time of a UUID has unacceptable security
+ or business intelligence implications.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random). It provides much better data locality
+ than UUIDv4, which can greatly improve performance when UUID is used in a
+ B-tree index (the default index type in PostgreSQL). To achieve this data
+ locality, UUIDv7 embeds its own generation time into the UUID. If exposing
+ such a timestamp has unacceptable security or business intelligence
+ implications, then uuidv4() should be used instead.
+<synopsis>
+<function>uuid_extract_time</function> (uuid) <returnvalue>timestamptz</returnvalue>
+</synopsis>
+ This function extracts a timestamptz from UUID versions 1, 6 and 7. For other
+ versions and variants this function returns NULL. The extracted timestamp
+ does not necessarily equate to the time of UUID generation. How close it is
+ to the actual time depends on the implementation that generated to UUID.
+ The uuidv7() function provided by PostgreSQL will normally store the actual time of
+ generation to in the UUID, but if large batches of UUIDs are generated at the
+ same time it's possible that some UUIDs will store a time that is slightly later
+ than their actual generation time.
+<synopsis>
+<function>uuid_extract_ver</function> (uuid) <returnvalue>int2</returnvalue>
+</synopsis>
+ This function extracts a version bits from UUID of variant described by
+ <ulink url="https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis">IETF standard</ulink>
+ (b10xx variant). For other variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_var</function> (uuid) <returnvalue>int2</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function extracts a vartiant bits from UUID.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 73dfd711c73..ef14d6cba27 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,13 +13,18 @@
#include "postgres.h"
+#include <sys/time.h>
+
+#include "access/xlog.h"
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
+#include "utils/datetime.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
+#include "utils/timestamp.h"
#include "utils/uuid.h"
/* sortsupport for uuid */
@@ -421,3 +426,177 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+
+static uint32_t sequence_counter;
+static uint64_t previous_timestamp = 0;
+
+
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ TimestampTz ts;
+ uint64_t tms;
+ struct timeval tp;
+ bool increment_counter;
+
+ gettimeofday(&tp, NULL);
+ tms = ((uint64_t)tp.tv_sec) * 1000 + (tp.tv_usec) / 1000;
+ /* time from clock is protected from backward leaps */
+ increment_counter = (tms <= previous_timestamp);
+
+ if (increment_counter)
+ {
+ /* Time did not advance from the previous generation, we must increment counter */
+ ++sequence_counter;
+ if (sequence_counter > 0x3ffff)
+ {
+ /* We only have 18-bit counter */
+ sequence_counter = 0;
+ previous_timestamp++;
+ }
+
+ /* protection from leap backward */
+ tms = previous_timestamp;
+
+ /* fill everything after the timestamp and counter with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /* most significant 4 bits of 18-bit counter */
+ uuid->data[6] = (unsigned char)(sequence_counter >> 14);
+ /* next 8 bits */
+ uuid->data[7] = (unsigned char)(sequence_counter >> 6);
+ /* least significant 6 bits */
+ uuid->data[8] = (unsigned char)(sequence_counter);
+ }
+ else
+ {
+ /* fill everything after the timestamp with random bytes */
+ if (!pg_strong_random(&uuid->data[6], UUID_LEN - 6))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Left-most counter bits are initialized as zero for the sole purpose
+ * of guarding against counter rollovers.
+ * See section "Fixed-Length Dedicated Counter Seeding"
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-09#monotonicity_counters
+ */
+ uuid->data[6] = (uuid->data[6] & 0xf7);
+
+ /* read randomly initialized bits of counter */
+ sequence_counter = ((uint32_t)uuid->data[8] & 0x3f) +
+ (((uint32_t)uuid->data[7]) << 6) +
+ (((uint32_t)uuid->data[6] & 0x0f) << 14);
+
+ previous_timestamp = tms;
+ }
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char)(tms >> 40);
+ uuid->data[1] = (unsigned char)(tms >> 32);
+ uuid->data[2] = (unsigned char)(tms >> 24);
+ uuid->data[3] = (unsigned char)(tms >> 16);
+ uuid->data[4] = (unsigned char)(tms >> 8);
+ uuid->data[5] = (unsigned char)tms;
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis
+ */
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+Datum
+uuid_extract_time(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ TimestampTz ts;
+ uint64_t tms;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+
+ if ((uuid->data[6] & 0xf0) == 0x70)
+ {
+ tms = uuid->data[5];
+ tms += ((uint64_t)uuid->data[4]) << 8;
+ tms += ((uint64_t)uuid->data[3]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 32;
+ tms += ((uint64_t)uuid->data[0]) << 40;
+
+ ts = (TimestampTz) (tms * 1000) - /* convert ms to us, than adjust */
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x10)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 8;
+ tms += ((uint64_t)uuid->data[3]);
+ tms += ((uint64_t)uuid->data[4]) << 40;
+ tms += ((uint64_t)uuid->data[5]) << 32;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 56;
+ tms += ((uint64_t)uuid->data[7]) << 48;
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x60)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 52;
+ tms += ((uint64_t)uuid->data[1]) << 44;
+ tms += ((uint64_t)uuid->data[2]) << 36;
+ tms += ((uint64_t)uuid->data[3]) << 28;
+ tms += ((uint64_t)uuid->data[4]) << 20;
+ tms += ((uint64_t)uuid->data[5]) << 12;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 8;
+ tms += ((uint64_t)uuid->data[7]);
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ PG_RETURN_NULL();
+}
+
+Datum
+uuid_extract_ver(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+ result = uuid->data[6] >> 4;
+
+ PG_RETURN_UINT16(result);
+}
+
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 29af4ce65d5..f9be09464be 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9174,6 +9174,21 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate random UUID',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'extract timestamp from UUID version 7',
+ proname => 'uuid_extract_time', proleakproof => 't',
+ prorettype => 'timestamptz', proargtypes => 'uuid', prosrc => 'uuid_extract_time' },
+{ oid => '9898', descr => 'extract version from RFC 4122 UUID',
+ proname => 'uuid_extract_ver', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_ver' },
+{ oid => '9899', descr => 'extract variant from UUID',
+ proname => 'uuid_extract_var', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_var' },
# pg_lsn
{ oid => '3229', descr => 'I/O',
diff --git a/src/include/datatype/timestamp.h b/src/include/datatype/timestamp.h
index 3a37cb661e3..652aeb428e2 100644
--- a/src/include/datatype/timestamp.h
+++ b/src/include/datatype/timestamp.h
@@ -230,9 +230,10 @@ struct pg_itm_in
((y) < JULIAN_MAXYEAR || \
((y) == JULIAN_MAXYEAR && ((m) < JULIAN_MAXMONTH))))
-/* Julian-date equivalents of Day 0 in Unix and Postgres reckoning */
+/* Julian-date equivalents of Day 0 in Unix, Postgres and Gregorian epochs */
#define UNIX_EPOCH_JDATE 2440588 /* == date2j(1970, 1, 1) */
#define POSTGRES_EPOCH_JDATE 2451545 /* == date2j(2000, 1, 1) */
+#define GREGORIAN_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
/*
* Range limits for dates and timestamps.
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7610b011d68..f4b9ff654ab 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -872,6 +872,11 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+uuidv4()
+uuidv7()
+uuid_extract_time(uuid)
+uuid_extract_ver(uuid)
+uuid_extract_var(uuid)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8e7f21910d6..f401a550885 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,5 +168,76 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+ uuid_extract_ver
+------------------
+ 7
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+ uuid_extract_ver
+------------------
+ 5
+(1 row)
+
+SELECT uuid_extract_var(uuidv7());
+ uuid_extract_var
+------------------
+ 2
+(1 row)
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+ uuid_extract_time
+-------------------
+
+(1 row)
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 9a8f437c7d2..c7362cf4e13 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,5 +85,31 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+SELECT uuid_extract_var(uuidv7());
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
--
2.42.0
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-29 20:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-30 06:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-30 07:28 ` Sergey Prokhorenko <[email protected]>
2024-01-30 09:56 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-30 07:28 UTC (permalink / raw)
To: Jelte Fennema-Nio <[email protected]>; Andrey M. Borodin <[email protected]>; +Cc: pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Andrey,
I think this phrase is outdated: "This function can optionally accept a timestamp used instead of current time.This allows implementation of k-way sotable identifiers."
This phrase is wrong: "Both functions return a version 4 (random) UUID."
For this phrase the reason is unclear and the phrase is most likely incorrect:
if large batches of UUIDs are generated at the+ same time it's possible that some UUIDs will store a time that is slightly later+ than their actual generation time
Sergey Prokhorenko
[email protected]
On Tuesday, 30 January 2024 at 09:55:04 am GMT+3, Andrey M. Borodin <[email protected]> wrote:
> On 30 Jan 2024, at 01:38, Jelte Fennema-Nio <[email protected]> wrote:
>
> Yeah, I liked the feature to generate UUIDv7 based on timestamp too.
> But following the spec seems more important than a nice feature to me.
PFA v15. Changes: removed timestamp argument, incorporated Jelte’s documentation addons.
Thanks!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-29 20:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-30 06:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 07:28 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
@ 2024-01-30 09:56 ` Andrey M. Borodin <[email protected]>
2024-01-30 10:33 ` Re: UUID v7 Junwang Zhao <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey M. Borodin @ 2024-01-30 09:56 UTC (permalink / raw)
To: Sergey Prokhorenko <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 30 Jan 2024, at 12:28, Sergey Prokhorenko <[email protected]> wrote:
>
>
> I think this phrase is outdated: "This function can optionally accept a timestamp used instead of current time.
> This allows implementation of k-way sotable identifiers.”
Fixed.
> This phrase is wrong: "Both functions return a version 4 (random) UUID.”
This applies to functions gen_random_uuid() and uuidv4().
>
> For this phrase the reason is unclear and the phrase is most likely incorrect:
> if large batches of UUIDs are generated at the
> + same time it's possible that some UUIDs will store a time that is slightly later
> + than their actual generation time
I’ve rewritten this phrase, hope it’s more clear now.
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v16-0001-Implement-UUID-v7.patch (16.5K, ../../[email protected]/2-v16-0001-Implement-UUID-v7.patch)
download | inline diff:
From 8d07eec66f93adf1cfa2512f5b89f41fe558dfe6 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 20 Aug 2023 23:55:31 +0300
Subject: [PATCH v16] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
For code readability this commit adds alias uuidv4() to function gen_random_uuid().
Also we add a function to extract timestamp from UUID v1, v6 and v7.
To allow user to distinguish various UUID versions and variants
we add functions uuid_extract_ver() and uuid_extract_var().
Author: Andrey Borodin
Reviewers: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewers: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewers: Peter Eisentraut, Chris Travers, Lukas Fittl
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/func.sgml | 61 +++++++-
src/backend/utils/adt/uuid.c | 179 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 15 ++
src/include/datatype/timestamp.h | 3 +-
src/test/regress/expected/opr_sanity.out | 5 +
src/test/regress/expected/uuid.out | 71 +++++++++
src/test/regress/sql/uuid.sql | 26 ++++
7 files changed, 356 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 6788ba8ef4a..97abf7f4c69 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14128,13 +14128,68 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_time</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_ver</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_var</primary>
+ </indexterm>
+
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID:
+ <function>gen_random_uuid</function>, <function>uuidv4</function>, and <function>uuidv7</function>.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID. UUIDv4 is one of the
+ most commonly used types of UUID. It is appropriate when random
+ distribution of keys does not affect performance of an application or
+ when exposing the generation time of a UUID has unacceptable security
+ or business intelligence implications.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random). It provides much better data locality
+ than UUIDv4, which can greatly improve performance when UUID is used in a
+ B-tree index (the default index type in PostgreSQL). To achieve this data
+ locality, UUIDv7 embeds its own generation time into the UUID. If exposing
+ such a timestamp has unacceptable security or business intelligence
+ implications, then uuidv4() should be used instead.
+<synopsis>
+<function>uuid_extract_time</function> (uuid) <returnvalue>timestamptz</returnvalue>
+</synopsis>
+ This function extracts a timestamptz from UUID versions 1, 6 and 7. For other
+ versions and variants this function returns NULL. The extracted timestamp
+ does not necessarily equate to the time of UUID generation. How close it is
+ to the actual time depends on the implementation that generated to UUID.
+ The uuidv7() function provided by PostgreSQL will normally store the actual time,
+ with some exceptions: prevention of time leaps backwards and counter overflow
+ being carried to time step.
+<synopsis>
+<function>uuid_extract_ver</function> (uuid) <returnvalue>int2</returnvalue>
+</synopsis>
+ This function extracts a version bits from UUID of variant described by
+ <ulink url="https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis">IETF standard</ulink>
+ (b10xx variant). For other variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_var</function> (uuid) <returnvalue>int2</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function extracts a vartiant bits from UUID.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 73dfd711c73..ef14d6cba27 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,13 +13,18 @@
#include "postgres.h"
+#include <sys/time.h>
+
+#include "access/xlog.h"
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
+#include "utils/datetime.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
+#include "utils/timestamp.h"
#include "utils/uuid.h"
/* sortsupport for uuid */
@@ -421,3 +426,177 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+
+static uint32_t sequence_counter;
+static uint64_t previous_timestamp = 0;
+
+
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ TimestampTz ts;
+ uint64_t tms;
+ struct timeval tp;
+ bool increment_counter;
+
+ gettimeofday(&tp, NULL);
+ tms = ((uint64_t)tp.tv_sec) * 1000 + (tp.tv_usec) / 1000;
+ /* time from clock is protected from backward leaps */
+ increment_counter = (tms <= previous_timestamp);
+
+ if (increment_counter)
+ {
+ /* Time did not advance from the previous generation, we must increment counter */
+ ++sequence_counter;
+ if (sequence_counter > 0x3ffff)
+ {
+ /* We only have 18-bit counter */
+ sequence_counter = 0;
+ previous_timestamp++;
+ }
+
+ /* protection from leap backward */
+ tms = previous_timestamp;
+
+ /* fill everything after the timestamp and counter with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /* most significant 4 bits of 18-bit counter */
+ uuid->data[6] = (unsigned char)(sequence_counter >> 14);
+ /* next 8 bits */
+ uuid->data[7] = (unsigned char)(sequence_counter >> 6);
+ /* least significant 6 bits */
+ uuid->data[8] = (unsigned char)(sequence_counter);
+ }
+ else
+ {
+ /* fill everything after the timestamp with random bytes */
+ if (!pg_strong_random(&uuid->data[6], UUID_LEN - 6))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Left-most counter bits are initialized as zero for the sole purpose
+ * of guarding against counter rollovers.
+ * See section "Fixed-Length Dedicated Counter Seeding"
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-09#monotonicity_counters
+ */
+ uuid->data[6] = (uuid->data[6] & 0xf7);
+
+ /* read randomly initialized bits of counter */
+ sequence_counter = ((uint32_t)uuid->data[8] & 0x3f) +
+ (((uint32_t)uuid->data[7]) << 6) +
+ (((uint32_t)uuid->data[6] & 0x0f) << 14);
+
+ previous_timestamp = tms;
+ }
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char)(tms >> 40);
+ uuid->data[1] = (unsigned char)(tms >> 32);
+ uuid->data[2] = (unsigned char)(tms >> 24);
+ uuid->data[3] = (unsigned char)(tms >> 16);
+ uuid->data[4] = (unsigned char)(tms >> 8);
+ uuid->data[5] = (unsigned char)tms;
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis
+ */
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+Datum
+uuid_extract_time(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ TimestampTz ts;
+ uint64_t tms;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+
+ if ((uuid->data[6] & 0xf0) == 0x70)
+ {
+ tms = uuid->data[5];
+ tms += ((uint64_t)uuid->data[4]) << 8;
+ tms += ((uint64_t)uuid->data[3]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 32;
+ tms += ((uint64_t)uuid->data[0]) << 40;
+
+ ts = (TimestampTz) (tms * 1000) - /* convert ms to us, than adjust */
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x10)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 8;
+ tms += ((uint64_t)uuid->data[3]);
+ tms += ((uint64_t)uuid->data[4]) << 40;
+ tms += ((uint64_t)uuid->data[5]) << 32;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 56;
+ tms += ((uint64_t)uuid->data[7]) << 48;
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x60)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 52;
+ tms += ((uint64_t)uuid->data[1]) << 44;
+ tms += ((uint64_t)uuid->data[2]) << 36;
+ tms += ((uint64_t)uuid->data[3]) << 28;
+ tms += ((uint64_t)uuid->data[4]) << 20;
+ tms += ((uint64_t)uuid->data[5]) << 12;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 8;
+ tms += ((uint64_t)uuid->data[7]);
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ PG_RETURN_NULL();
+}
+
+Datum
+uuid_extract_ver(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+ result = uuid->data[6] >> 4;
+
+ PG_RETURN_UINT16(result);
+}
+
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
\ No newline at end of file
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 29af4ce65d5..f9be09464be 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9174,6 +9174,21 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate random UUID',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'extract timestamp from UUID version 7',
+ proname => 'uuid_extract_time', proleakproof => 't',
+ prorettype => 'timestamptz', proargtypes => 'uuid', prosrc => 'uuid_extract_time' },
+{ oid => '9898', descr => 'extract version from RFC 4122 UUID',
+ proname => 'uuid_extract_ver', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_ver' },
+{ oid => '9899', descr => 'extract variant from UUID',
+ proname => 'uuid_extract_var', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_var' },
# pg_lsn
{ oid => '3229', descr => 'I/O',
diff --git a/src/include/datatype/timestamp.h b/src/include/datatype/timestamp.h
index 3a37cb661e3..652aeb428e2 100644
--- a/src/include/datatype/timestamp.h
+++ b/src/include/datatype/timestamp.h
@@ -230,9 +230,10 @@ struct pg_itm_in
((y) < JULIAN_MAXYEAR || \
((y) == JULIAN_MAXYEAR && ((m) < JULIAN_MAXMONTH))))
-/* Julian-date equivalents of Day 0 in Unix and Postgres reckoning */
+/* Julian-date equivalents of Day 0 in Unix, Postgres and Gregorian epochs */
#define UNIX_EPOCH_JDATE 2440588 /* == date2j(1970, 1, 1) */
#define POSTGRES_EPOCH_JDATE 2451545 /* == date2j(2000, 1, 1) */
+#define GREGORIAN_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
/*
* Range limits for dates and timestamps.
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7610b011d68..f4b9ff654ab 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -872,6 +872,11 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+uuidv4()
+uuidv7()
+uuid_extract_time(uuid)
+uuid_extract_ver(uuid)
+uuid_extract_var(uuid)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8e7f21910d6..f401a550885 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,5 +168,76 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+ uuid_extract_ver
+------------------
+ 7
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+ uuid_extract_ver
+------------------
+ 5
+(1 row)
+
+SELECT uuid_extract_var(uuidv7());
+ uuid_extract_var
+------------------
+ 2
+(1 row)
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+ uuid_extract_time
+-------------------
+
+(1 row)
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 9a8f437c7d2..c7362cf4e13 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,5 +85,31 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+SELECT uuid_extract_var(uuidv7());
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
--
2.42.0
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-29 20:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-30 06:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 07:28 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-30 09:56 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-30 10:33 ` Junwang Zhao <[email protected]>
2024-01-30 13:35 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Junwang Zhao @ 2024-01-30 10:33 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi Andrey,
On Tue, Jan 30, 2024 at 5:56 PM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 30 Jan 2024, at 12:28, Sergey Prokhorenko <[email protected]> wrote:
> >
> >
> > I think this phrase is outdated: "This function can optionally accept a timestamp used instead of current time.
> > This allows implementation of k-way sotable identifiers.”
> Fixed.
>
> > This phrase is wrong: "Both functions return a version 4 (random) UUID.”
> This applies to functions gen_random_uuid() and uuidv4().
> >
> > For this phrase the reason is unclear and the phrase is most likely incorrect:
> > if large batches of UUIDs are generated at the
> > + same time it's possible that some UUIDs will store a time that is slightly later
> > + than their actual generation time
>
> I’ve rewritten this phrase, hope it’s more clear now.
>
>
> Best regards, Andrey Borodin.
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
\ No newline at end of file
It's always good to add a newline at the end of a source file, though
this might be nitpicky.
--
Regards
Junwang Zhao
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-29 20:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-30 06:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 07:28 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-30 09:56 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 10:33 ` Re: UUID v7 Junwang Zhao <[email protected]>
@ 2024-01-30 13:35 ` Andrey M. Borodin <[email protected]>
2024-01-30 18:37 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey M. Borodin @ 2024-01-30 13:35 UTC (permalink / raw)
To: Junwang Zhao <[email protected]>; +Cc: Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
> On 30 Jan 2024, at 15:33, Junwang Zhao <[email protected]> wrote:
>
> It's always good to add a newline at the end of a source file, though
> this might be nitpicky.
Thanks, also fixed warning found by CFBot.
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v17-0001-Implement-UUID-v7.patch (16.5K, ../../[email protected]/2-v17-0001-Implement-UUID-v7.patch)
download | inline diff:
From 148750ca11235bc24ef07ceb549b910ba2a862c2 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 20 Aug 2023 23:55:31 +0300
Subject: [PATCH v17] Implement UUID v7
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This commit adds function for UUID generation. Most important function here
is uuidv7() which generates new UUID according to the new standard.
For code readability this commit adds alias uuidv4() to function gen_random_uuid().
Also we add a function to extract timestamp from UUID v1, v6 and v7.
To allow user to distinguish various UUID versions and variants
we add functions uuid_extract_ver() and uuid_extract_var().
Author: Andrey Borodin
Reviewers: Sergey Prokhorenko, Kirk Wolak, Przemysław Sztoch
Reviewers: Nikolay Samokhvalov, Jelte Fennema-Nio, Aleksander Alekseev
Reviewers: Peter Eisentraut, Chris Travers, Lukas Fittl
Discussion: https://postgr.es/m/CAAhFRxitJv%3DyoGnXUgeLB_O%2BM7J2BJAmb5jqAT9gZ3bij3uLDA%40mail.gmail.com
---
doc/src/sgml/func.sgml | 61 +++++++-
src/backend/utils/adt/uuid.c | 178 +++++++++++++++++++++++
src/include/catalog/pg_proc.dat | 15 ++
src/include/datatype/timestamp.h | 3 +-
src/test/regress/expected/opr_sanity.out | 5 +
src/test/regress/expected/uuid.out | 71 +++++++++
src/test/regress/sql/uuid.sql | 26 ++++
7 files changed, 355 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 6788ba8ef4a..97abf7f4c69 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14128,13 +14128,68 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
<primary>gen_random_uuid</primary>
</indexterm>
+ <indexterm>
+ <primary>uuidv4</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuidv7</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_time</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_ver</primary>
+ </indexterm>
+
+ <indexterm>
+ <primary>uuid_extract_var</primary>
+ </indexterm>
+
<para>
- <productname>PostgreSQL</productname> includes one function to generate a UUID:
+ <productname>PostgreSQL</productname> includes several functions to generate a UUID:
+ <function>gen_random_uuid</function>, <function>uuidv4</function>, and <function>uuidv7</function>.
<synopsis>
<function>gen_random_uuid</function> () <returnvalue>uuid</returnvalue>
+<function>uuidv4</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ These functions return a version 4 (random) UUID. UUIDv4 is one of the
+ most commonly used types of UUID. It is appropriate when random
+ distribution of keys does not affect performance of an application or
+ when exposing the generation time of a UUID has unacceptable security
+ or business intelligence implications.
+<synopsis>
+<function>uuidv7</function> () <returnvalue>uuid</returnvalue>
+</synopsis>
+ This function returns a version 7 UUID (UNIX timestamp with 1ms precision +
+ randomly seeded counter + random). It provides much better data locality
+ than UUIDv4, which can greatly improve performance when UUID is used in a
+ B-tree index (the default index type in PostgreSQL). To achieve this data
+ locality, UUIDv7 embeds its own generation time into the UUID. If exposing
+ such a timestamp has unacceptable security or business intelligence
+ implications, then uuidv4() should be used instead.
+<synopsis>
+<function>uuid_extract_time</function> (uuid) <returnvalue>timestamptz</returnvalue>
+</synopsis>
+ This function extracts a timestamptz from UUID versions 1, 6 and 7. For other
+ versions and variants this function returns NULL. The extracted timestamp
+ does not necessarily equate to the time of UUID generation. How close it is
+ to the actual time depends on the implementation that generated to UUID.
+ The uuidv7() function provided by PostgreSQL will normally store the actual time,
+ with some exceptions: prevention of time leaps backwards and counter overflow
+ being carried to time step.
+<synopsis>
+<function>uuid_extract_ver</function> (uuid) <returnvalue>int2</returnvalue>
+</synopsis>
+ This function extracts a version bits from UUID of variant described by
+ <ulink url="https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis">IETF standard</ulink>
+ (b10xx variant). For other variants this function returns NULL.
+<synopsis>
+<function>uuid_extract_var</function> (uuid) <returnvalue>int2</returnvalue>
</synopsis>
- This function returns a version 4 (random) UUID. This is the most commonly
- used type of UUID and is appropriate for most applications.
+ This function extracts a vartiant bits from UUID.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 73dfd711c73..a157f69c2b7 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -13,13 +13,18 @@
#include "postgres.h"
+#include <sys/time.h>
+
+#include "access/xlog.h"
#include "common/hashfn.h"
#include "lib/hyperloglog.h"
#include "libpq/pqformat.h"
#include "port/pg_bswap.h"
#include "utils/builtins.h"
+#include "utils/datetime.h"
#include "utils/guc.h"
#include "utils/sortsupport.h"
+#include "utils/timestamp.h"
#include "utils/uuid.h"
/* sortsupport for uuid */
@@ -421,3 +426,176 @@ gen_random_uuid(PG_FUNCTION_ARGS)
PG_RETURN_UUID_P(uuid);
}
+
+static uint32_t sequence_counter;
+static uint64_t previous_timestamp = 0;
+
+
+Datum
+uuidv7(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = palloc(UUID_LEN);
+ uint64_t tms;
+ struct timeval tp;
+ bool increment_counter;
+
+ gettimeofday(&tp, NULL);
+ tms = ((uint64_t)tp.tv_sec) * 1000 + (tp.tv_usec) / 1000;
+ /* time from clock is protected from backward leaps */
+ increment_counter = (tms <= previous_timestamp);
+
+ if (increment_counter)
+ {
+ /* Time did not advance from the previous generation, we must increment counter */
+ ++sequence_counter;
+ if (sequence_counter > 0x3ffff)
+ {
+ /* We only have 18-bit counter */
+ sequence_counter = 0;
+ previous_timestamp++;
+ }
+
+ /* protection from leap backward */
+ tms = previous_timestamp;
+
+ /* fill everything after the timestamp and counter with random bytes */
+ if (!pg_strong_random(&uuid->data[8], UUID_LEN - 8))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /* most significant 4 bits of 18-bit counter */
+ uuid->data[6] = (unsigned char)(sequence_counter >> 14);
+ /* next 8 bits */
+ uuid->data[7] = (unsigned char)(sequence_counter >> 6);
+ /* least significant 6 bits */
+ uuid->data[8] = (unsigned char)(sequence_counter);
+ }
+ else
+ {
+ /* fill everything after the timestamp with random bytes */
+ if (!pg_strong_random(&uuid->data[6], UUID_LEN - 6))
+ ereport(ERROR,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("could not generate random values")));
+
+ /*
+ * Left-most counter bits are initialized as zero for the sole purpose
+ * of guarding against counter rollovers.
+ * See section "Fixed-Length Dedicated Counter Seeding"
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis-09#monotonicity_counters
+ */
+ uuid->data[6] = (uuid->data[6] & 0xf7);
+
+ /* read randomly initialized bits of counter */
+ sequence_counter = ((uint32_t)uuid->data[8] & 0x3f) +
+ (((uint32_t)uuid->data[7]) << 6) +
+ (((uint32_t)uuid->data[6] & 0x0f) << 14);
+
+ previous_timestamp = tms;
+ }
+
+ /* Fill in time part */
+ uuid->data[0] = (unsigned char)(tms >> 40);
+ uuid->data[1] = (unsigned char)(tms >> 32);
+ uuid->data[2] = (unsigned char)(tms >> 24);
+ uuid->data[3] = (unsigned char)(tms >> 16);
+ uuid->data[4] = (unsigned char)(tms >> 8);
+ uuid->data[5] = (unsigned char)tms;
+
+ /*
+ * Set magic numbers for a "version 7" (pseudorandom) UUID, see
+ * https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis
+ */
+ /* set version field, top four bits are 0, 1, 1, 1 */
+ uuid->data[6] = (uuid->data[6] & 0x0f) | 0x70;
+ /* set variant field, top two bits are 1, 0 */
+ uuid->data[8] = (uuid->data[8] & 0x3f) | 0x80;
+
+ PG_RETURN_UUID_P(uuid);
+}
+
+Datum
+uuid_extract_time(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ TimestampTz ts;
+ uint64_t tms;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+
+ if ((uuid->data[6] & 0xf0) == 0x70)
+ {
+ tms = uuid->data[5];
+ tms += ((uint64_t)uuid->data[4]) << 8;
+ tms += ((uint64_t)uuid->data[3]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 32;
+ tms += ((uint64_t)uuid->data[0]) << 40;
+
+ ts = (TimestampTz) (tms * 1000) - /* convert ms to us, than adjust */
+ (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x10)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 24;
+ tms += ((uint64_t)uuid->data[1]) << 16;
+ tms += ((uint64_t)uuid->data[2]) << 8;
+ tms += ((uint64_t)uuid->data[3]);
+ tms += ((uint64_t)uuid->data[4]) << 40;
+ tms += ((uint64_t)uuid->data[5]) << 32;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 56;
+ tms += ((uint64_t)uuid->data[7]) << 48;
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ if ((uuid->data[6] & 0xf0) == 0x60)
+ {
+ tms = ((uint64_t)uuid->data[0]) << 52;
+ tms += ((uint64_t)uuid->data[1]) << 44;
+ tms += ((uint64_t)uuid->data[2]) << 36;
+ tms += ((uint64_t)uuid->data[3]) << 28;
+ tms += ((uint64_t)uuid->data[4]) << 20;
+ tms += ((uint64_t)uuid->data[5]) << 12;
+ tms += (((uint64_t)uuid->data[6])&0xf) << 8;
+ tms += ((uint64_t)uuid->data[7]);
+
+ ts = (TimestampTz) (tms / 10) - /* convert 100-ns intervals to us, than adjust */
+ ((uint64_t)POSTGRES_EPOCH_JDATE - GREGORIAN_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
+
+ PG_RETURN_TIMESTAMPTZ(ts);
+ }
+
+ PG_RETURN_NULL();
+}
+
+Datum
+uuid_extract_ver(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+
+ if ((uuid->data[8] & 0xc0) != 0x80)
+ PG_RETURN_NULL();
+ result = uuid->data[6] >> 4;
+
+ PG_RETURN_UINT16(result);
+}
+
+Datum
+uuid_extract_var(PG_FUNCTION_ARGS)
+{
+ pg_uuid_t *uuid = PG_GETARG_UUID_P(0);
+ uint16_t result;
+ result = uuid->data[8] >> 6;
+
+ PG_RETURN_UINT16(result);
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 29af4ce65d5..f9be09464be 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9174,6 +9174,21 @@
{ oid => '3432', descr => 'generate random UUID',
proname => 'gen_random_uuid', proleakproof => 't', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9895', descr => 'generate random UUID',
+ proname => 'uuidv4', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'gen_random_uuid' },
+{ oid => '9896', descr => 'generate UUID version 7',
+ proname => 'uuidv7', proleakproof => 't', provolatile => 'v',
+ prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
+{ oid => '9897', descr => 'extract timestamp from UUID version 7',
+ proname => 'uuid_extract_time', proleakproof => 't',
+ prorettype => 'timestamptz', proargtypes => 'uuid', prosrc => 'uuid_extract_time' },
+{ oid => '9898', descr => 'extract version from RFC 4122 UUID',
+ proname => 'uuid_extract_ver', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_ver' },
+{ oid => '9899', descr => 'extract variant from UUID',
+ proname => 'uuid_extract_var', proleakproof => 't',
+ prorettype => 'int2', proargtypes => 'uuid', prosrc => 'uuid_extract_var' },
# pg_lsn
{ oid => '3229', descr => 'I/O',
diff --git a/src/include/datatype/timestamp.h b/src/include/datatype/timestamp.h
index 3a37cb661e3..652aeb428e2 100644
--- a/src/include/datatype/timestamp.h
+++ b/src/include/datatype/timestamp.h
@@ -230,9 +230,10 @@ struct pg_itm_in
((y) < JULIAN_MAXYEAR || \
((y) == JULIAN_MAXYEAR && ((m) < JULIAN_MAXMONTH))))
-/* Julian-date equivalents of Day 0 in Unix and Postgres reckoning */
+/* Julian-date equivalents of Day 0 in Unix, Postgres and Gregorian epochs */
#define UNIX_EPOCH_JDATE 2440588 /* == date2j(1970, 1, 1) */
#define POSTGRES_EPOCH_JDATE 2451545 /* == date2j(2000, 1, 1) */
+#define GREGORIAN_EPOCH_JDATE 2299161 /* == date2j(1582,10,15) */
/*
* Range limits for dates and timestamps.
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 7610b011d68..f4b9ff654ab 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -872,6 +872,11 @@ xid8ge(xid8,xid8)
xid8eq(xid8,xid8)
xid8ne(xid8,xid8)
xid8cmp(xid8,xid8)
+uuidv4()
+uuidv7()
+uuid_extract_time(uuid)
+uuid_extract_ver(uuid)
+uuid_extract_var(uuid)
-- restore normal output mode
\a\t
-- List of functions used by libpq's fe-lobj.c
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 8e7f21910d6..f401a550885 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -168,5 +168,76 @@ SELECT count(DISTINCT guid_field) FROM guid1;
2
(1 row)
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+ count
+-------
+ 2
+(1 row)
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+ uuid_extract_ver
+------------------
+ 7
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+ uuid_extract_ver
+------------------
+ 5
+(1 row)
+
+SELECT uuid_extract_var(uuidv7());
+ uuid_extract_var
+------------------
+ 2
+(1 row)
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+ uuid_extract_time
+-------------------
+
+(1 row)
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+ ?column?
+----------
+ t
+(1 row)
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 9a8f437c7d2..c7362cf4e13 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -85,5 +85,31 @@ INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
INSERT INTO guid1 (guid_field) VALUES (gen_random_uuid());
SELECT count(DISTINCT guid_field) FROM guid1;
+-- test of uuidv4() alias
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+INSERT INTO guid1 (guid_field) VALUES (uuidv4());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- generation test for v7
+TRUNCATE guid1;
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+INSERT INTO guid1 (guid_field) VALUES (uuidv7());
+SELECT count(DISTINCT guid_field) FROM guid1;
+
+-- support functions for UUID versions and variants
+SELECT uuid_extract_ver(uuidv7());
+SELECT uuid_extract_ver('{11111111-1111-1111-1111-111111111111}') IS NULL;
+SELECT uuid_extract_ver('{11111111-1111-5111-8111-111111111111}');
+SELECT uuid_extract_var(uuidv7());
+
+-- uuid_extract_time() must refuse to accept non-UUIDv7
+SELECT uuid_extract_time(gen_random_uuid());
+
+-- extract UUID v1, v6 and v7 timestamp
+SELECT uuid_extract_time('C232AB00-9414-11EC-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('1EC9414C-232A-6B00-B3C8-9F6BDECED846') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+SELECT uuid_extract_time('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday, February 22, 2022 2:22:22.00 PM GMT+05:00';
+
-- clean up
DROP TABLE guid1, guid2 CASCADE;
--
2.42.0
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-29 20:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-30 06:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 07:28 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-30 09:56 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 10:33 ` Re: UUID v7 Junwang Zhao <[email protected]>
2024-01-30 13:35 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-30 18:37 ` Sergey Prokhorenko <[email protected]>
0 siblings, 0 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-30 18:37 UTC (permalink / raw)
To: Junwang Zhao <[email protected]>; Andrey M. Borodin <[email protected]>; +Cc: Jelte Fennema-Nio <[email protected]>; pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
typo:
being carried to time step
should be:being carried to timestemp
Sergey Prokhorenko [email protected]
On Tuesday, 30 January 2024 at 04:35:45 pm GMT+3, Andrey M. Borodin <[email protected]> wrote:
> On 30 Jan 2024, at 15:33, Junwang Zhao <[email protected]> wrote:
>
> It's always good to add a newline at the end of a source file, though
> this might be nitpicky.
Thanks, also fixed warning found by CFBot.
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
@ 2024-01-30 00:27 ` Sergey Prokhorenko <[email protected]>
1 sibling, 0 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-30 00:27 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: pgsql-hackers; Aleksander Alekseev <[email protected]>; Przemysław Sztoch <[email protected]>; Nikolay Samokhvalov <[email protected]>; David G. Johnston <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Andrey,
I understand and agree with your goals. But instead of dangerous universal functions, it is better to develop safe highly specialized functions that implement only these goals.
There should not be a function uuidv7(T) from an arbitrary timestamp, but there should be a special function that implements your algorithm: uuidv8(now() + '1 century' * random(0,10)).
I replaced 1 day with 1 century because the spread of 1 day is too small. Over time, records will be inserted between existing records, which is undesirable.
Similarly, if we need to calculate the partition id, then we do not need to use the uuid_extract_time() function to provide the extracted timestamp, the accuracy of which cannot be guaranteed. Instead, we need to give exactly the partition id, calculated using the uuidv7 timestamp. For example, partitions may have approximately a month interval between each other.
As for the documentation, it must be indicated that the UUIDv7 structure is not timestamp + random, but timestamp + randomly seeded counter + random, like in all advanced implementations.
Sergey Prokhorenko
[email protected]
______________________________________________________________
On Monday, 29 January 2024 at 09:32:54 pm GMT+3, Andrey M. Borodin <[email protected]> wrote:
> On 25 Jan 2024, at 22:04, Sergey Prokhorenko <[email protected]> wrote:
>
> Aleksander,
>
> In this case the documentation must state that the functions uuid_extract_time() and uuidv7(T) are against the RFC requirements, and that developers may use these functions with caution at their own risk, and these functions are not recommended for production environment.
Refining documentation is good. However, saying that these functions are not recommended for production must be based on some real threats.
>
> The function uuidv7(T) is not better than uuid_extract_time(). Careless developers may well pass any business date into this function: document date, registration date, payment date, reporting date, start date of the current month, data download date, and even a constant. This would be a profanation of UUIDv7 with very negative consequences.
Even if the developer pass constant time to uuidv7(T) they will get what they asked for - unique identifier. Moreover - it still will be keeping locality. There will be no negative consequences at all.
On the contrary, experienced developer can leverage parameter when data locality should be reduced. If you have serveral streams of data, you might want to introduce some shift in reduce contention.
For example, you can generate uuidv7(now() + '1 day' * random(0,10)). This will split 1 contention point to 10 and increase ingestion performance 10x-fold.
> On 29 Jan 2024, at 18:58, Junwang Zhao <[email protected]> wrote:
>
> If other timestamp sources or
> a custom timestamp epoch are required, UUIDv8 MUST be used.
Well, yeah. RFC says this... in 4 capital letters :) I believe it's kind of a big deficiency that k-way sortable identifiers are not implementable on top of UUIDv7. Well, let's go without this function. UUIDv7 is still an improvement over previous versions.
Jelte, your documentation corrections looks good to me, I'll include them in next version.
Thanks!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
@ 2024-01-28 12:42 ` Sergey Prokhorenko <[email protected]>
1 sibling, 0 replies; 68+ messages in thread
From: Sergey Prokhorenko @ 2024-01-28 12:42 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; Nikolay Samokhvalov <[email protected]>; +Cc: David G. Johnston <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; Przemysław Sztoch <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
By the way, the Go language has also already implemented a function for UUIDv7: https://pkg.go.dev/github.com/gofrs/uuid#NewV7
Sergey Prokhorenko [email protected]
On Thursday, 25 January 2024 at 12:49:46 am GMT+3, Sergey Prokhorenko <[email protected]> wrote:
That's right! There is no point in waiting for the official approval of the new RFC, which obviously will not change anything. I have been a contributor to this RFC for several years, and I can testify that every aspect imaginable has been thoroughly researched and agreed upon. Nothing new will definitely appear in the new RFC.
Sergey [email protected]
On Monday, 22 January 2024 at 07:22:32 am GMT+3, Nikolay Samokhvalov <[email protected]> wrote:
On Fri, Jan 19, 2024 at 10:07 AM Andrey Borodin <[email protected]> wrote:
> On 19 Jan 2024, at 13:25, Andrey Borodin <[email protected]> wrote:
>
> Also, I've added some documentation on all functions.
Here's v12. Changes:
1. Documentation improvements
2. Code comments
3. Better commit message and reviews list
Thank you, Andrey! I have just checked v12 – cleanly applied to HEAD, and functions work well. I especially like that fact that we keep uuid_extract_time(..) here – this is a great thing to have for time-based partitioning, and in many cases we will be able to decide not to have a creation column timestamp (e.g., "created_at") at all, saving 8 bytes.
The docs and comments look great too.
Overall, the patch looks mature enough. It would be great to have it in pg17. Yes, the RFC is not fully finalized yet, but it's very close. And many libraries are already including implementation of UUIDv7 – here are some examples:
- https://www.npmjs.com/package/uuidv7
- https://crates.io/crates/uuidv7
- https://github.com/google/uuid/pull/139
Nik
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2024-01-18 20:49 ` Przemysław Sztoch <[email protected]>
2 siblings, 0 replies; 68+ messages in thread
From: Przemysław Sztoch @ 2024-01-18 20:49 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Aleksander Alekseev <[email protected]>; pgsql-hackers; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
We are not allowed to consider any time other than UTC.
You need to write to the authors of the standard. I suppose this is a
mistake.
I know from experience that errors in such standards most often appear
in examples.
Nobody detects them at first.
Everyone reads and checks ideas, not calculations.
Then developers during implementation tears out their hair.
Andrey Borodin wrote on 1/18/2024 4:39 PM:
>
>> On 18 Jan 2024, at 19:20, Aleksander Alekseev <[email protected]> wrote:
>>
>> Timestamp and TimestampTz are absolutely the same thing.
> My question is not about Postgres data types. I'm asking about examples in the standard.
>
> There's an example 017F22E2-79B0-7CC3-98C4-DC0C0C07398F. It is expected to be generated on "Tuesday, February 22, 2022 2:22:22.00 PM GMT-05:00".
> It's exaplained to be 164555774200000ns after 1582-10-15 00:00:00 UTC.
>
> But 164555774200000ns after 1582-10-15 00:00:00 UTC was 2022-02-22 19:22:22 UTC. And that was 2022-02-23 00:22:22 in UTC-05.
>
>
> Best regards, Andrey Borodin.
--
Przemysław Sztoch | Mobile +48 509 99 00 66
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
@ 2024-01-18 20:39 ` Przemysław Sztoch <[email protected]>
2024-01-19 11:07 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
1 sibling, 1 reply; 68+ messages in thread
From: Przemysław Sztoch @ 2024-01-18 20:39 UTC (permalink / raw)
To: Aleksander Alekseev <[email protected]>; +Cc: pgsql-hackers; Andrey Borodin <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Aleksander Alekseev wrote on 1/18/2024 3:20 PM:
> Hi,
>
>> Another question: how did you choose between using TimestampTz and
>> Timestamp types? I realize that internally it's all the same. Maybe
>> Timestamp will be slightly better since the way it is displayed
>> doesn't depend on the session settings. Many people I talked to find
>> this part of TimestampTz confusing.
>>
>> timstamptz internally always store UTC.
>> I believe that in SQL, when operating with time in UTC, you should always use timestamptz.
>> timestamp is theoretically the same thing. But internally it does not convert time to UTC and will lead to incorrect use.
> No.
>
> Timestamp and TimestampTz are absolutely the same thing. The only
> difference is how they are shown to the user. TimestampTz uses session
> context in order to be displayed in the TZ chosen by the user. Thus
> typically it is somewhat more confusing to the users and thus I asked
> whether there was a good reason to choose TimestampTz over Timestamp.
>
Theoretically, you're right. But look at this example:
SET timezone TO 'Europe/Warsaw';
SELECT extract(epoch from '2024-01-18 9:27:30'::timestamp),
extract(epoch from '2024-01-18 9:27:30'::timestamptz);
date_part | date_part
------------+------------
1705570050 | 1705566450
(1 row)
In my opinion, timestamptz gives greater guarantees that the time
internally is in UTC and the user gets the time in his/her time zone.
In the case of timestamp, it is never certain whether it keeps time in
UTC or in the local zone.
In the case of argument's type, there would be no problem because we
could create two functions.
Of course timestamp would be treated the same as timestamptz.
But here we have a problem with the function return type, which can only
be one. And since the time returned is in UTC, it should be timestamptz.
--
Przemysław Sztoch | Mobile +48 509 99 00 66
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 20:39 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
@ 2024-01-19 11:07 ` Aleksander Alekseev <[email protected]>
0 siblings, 0 replies; 68+ messages in thread
From: Aleksander Alekseev @ 2024-01-19 11:07 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Przemysław Sztoch <[email protected]>; Andrey Borodin <[email protected]>; Sergey Prokhorenko <[email protected]>; Jelte Fennema-Nio <[email protected]>; Nick Babadzhanian <[email protected]>; Mat Arye <[email protected]>; Peter Eisentraut <[email protected]>; Tom Lane <[email protected]>; Daniel Gustafsson <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Kyzer Davis (kydavis) <[email protected]>; Andres Freund <[email protected]>; [email protected] <[email protected]>; Kirk Wolak <[email protected]>
Hi,
> No.
>
> Timestamp and TimestampTz are absolutely the same thing. The only
> difference is how they are shown to the user. TimestampTz uses session
> context in order to be displayed in the TZ chosen by the user. Thus
> typically it is somewhat more confusing to the users and thus I asked
> whether there was a good reason to choose TimestampTz over Timestamp.
>
>
> Theoretically, you're right. But look at this example:
>
> SET timezone TO 'Europe/Warsaw';
> SELECT extract(epoch from '2024-01-18 9:27:30'::timestamp), extract(epoch from '2024-01-18 9:27:30'::timestamptz);
>
> date_part | date_part
> ------------+------------
> 1705570050 | 1705566450
> (1 row)
>
> In my opinion, timestamptz gives greater guarantees that the time internally is in UTC and the user gets the time in his/her time zone.
I believe you didn't notice, but this example just proves my point.
In this case you have two timestamps that are different _internally_,
but the way they are _shown_ is the same because the first one is in
UTC and the second one in your local session timezone, Europe/Warsaw.
extract(epoch ...) extract UNIX epoch, i.e. relies on the _internal_
representation. This is why you got different results.
This demonstrates that TimestampTz is a permanent source of confusion
for the users and the reason why personally I would prefer if UUIDv7
always used Timestamp (no Tz). TimestampTz can be converted to
TimestampTz by users who need them and have experience using them.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
@ 2024-12-12 18:08 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Masahiko Sawada @ 2024-12-12 18:08 UTC (permalink / raw)
To: Andrey M. Borodin <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Mon, Dec 9, 2024 at 7:42 PM Andrey M. Borodin <[email protected]> wrote:
>
>
>
> > On 10 Dec 2024, at 03:34, Masahiko Sawada <[email protected]> wrote:
> >
> > I've attached the updated patches.
>
> Both patches look good to me.
> I'm not sure, but, perhaps, commit message of unleakproofing a function should mention that the problem was reported in Peter E. review.
Pushed both patches.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
@ 2025-01-30 08:59 ` Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey Borodin @ 2025-01-30 08:59 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 12 Dec 2024, at 23:08, Masahiko Sawada <[email protected]> wrote:
>
> Pushed
Hi Masahiko!
I’ve found some inconsistency in handling of overflow. I’m not sure we should handle it, but anyway.
postgres=# select x,
uuid_extract_timestamp(uuidv7((x::text || ' year'::text)::interval)),
(x::text || ' year'::text)::interval
from generate_series(237,238) x;;
x | uuid_extract_timestamp | interval
-----+-----------------------------+-----------
237 | 2262-01-30 13:43:23.737+05 | 237 years
238 | 10598-02-10 19:41:13.736+05 | 238 years
(2 rows)
The thing is per RFC we represent time as number of nanoseconds since UNIX epoch. And we use int64, which will overflow in year 2262. I sincerely wish us to see this great year.
We can have a couple more centuries if we resort to unsigned int 64.
But it would be great to make our code work until
postgres=# select uuid_extract_timestamp('FFFFFFFF-FFFF-7FFF-bFFF-FFFFFFFFFFFF');
uuid_extract_timestamp
-----------------------------
10889-08-02 10:31:50.655+05
(1 row)
And using uint64 won’t help us.
Can we use int128 in code? Or, perhaps, carry this extra 10 bits in the extra argument of generate_uuidv7()? Or, perhaps, leave things as they stand now?
Thanks!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2025-01-30 19:54 ` Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Masahiko Sawada @ 2025-01-30 19:54 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Thu, Jan 30, 2025 at 12:59 AM Andrey Borodin <[email protected]> wrote:
>
>
>
> > On 12 Dec 2024, at 23:08, Masahiko Sawada <[email protected]> wrote:
> >
> > Pushed
>
> Hi Masahiko!
>
> I’ve found some inconsistency in handling of overflow. I’m not sure we should handle it, but anyway.
Thank you for the report!
>
> postgres=# select x,
> uuid_extract_timestamp(uuidv7((x::text || ' year'::text)::interval)),
> (x::text || ' year'::text)::interval
> from generate_series(237,238) x;;
> x | uuid_extract_timestamp | interval
> -----+-----------------------------+-----------
> 237 | 2262-01-30 13:43:23.737+05 | 237 years
> 238 | 10598-02-10 19:41:13.736+05 | 238 years
> (2 rows)
>
> The thing is per RFC we represent time as number of nanoseconds since UNIX epoch. And we use int64, which will overflow in year 2262. I sincerely wish us to see this great year.
> We can have a couple more centuries if we resort to unsigned int 64.
>
> But it would be great to make our code work until
>
> postgres=# select uuid_extract_timestamp('FFFFFFFF-FFFF-7FFF-bFFF-FFFFFFFFFFFF');
> uuid_extract_timestamp
> -----------------------------
> 10889-08-02 10:31:50.655+05
> (1 row)
>
> And using uint64 won’t help us.
I don't think using uint64 instead of int64 for nanoseconds doesn't
resolve the problem. We will not be able to shift the timestamp for a
date before 1970/1/1.
> Or, perhaps, carry this extra 10 bits in the extra argument of generate_uuidv7()?
I like this idea. Would you like to write a patch, or shall I?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
@ 2025-01-31 07:08 ` Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey Borodin @ 2025-01-31 07:08 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 31 Jan 2025, at 00:54, Masahiko Sawada <[email protected]> wrote:
>
> I like this idea. Would you like to write a patch, or shall I?
I propose to separate milliseconds from nanoseconds. Please find attached implementation of this.
With this patch we can generate correct UUIDs in a very distant future.
postgres=# select x, uuid_extract_timestamp(uuidv7((x::text || ' year'::text)::interval)),
(x::text || ' year'::text)::interval
from generate_series(1,9000,1000) x;
x | uuid_extract_timestamp | interval
------+-----------------------------+------------
1 | 2026-01-31 12:00:53.084+05 | 1 year
1001 | 3026-01-31 12:00:53.084+05 | 1001 years
2001 | 4026-01-31 12:00:53.084+05 | 2001 years
3001 | 5026-01-31 12:00:53.084+05 | 3001 years
4001 | 6026-01-31 12:00:53.084+05 | 4001 years
5001 | 7026-01-31 12:00:53.085+05 | 5001 years
6001 | 8026-01-31 12:00:53.085+05 | 6001 years
7001 | 9026-01-31 12:00:53.085+05 | 7001 years
8001 | 10026-01-31 12:00:53.085+05 | 8001 years
(9 rows)
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] 0001-UUDv7-fix-offset-computations-in-dates-after-2262.patch (3.6K, ../../[email protected]/2-0001-UUDv7-fix-offset-computations-in-dates-after-2262.patch)
download | inline diff:
From 8b52d0942f657c35e238bd95bd2d95aa5c4a5b2e Mon Sep 17 00:00:00 2001
From: Andrey Borodin <[email protected]>
Date: Fri, 31 Jan 2025 12:03:16 +0500
Subject: [PATCH] UUDv7: fix offset computations in dates after 2262
We used nanosecond representation of offsetted time values which
cannot be stored in 64-bit integer for dates significantly after
beginning of UNIX epoch. To prevent overflow we separate millisecond
part from nanoseconds, thus allowing us to store both parts in 64-bit
integers.
---
src/backend/utils/adt/uuid.c | 26 ++++++++++++++------------
1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 4f8402ef925..3349c2674c8 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -68,7 +68,7 @@ static int uuid_fast_cmp(Datum x, Datum y, SortSupport ssup);
static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
-static inline int64 get_real_time_ns_ascending();
+static inline uint64 get_real_time_ns_ascending();
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -476,11 +476,11 @@ gen_random_uuid(PG_FUNCTION_ARGS)
* The returned timestamp is ensured to be at least SUBMS_MINIMAL_STEP greater
* than the previous returned timestamp (on this backend).
*/
-static inline int64
+static inline uint64
get_real_time_ns_ascending()
{
- static int64 previous_ns = 0;
- int64 ns;
+ static uint64 previous_ns = 0;
+ uint64 ns;
/* Get the current real timestamp */
@@ -527,13 +527,13 @@ get_real_time_ns_ascending()
* used for time-dependent bits of UUID.
*/
static pg_uuid_t *
-generate_uuidv7(int64 ns)
+generate_uuidv7(uint64 ms, uint64 ns_in_ms)
{
pg_uuid_t *uuid = palloc(UUID_LEN);
int64 unix_ts_ms;
int32 increased_clock_precision;
- unix_ts_ms = ns / NS_PER_MS;
+ unix_ts_ms = ms;
/* Fill in time part */
uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
@@ -547,7 +547,7 @@ generate_uuidv7(int64 ns)
* sub-millisecond timestamp fraction (SUBMS_BITS bits, not
* SUBMS_MINIMAL_STEP_BITS)
*/
- increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+ increased_clock_precision = ((ns_in_ms) * (1 << SUBMS_BITS)) / NS_PER_MS;
/* Fill the increased clock precision to "rand_a" bits */
uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
@@ -586,7 +586,8 @@ generate_uuidv7(int64 ns)
Datum
uuidv7(PG_FUNCTION_ARGS)
{
- pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
+ uint64 ns = get_real_time_ns_ascending();
+ pg_uuid_t *uuid = generate_uuidv7(ns / NS_PER_MS, ns % NS_PER_MS);
PG_RETURN_UUID_P(uuid);
}
@@ -600,7 +601,9 @@ uuidv7_interval(PG_FUNCTION_ARGS)
Interval *shift = PG_GETARG_INTERVAL_P(0);
TimestampTz ts;
pg_uuid_t *uuid;
- int64 ns = get_real_time_ns_ascending();
+ /* 64 bits is enough for real time, but not for a time range of UUID */
+ uint64 ns = get_real_time_ns_ascending();
+ uint64 us;
/*
* Shift the current timestamp by the given interval. To calculate time
@@ -621,11 +624,10 @@ uuidv7_interval(PG_FUNCTION_ARGS)
/*
* Convert a TimestampTz value back to an UNIX epoch and back nanoseconds.
*/
- ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
- * NS_PER_US + ns % NS_PER_US;
+ us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC);
/* Generate an UUIDv7 */
- uuid = generate_uuidv7(ns);
+ uuid = generate_uuidv7(us / 1000, (us % 1000) * 1000 + ns % NS_PER_US);
PG_RETURN_UUID_P(uuid);
}
--
2.42.0
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2025-01-31 18:49 ` Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Masahiko Sawada @ 2025-01-31 18:49 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Thu, Jan 30, 2025 at 11:09 PM Andrey Borodin <[email protected]> wrote:
>
>
>
> > On 31 Jan 2025, at 00:54, Masahiko Sawada <[email protected]> wrote:
> >
> > I like this idea. Would you like to write a patch, or shall I?
>
> I propose to separate milliseconds from nanoseconds. Please find attached implementation of this.
> With this patch we can generate correct UUIDs in a very distant future.
> postgres=# select x, uuid_extract_timestamp(uuidv7((x::text || ' year'::text)::interval)),
> (x::text || ' year'::text)::interval
> from generate_series(1,9000,1000) x;
> x | uuid_extract_timestamp | interval
> ------+-----------------------------+------------
> 1 | 2026-01-31 12:00:53.084+05 | 1 year
> 1001 | 3026-01-31 12:00:53.084+05 | 1001 years
> 2001 | 4026-01-31 12:00:53.084+05 | 2001 years
> 3001 | 5026-01-31 12:00:53.084+05 | 3001 years
> 4001 | 6026-01-31 12:00:53.084+05 | 4001 years
> 5001 | 7026-01-31 12:00:53.085+05 | 5001 years
> 6001 | 8026-01-31 12:00:53.085+05 | 6001 years
> 7001 | 9026-01-31 12:00:53.085+05 | 7001 years
> 8001 | 10026-01-31 12:00:53.085+05 | 8001 years
> (9 rows)
Thank you for the patch! I agree with the basic direction of this fix.
Here are some review comments:
---
-static inline int64 get_real_time_ns_ascending();
+static inline uint64 get_real_time_ns_ascending();
IIUC we don't need to replace int64 with uint64 if we have two
separate parameters for generate_uuidv7(). It seems to be conventional
to use a signed int for timestamps.
---
Need to update the function comment of generate_uuidv7() as we changed
the function arguments.
---
- ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
SECS_PER_DAY * USECS_PER_SEC)
- * NS_PER_US + ns % NS_PER_US;
+ us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
SECS_PER_DAY * USECS_PER_SEC);
/* Generate an UUIDv7 */
- uuid = generate_uuidv7(ns);
+ uuid = generate_uuidv7(us / 1000, (us % 1000) * 1000 + ns % NS_PER_US);
I think we can have an inline function or a marco (or use TMODULO()?)
to split nanoseconds into milliseconds and sub-milliseconds so that
uuidv7() and uuidv7_interval() can pass them to generate_uuidv7().
The comments in uuidv7_interval() also need to be updated accordingly.
---
I think we need to consider how we can handle the timestamp shifting.
UUIDv7 contains 48 bits Unix timestamp at milliseconds precision,
which can represent timestamps approximately between 2493 BC and 6432
AC. If users specify an interval to shift the timestamp beyond the
range, 48-bits timestamp would be wrapped around and they would not be
able to get an expected result. Do we need to raise an error in that
case?
---
Another problem I found in uuid_extract_timestamp() is that it cannot
correctly extract a timestamp before 1970/1/1 stored in a UUIDv7
value:
postgres(1:1795331)=# select year, uuid_extract_timestamp(uuidv7((year
|| 'year ago')::interval)) from generate_series(54, 56) year;
year | uuid_extract_timestamp
------+-----------------------------
54 | 1971-01-31 10:46:25.111-08
55 | 1970-01-31 10:46:25.111-08
56 | 10888-09-01 17:18:15.768-07
(3 rows)
The problem is that we correctly store a negative timestamp value in a
UUIDv7 value but uuid_extract_timestamp() unconditionally treats it as
a positive timestamp value. I think this is a separate bug we need to
fix.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
@ 2025-02-02 10:15 ` Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey Borodin @ 2025-02-02 10:15 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 31 Jan 2025, at 23:49, Masahiko Sawada <[email protected]> wrote:
>
> Thank you for the patch! I agree with the basic direction of this fix.
> Here are some review comments:
>
> ---
> -static inline int64 get_real_time_ns_ascending();
> +static inline uint64 get_real_time_ns_ascending();
>
> IIUC we don't need to replace int64 with uint64 if we have two
> separate parameters for generate_uuidv7(). It seems to be conventional
> to use a signed int for timestamps.
OK, done.
>
> ---
> Need to update the function comment of generate_uuidv7() as we changed
> the function arguments.
Done.
>
> ---
> - ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
> SECS_PER_DAY * USECS_PER_SEC)
> - * NS_PER_US + ns % NS_PER_US;
> + us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
> SECS_PER_DAY * USECS_PER_SEC);
>
> /* Generate an UUIDv7 */
> - uuid = generate_uuidv7(ns);
> + uuid = generate_uuidv7(us / 1000, (us % 1000) * 1000 + ns % NS_PER_US);
>
> I think we can have an inline function or a marco (or use TMODULO()?)
> to split nanoseconds into milliseconds and sub-milliseconds so that
> uuidv7() and uuidv7_interval() can pass them to generate_uuidv7().
I doubt that such macro will make core more readable. I've replaced 1000 with macros.
>
> The comments in uuidv7_interval() also need to be updated accordingly.
Done.
>
> ---
> I think we need to consider how we can handle the timestamp shifting.
> UUIDv7 contains 48 bits Unix timestamp at milliseconds precision,
> which can represent timestamps approximately between 2493 BC and 6432
> AC. If users specify an interval to shift the timestamp beyond the
> range, 48-bits timestamp would be wrapped around and they would not be
> able to get an expected result. Do we need to raise an error in that
> case?
>
> ---
> Another problem I found in uuid_extract_timestamp() is that it cannot
> correctly extract a timestamp before 1970/1/1 stored in a UUIDv7
> value:
>
> postgres(1:1795331)=# select year, uuid_extract_timestamp(uuidv7((year
> || 'year ago')::interval)) from generate_series(54, 56) year;
> year | uuid_extract_timestamp
> ------+-----------------------------
> 54 | 1971-01-31 10:46:25.111-08
> 55 | 1970-01-31 10:46:25.111-08
> 56 | 10888-09-01 17:18:15.768-07
> (3 rows)
>
> The problem is that we correctly store a negative timestamp value in a
> UUIDv7 value but uuid_extract_timestamp() unconditionally treats it as
> a positive timestamp value. I think this is a separate bug we need to
> fix.
RFC says unix_ts_ms is unsigned. So, luckily, no BC dates. I bet Pharaohs could not measure nanoseconds.
I think it's totally fine to wrap UUID values around year 10598 without an error.
I was thinking about incorporating test like this.
>> With this patch we can generate correct UUIDs in a very distant future.
>> postgres=# select x, uuid_extract_timestamp(uuidv7((x::text || ' year'::text)::interval)),
>> (x::text || ' year'::text)::interval
>> from generate_series(1,9000,1000) x;
>> x | uuid_extract_timestamp | interval
>> ------+-----------------------------+------------
>> 1 | 2026-01-31 12:00:53.084+05 | 1 year
>> 1001 | 3026-01-31 12:00:53.084+05 | 1001 years
>> 2001 | 4026-01-31 12:00:53.084+05 | 2001 years
>> 3001 | 5026-01-31 12:00:53.084+05 | 3001 years
>> 4001 | 6026-01-31 12:00:53.084+05 | 4001 years
>> 5001 | 7026-01-31 12:00:53.085+05 | 5001 years
>> 6001 | 8026-01-31 12:00:53.085+05 | 6001 years
>> 7001 | 9026-01-31 12:00:53.085+05 | 7001 years
>> 8001 | 10026-01-31 12:00:53.085+05 | 8001 years
>> (9 rows)
or maybe something simple like
with u as (select uuidv7() id) select uuid_extract_timestamp(uuidv7('9999-09-09 12:34:56.789+05' - uuid_extract_timestamp(u.id))) from u;
But it would still be flaky, second call to uuidv7() can overflow a millisecond.
Thanks!
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v2-0001-UUDv7-fix-offset-computations-in-dates-after-2262.patch (3.8K, ../../[email protected]/2-v2-0001-UUDv7-fix-offset-computations-in-dates-after-2262.patch)
download | inline diff:
From 6890f52395a924f6af33eb86c2c3addb204ef483 Mon Sep 17 00:00:00 2001
From: Andrey Borodin <[email protected]>
Date: Fri, 31 Jan 2025 12:03:16 +0500
Subject: [PATCH v2] UUDv7: fix offset computations in dates after 2262
We used nanosecond representation of offsetted time values which
cannot be stored in 64-bit integer for dates significantly after
beginning of UNIX epoch. To prevent overflow we separate millisecond
part from nanoseconds, thus allowing us to store both parts in 64-bit
integers.
---
src/backend/utils/adt/uuid.c | 22 ++++++++++++----------
1 file changed, 12 insertions(+), 10 deletions(-)
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 4f8402ef92..f579bb3a64 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -28,6 +28,7 @@
/* helper macros */
#define NS_PER_S INT64CONST(1000000000)
#define NS_PER_MS INT64CONST(1000000)
+#define US_PER_MS INT64CONST(1000)
#define NS_PER_US INT64CONST(1000)
/*
@@ -69,6 +70,7 @@ static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
static inline int64 get_real_time_ns_ascending();
+static pg_uuid_t *generate_uuidv7(uint64 unix_ts_ms, uint32 ns_in_ms);
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -523,18 +525,16 @@ get_real_time_ns_ascending()
* described in the RFC. This method utilizes 12 bits from the "rand_a" bits
* to store a 1/4096 (or 2^12) fraction of sub-millisecond precision.
*
- * ns is a number of nanoseconds since start of the UNIX epoch. This value is
+ * unix_ts_ms is a number of milliseconds since start of the UNIX epoch,
+ * ns_in_ms is a number of nanoseconds within millisecond. These values are
* used for time-dependent bits of UUID.
*/
static pg_uuid_t *
-generate_uuidv7(int64 ns)
+generate_uuidv7(uint64 unix_ts_ms, uint32 ns_in_ms)
{
pg_uuid_t *uuid = palloc(UUID_LEN);
- int64 unix_ts_ms;
int32 increased_clock_precision;
- unix_ts_ms = ns / NS_PER_MS;
-
/* Fill in time part */
uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
@@ -547,7 +547,7 @@ generate_uuidv7(int64 ns)
* sub-millisecond timestamp fraction (SUBMS_BITS bits, not
* SUBMS_MINIMAL_STEP_BITS)
*/
- increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+ increased_clock_precision = ((ns_in_ms) * (1 << SUBMS_BITS)) / NS_PER_MS;
/* Fill the increased clock precision to "rand_a" bits */
uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
@@ -586,7 +586,8 @@ generate_uuidv7(int64 ns)
Datum
uuidv7(PG_FUNCTION_ARGS)
{
- pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
+ int64 ns = get_real_time_ns_ascending();
+ pg_uuid_t *uuid = generate_uuidv7(ns / NS_PER_MS, ns % NS_PER_MS);
PG_RETURN_UUID_P(uuid);
}
@@ -600,7 +601,9 @@ uuidv7_interval(PG_FUNCTION_ARGS)
Interval *shift = PG_GETARG_INTERVAL_P(0);
TimestampTz ts;
pg_uuid_t *uuid;
+ /* 64 bits is enough for real time, but not for a time range of UUID */
int64 ns = get_real_time_ns_ascending();
+ int64 us;
/*
* Shift the current timestamp by the given interval. To calculate time
@@ -621,11 +624,10 @@ uuidv7_interval(PG_FUNCTION_ARGS)
/*
* Convert a TimestampTz value back to an UNIX epoch and back nanoseconds.
*/
- ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
- * NS_PER_US + ns % NS_PER_US;
+ us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC);
/* Generate an UUIDv7 */
- uuid = generate_uuidv7(ns);
+ uuid = generate_uuidv7(us / US_PER_MS, (us % US_PER_MS) * NS_PER_US + ns % NS_PER_US);
PG_RETURN_UUID_P(uuid);
}
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2025-02-04 22:02 ` Masahiko Sawada <[email protected]>
2025-02-09 17:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Masahiko Sawada @ 2025-02-04 22:02 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Sun, Feb 2, 2025 at 2:15 AM Andrey Borodin <[email protected]> wrote:
>
>
>
> > On 31 Jan 2025, at 23:49, Masahiko Sawada <[email protected]> wrote:
> >
> > Thank you for the patch! I agree with the basic direction of this fix.
> > Here are some review comments:
> >
> > ---
> > -static inline int64 get_real_time_ns_ascending();
> > +static inline uint64 get_real_time_ns_ascending();
> >
> > IIUC we don't need to replace int64 with uint64 if we have two
> > separate parameters for generate_uuidv7(). It seems to be conventional
> > to use a signed int for timestamps.
>
> OK, done.
>
> >
> > ---
> > Need to update the function comment of generate_uuidv7() as we changed
> > the function arguments.
>
> Done.
>
> >
> > ---
> > - ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
> > SECS_PER_DAY * USECS_PER_SEC)
> > - * NS_PER_US + ns % NS_PER_US;
> > + us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
> > SECS_PER_DAY * USECS_PER_SEC);
> >
> > /* Generate an UUIDv7 */
> > - uuid = generate_uuidv7(ns);
> > + uuid = generate_uuidv7(us / 1000, (us % 1000) * 1000 + ns % NS_PER_US);
> >
> > I think we can have an inline function or a marco (or use TMODULO()?)
> > to split nanoseconds into milliseconds and sub-milliseconds so that
> > uuidv7() and uuidv7_interval() can pass them to generate_uuidv7().
>
> I doubt that such macro will make core more readable. I've replaced 1000 with macros.
>
> >
> > The comments in uuidv7_interval() also need to be updated accordingly.
>
> Done.
>
> >
> > ---
> > I think we need to consider how we can handle the timestamp shifting.
> > UUIDv7 contains 48 bits Unix timestamp at milliseconds precision,
> > which can represent timestamps approximately between 2493 BC and 6432
> > AC. If users specify an interval to shift the timestamp beyond the
> > range, 48-bits timestamp would be wrapped around and they would not be
> > able to get an expected result. Do we need to raise an error in that
> > case?
> >
> > ---
> > Another problem I found in uuid_extract_timestamp() is that it cannot
> > correctly extract a timestamp before 1970/1/1 stored in a UUIDv7
> > value:
> >
> > postgres(1:1795331)=# select year, uuid_extract_timestamp(uuidv7((year
> > || 'year ago')::interval)) from generate_series(54, 56) year;
> > year | uuid_extract_timestamp
> > ------+-----------------------------
> > 54 | 1971-01-31 10:46:25.111-08
> > 55 | 1970-01-31 10:46:25.111-08
> > 56 | 10888-09-01 17:18:15.768-07
> > (3 rows)
> >
> > The problem is that we correctly store a negative timestamp value in a
> > UUIDv7 value but uuid_extract_timestamp() unconditionally treats it as
> > a positive timestamp value. I think this is a separate bug we need to
> > fix.
>
>
> RFC says unix_ts_ms is unsigned. So, luckily, no BC dates.
Good to know.
> I think it's totally fine to wrap UUID values around year 10598 without an error.
Okay.
>
> I was thinking about incorporating test like this.
>
> >> With this patch we can generate correct UUIDs in a very distant future.
> >> postgres=# select x, uuid_extract_timestamp(uuidv7((x::text || ' year'::text)::interval)),
> >> (x::text || ' year'::text)::interval
> >> from generate_series(1,9000,1000) x;
> >> x | uuid_extract_timestamp | interval
> >> ------+-----------------------------+------------
> >> 1 | 2026-01-31 12:00:53.084+05 | 1 year
> >> 1001 | 3026-01-31 12:00:53.084+05 | 1001 years
> >> 2001 | 4026-01-31 12:00:53.084+05 | 2001 years
> >> 3001 | 5026-01-31 12:00:53.084+05 | 3001 years
> >> 4001 | 6026-01-31 12:00:53.084+05 | 4001 years
> >> 5001 | 7026-01-31 12:00:53.085+05 | 5001 years
> >> 6001 | 8026-01-31 12:00:53.085+05 | 6001 years
> >> 7001 | 9026-01-31 12:00:53.085+05 | 7001 years
> >> 8001 | 10026-01-31 12:00:53.085+05 | 8001 years
> >> (9 rows)
>
>
> or maybe something simple like
>
> with u as (select uuidv7() id) select uuid_extract_timestamp(uuidv7('9999-09-09 12:34:56.789+05' - uuid_extract_timestamp(u.id))) from u;
>
> But it would still be flaky, second call to uuidv7() can overflow a millisecond.
>
Something like following queries might be workable for example?
create table test (c serial, d uuid, t timestamptz generated always as
(uuid_extract_timestamp(d)) stored);
insert into test (d) select uuidv7((n || 'years')::interval) from
generate_series(1, 2000) n;
select count(*) from (select t - lag(t) over (order by c) as diff from
test) where diff > '10 year' ;
Here are some review comments:
#define NS_PER_S INT64CONST(1000000000)
#define NS_PER_MS INT64CONST(1000000)
+#define US_PER_MS INT64CONST(1000)
#define NS_PER_US INT64CONST(1000)
I think it's clear if we put US_PER_MS below NS_PER_US.
---
*
- * ns is a number of nanoseconds since start of the UNIX epoch. This value is
+ * unix_ts_ms is a number of milliseconds since start of the UNIX epoch,
+ * ns_in_ms is a number of nanoseconds within millisecond. These values are
* used for time-dependent bits of UUID.
I think we can mention that the RFC describes that stored unix
timestamp as an unsigned integer.
---
static pg_uuid_t *
-generate_uuidv7(int64 ns)
+generate_uuidv7(uint64 unix_ts_ms, uint32 ns_in_ms)
How about renaming ns_in_ms with sub_ms?
---
+ /* 64 bits is enough for real time, but not for a time range of UUID */
I could not understand the point of this comment. It seems to say that
64-bits is not enough for a time range of UUID, but doesn't the time
range of UUIDv7 use only 48 bits? It seems to need more comments.
---
- ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
SECS_PER_DAY * USECS_PER_SEC)
- * NS_PER_US + ns % NS_PER_US;
+ us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
SECS_PER_DAY * USECS_PER_SEC);
/* Generate an UUIDv7 */
- uuid = generate_uuidv7(ns);
+ uuid = generate_uuidv7(us / US_PER_MS, (us % US_PER_MS) *
NS_PER_US + ns % NS_PER_US);
Need to update comments in uuidv7_internval() such as:
/*
* Shift the current timestamp by the given interval. To calculate time
* shift correctly, we convert the UNIX epoch to TimestampTz and use
* timestamptz_pl_interval(). Since this calculation is done with
* microsecond precision, we carry nanoseconds from original ns value to
* shifted ns value.
*/
and
/*
* Convert a TimestampTz value back to an UNIX epoch and back nanoseconds.
*/
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
@ 2025-02-09 17:07 ` Andrey Borodin <[email protected]>
2025-03-26 04:32 ` Re: UUID v7 Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Andrey Borodin @ 2025-02-09 17:07 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
I've took into account note from Sergey that "offset" is better name for uuidv7() argument than "shift".
> On 5 Feb 2025, at 03:02, Masahiko Sawada <[email protected]> wrote:
>
>>
>> I was thinking about incorporating test like this.
>>
>>>> With this patch we can generate correct UUIDs in a very distant future.
>>>> postgres=# select x, uuid_extract_timestamp(uuidv7((x::text || ' year'::text)::interval)),
>>>> (x::text || ' year'::text)::interval
>>>> from generate_series(1,9000,1000) x;
>>>> x | uuid_extract_timestamp | interval
>>>> ------+-----------------------------+------------
>>>> 1 | 2026-01-31 12:00:53.084+05 | 1 year
>>>> 1001 | 3026-01-31 12:00:53.084+05 | 1001 years
>>>> 2001 | 4026-01-31 12:00:53.084+05 | 2001 years
>>>> 3001 | 5026-01-31 12:00:53.084+05 | 3001 years
>>>> 4001 | 6026-01-31 12:00:53.084+05 | 4001 years
>>>> 5001 | 7026-01-31 12:00:53.085+05 | 5001 years
>>>> 6001 | 8026-01-31 12:00:53.085+05 | 6001 years
>>>> 7001 | 9026-01-31 12:00:53.085+05 | 7001 years
>>>> 8001 | 10026-01-31 12:00:53.085+05 | 8001 years
>>>> (9 rows)
>>
>
> Something like following queries might be workable for example?
>
> create table test (c serial, d uuid, t timestamptz generated always as
> (uuid_extract_timestamp(d)) stored);
> insert into test (d) select uuidv7((n || 'years')::interval) from
> generate_series(1, 2000) n;
> select count(*) from (select t - lag(t) over (order by c) as diff from
> test) where diff > '10 year' ;
Yeah, makes sense. I reduced tolerance to 366+1 day. Must be stable if we've done all the time offset business right.
> Here are some review comments:
>
> #define NS_PER_S INT64CONST(1000000000)
> #define NS_PER_MS INT64CONST(1000000)
> +#define US_PER_MS INT64CONST(1000)
> #define NS_PER_US INT64CONST(1000)
>
> I think it's clear if we put US_PER_MS below NS_PER_US.
OK.
> ---
> *
> - * ns is a number of nanoseconds since start of the UNIX epoch. This value is
> + * unix_ts_ms is a number of milliseconds since start of the UNIX epoch,
> + * ns_in_ms is a number of nanoseconds within millisecond. These values are
> * used for time-dependent bits of UUID.
>
> I think we can mention that the RFC describes that stored unix
> timestamp as an unsigned integer.
Done. Feel free to adjust my wordings, I've no sense of idiomatic English.
>
> ---
> static pg_uuid_t *
> -generate_uuidv7(int64 ns)
> +generate_uuidv7(uint64 unix_ts_ms, uint32 ns_in_ms)
>
> How about renaming ns_in_ms with sub_ms?
OK.
>
> ---
> + /* 64 bits is enough for real time, but not for a time range of UUID */
>
> I could not understand the point of this comment. It seems to say that
> 64-bits is not enough for a time range of UUID, but doesn't the time
> range of UUIDv7 use only 48 bits? It seems to need more comments.
I've tried to say that acquiring current time as an int64 ns since UNIX epoch is still viable for the code (until year 2262).
> ---
> - ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
> SECS_PER_DAY * USECS_PER_SEC)
> - * NS_PER_US + ns % NS_PER_US;
> + us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
> SECS_PER_DAY * USECS_PER_SEC);
>
> /* Generate an UUIDv7 */
> - uuid = generate_uuidv7(ns);
> + uuid = generate_uuidv7(us / US_PER_MS, (us % US_PER_MS) *
> NS_PER_US + ns % NS_PER_US);
>
> Need to update comments in uuidv7_internval() such as:
>
> /*
> * Shift the current timestamp by the given interval. To calculate time
> * shift correctly, we convert the UNIX epoch to TimestampTz and use
> * timestamptz_pl_interval(). Since this calculation is done with
> * microsecond precision, we carry nanoseconds from original ns value to
> * shifted ns value.
> */
>
> and
>
> /*
> * Convert a TimestampTz value back to an UNIX epoch and back nanoseconds.
> */
I've tried. I'm not very satisfied with comments, but could not come up with easier description.
Thanks!
Best regards, Andrey Borodin.
Attachments:
[application/octet-stream] v3-0001-UUDv7-fix-offset-computations-in-dates-after-2262.patch (8.6K, ../../[email protected]/2-v3-0001-UUDv7-fix-offset-computations-in-dates-after-2262.patch)
download | inline diff:
From cd78c0872ec9791e100e4569a980f5988ec5a13d Mon Sep 17 00:00:00 2001
From: Andrey Borodin <[email protected]>
Date: Fri, 31 Jan 2025 12:03:16 +0500
Subject: [PATCH v3] UUDv7: fix offset computations in dates after 2262
We used nanosecond representation of offsetted time values which
cannot be stored in 64-bit integer for dates significantly after
beginning of UNIX epoch. To prevent overflow we separate millisecond
part from nanoseconds, thus allowing us to store both parts in 64-bit
integers.
---
doc/src/sgml/func.sgml | 6 ++---
src/backend/utils/adt/uuid.c | 37 ++++++++++++++++--------------
src/include/catalog/pg_proc.dat | 2 +-
src/test/regress/expected/uuid.out | 15 +++++++++++-
src/test/regress/sql/uuid.sql | 12 +++++++++-
5 files changed, 49 insertions(+), 23 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 7efc81936a..8cf3e374b8 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -14330,12 +14330,12 @@ CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple
</synopsis>
These functions return a version 4 (random) UUID.
<synopsis>
-<function>uuidv7</function> (<optional> <parameter>shift</parameter> <type>interval</type> </optional>) <returnvalue>uuid</returnvalue>
+<function>uuidv7</function> (<optional> <parameter>offset</parameter> <type>interval</type> </optional>) <returnvalue>uuid</returnvalue>
</synopsis>
This function returns a version 7 UUID (UNIX timestamp with millisecond
precision + sub-millisecond timestamp + random). This function can accept
- optional <parameter>shift</parameter> parameter of type <type>interval</type>
- which shift internal timestamp by the given interval.
+ optional <parameter>offset</parameter> parameter of type <type>interval</type>
+ which offset internal timestamp by the given interval.
</para>
<para>
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 4f8402ef92..f368081cc4 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -29,6 +29,7 @@
#define NS_PER_S INT64CONST(1000000000)
#define NS_PER_MS INT64CONST(1000000)
#define NS_PER_US INT64CONST(1000)
+#define US_PER_MS INT64CONST(1000)
/*
* UUID version 7 uses 12 bits in "rand_a" to store 1/4096 (or 2^12) fractions of
@@ -69,6 +70,7 @@ static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
static inline int64 get_real_time_ns_ascending();
+static pg_uuid_t *generate_uuidv7(uint64 unix_ts_ms, uint32 sub_ms);
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -523,18 +525,18 @@ get_real_time_ns_ascending()
* described in the RFC. This method utilizes 12 bits from the "rand_a" bits
* to store a 1/4096 (or 2^12) fraction of sub-millisecond precision.
*
- * ns is a number of nanoseconds since start of the UNIX epoch. This value is
+ * unix_ts_ms is a number of milliseconds since start of the UNIX epoch,
+ * sub_ms is a number of nanoseconds within millisecond. These values are
* used for time-dependent bits of UUID.
+ *
+ * NB: all numbers here are unsigned, unix_ts_ms cannot be negative per RFC.
*/
static pg_uuid_t *
-generate_uuidv7(int64 ns)
+generate_uuidv7(uint64 unix_ts_ms, uint32 sub_ms)
{
pg_uuid_t *uuid = palloc(UUID_LEN);
- int64 unix_ts_ms;
int32 increased_clock_precision;
- unix_ts_ms = ns / NS_PER_MS;
-
/* Fill in time part */
uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
uuid->data[1] = (unsigned char) (unix_ts_ms >> 32);
@@ -547,7 +549,7 @@ generate_uuidv7(int64 ns)
* sub-millisecond timestamp fraction (SUBMS_BITS bits, not
* SUBMS_MINIMAL_STEP_BITS)
*/
- increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+ increased_clock_precision = ((sub_ms) * (1 << SUBMS_BITS)) / NS_PER_MS;
/* Fill the increased clock precision to "rand_a" bits */
uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
@@ -586,7 +588,8 @@ generate_uuidv7(int64 ns)
Datum
uuidv7(PG_FUNCTION_ARGS)
{
- pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
+ int64 ns = get_real_time_ns_ascending();
+ pg_uuid_t *uuid = generate_uuidv7(ns / NS_PER_MS, ns % NS_PER_MS);
PG_RETURN_UUID_P(uuid);
}
@@ -600,14 +603,17 @@ uuidv7_interval(PG_FUNCTION_ARGS)
Interval *shift = PG_GETARG_INTERVAL_P(0);
TimestampTz ts;
pg_uuid_t *uuid;
+ /*
+ * 64 bits is enough for ns in our centuries(until 2200-ies), but not for
+ * a whole time range of UUID (year 10888).
+ */
int64 ns = get_real_time_ns_ascending();
+ int64 us;
/*
* Shift the current timestamp by the given interval. To calculate time
* shift correctly, we convert the UNIX epoch to TimestampTz and use
- * timestamptz_pl_interval(). Since this calculation is done with
- * microsecond precision, we carry nanoseconds from original ns value to
- * shifted ns value.
+ * timestamptz_pl_interval().
*/
ts = (TimestampTz) (ns / NS_PER_US) -
@@ -618,14 +624,11 @@ uuidv7_interval(PG_FUNCTION_ARGS)
TimestampTzGetDatum(ts),
IntervalPGetDatum(shift)));
- /*
- * Convert a TimestampTz value back to an UNIX epoch and back nanoseconds.
- */
- ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
- * NS_PER_US + ns % NS_PER_US;
+ /* Convert a TimestampTz value back to an UNIX epoch in us */
+ us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC);
- /* Generate an UUIDv7 */
- uuid = generate_uuidv7(ns);
+ /* Generate an UUIDv7, not forgetting ns remainder */
+ uuid = generate_uuidv7(us / US_PER_MS, (us % US_PER_MS) * NS_PER_US + ns % NS_PER_US);
PG_RETURN_UUID_P(uuid);
}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5b8c2ad2a5..c28cace01a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9392,7 +9392,7 @@
proname => 'uuidv7', provolatile => 'v',
prorettype => 'uuid', proargtypes => '', prosrc => 'uuidv7' },
{ oid => '9897', descr => 'generate UUID version 7 with a timestamp shifted by specified interval',
- proname => 'uuidv7', provolatile => 'v', proargnames => '{shift}',
+ proname => 'uuidv7', provolatile => 'v', proargnames => '{offset}',
prorettype => 'uuid', proargtypes => 'interval', prosrc => 'uuidv7_interval' },
{ oid => '6342', descr => 'extract timestamp from UUID',
proname => 'uuid_extract_timestamp', proleakproof => 't',
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 798633ad51..96e93fbb28 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -290,5 +290,18 @@ SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
(1 row)
+-- offset generation
+CREATE TABLE guid4(c SERIAL, d uuid, t timestamptz generated always as
+(uuid_extract_timestamp(d)) stored);
+-- generate UUIDs up to year 10000
+INSERT INTO guid4 (d) SELECT uuidv7((n || 'years')::interval) FROM generate_series(1, 8000) n; -- should work fine until year 28888 = 10888 (end of UUIDv7) - 8000
+SELECT count(*) FROM
+ (SELECT t - lag(t) OVER (ORDER BY c) AS diff FROM guid4)
+WHERE diff > '367 days'; -- If UUIDs would be generated instantly and without overlap we would have up to '366 days'. One day is extra tolerance in case of machine stalls
+ count
+-------
+ 0
+(1 row)
+
-- clean up
-DROP TABLE guid1, guid2, guid3 CASCADE;
+DROP TABLE guid1, guid2, guid3, guid4 CASCADE;
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 110188361d..9b0bec4912 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -134,6 +134,16 @@ SELECT uuid_extract_timestamp('017F22E2-79B0-7CC3-98C4-DC0C0C07398F') = 'Tuesday
SELECT uuid_extract_timestamp(gen_random_uuid()); -- null
SELECT uuid_extract_timestamp('11111111-1111-1111-1111-111111111111'); -- null
+-- offset generation
+CREATE TABLE guid4(c SERIAL, d uuid, t timestamptz generated always as
+(uuid_extract_timestamp(d)) stored);
+
+-- generate UUIDs up to year 10000
+INSERT INTO guid4 (d) SELECT uuidv7((n || 'years')::interval) FROM generate_series(1, 8000) n; -- should work fine until year 28888 = 10888 (end of UUIDv7) - 8000
+
+SELECT count(*) FROM
+ (SELECT t - lag(t) OVER (ORDER BY c) AS diff FROM guid4)
+WHERE diff > '367 days'; -- If UUIDs would be generated instantly and without overlap we would have up to '366 days'. One day is extra tolerance in case of machine stalls
-- clean up
-DROP TABLE guid1, guid2, guid3 CASCADE;
+DROP TABLE guid1, guid2, guid3, guid4 CASCADE;
--
2.39.5 (Apple Git-154)
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-09 17:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2025-03-26 04:32 ` Masahiko Sawada <[email protected]>
2025-03-26 13:00 ` Re: UUID v7 Andrey Borodin <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Masahiko Sawada @ 2025-03-26 04:32 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Sun, Feb 9, 2025 at 9:07 AM Andrey Borodin <[email protected]> wrote:
>
> I've took into account note from Sergey that "offset" is better name for uuidv7() argument than "shift".
>
> > On 5 Feb 2025, at 03:02, Masahiko Sawada <[email protected]> wrote:
> >
> >>
> >> I was thinking about incorporating test like this.
> >>
> >>>> With this patch we can generate correct UUIDs in a very distant future.
> >>>> postgres=# select x, uuid_extract_timestamp(uuidv7((x::text || ' year'::text)::interval)),
> >>>> (x::text || ' year'::text)::interval
> >>>> from generate_series(1,9000,1000) x;
> >>>> x | uuid_extract_timestamp | interval
> >>>> ------+-----------------------------+------------
> >>>> 1 | 2026-01-31 12:00:53.084+05 | 1 year
> >>>> 1001 | 3026-01-31 12:00:53.084+05 | 1001 years
> >>>> 2001 | 4026-01-31 12:00:53.084+05 | 2001 years
> >>>> 3001 | 5026-01-31 12:00:53.084+05 | 3001 years
> >>>> 4001 | 6026-01-31 12:00:53.084+05 | 4001 years
> >>>> 5001 | 7026-01-31 12:00:53.085+05 | 5001 years
> >>>> 6001 | 8026-01-31 12:00:53.085+05 | 6001 years
> >>>> 7001 | 9026-01-31 12:00:53.085+05 | 7001 years
> >>>> 8001 | 10026-01-31 12:00:53.085+05 | 8001 years
> >>>> (9 rows)
> >>
> >
> > Something like following queries might be workable for example?
> >
> > create table test (c serial, d uuid, t timestamptz generated always as
> > (uuid_extract_timestamp(d)) stored);
> > insert into test (d) select uuidv7((n || 'years')::interval) from
> > generate_series(1, 2000) n;
> > select count(*) from (select t - lag(t) over (order by c) as diff from
> > test) where diff > '10 year' ;
>
> Yeah, makes sense. I reduced tolerance to 366+1 day. Must be stable if we've done all the time offset business right.
>
> > Here are some review comments:
> >
> > #define NS_PER_S INT64CONST(1000000000)
> > #define NS_PER_MS INT64CONST(1000000)
> > +#define US_PER_MS INT64CONST(1000)
> > #define NS_PER_US INT64CONST(1000)
> >
> > I think it's clear if we put US_PER_MS below NS_PER_US.
>
> OK.
>
> > ---
> > *
> > - * ns is a number of nanoseconds since start of the UNIX epoch. This value is
> > + * unix_ts_ms is a number of milliseconds since start of the UNIX epoch,
> > + * ns_in_ms is a number of nanoseconds within millisecond. These values are
> > * used for time-dependent bits of UUID.
> >
> > I think we can mention that the RFC describes that stored unix
> > timestamp as an unsigned integer.
>
> Done. Feel free to adjust my wordings, I've no sense of idiomatic English.
>
> >
> > ---
> > static pg_uuid_t *
> > -generate_uuidv7(int64 ns)
> > +generate_uuidv7(uint64 unix_ts_ms, uint32 ns_in_ms)
> >
> > How about renaming ns_in_ms with sub_ms?
>
> OK.
>
> >
> > ---
> > + /* 64 bits is enough for real time, but not for a time range of UUID */
> >
> > I could not understand the point of this comment. It seems to say that
> > 64-bits is not enough for a time range of UUID, but doesn't the time
> > range of UUIDv7 use only 48 bits? It seems to need more comments.
>
> I've tried to say that acquiring current time as an int64 ns since UNIX epoch is still viable for the code (until year 2262).
>
>
> > ---
> > - ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
> > SECS_PER_DAY * USECS_PER_SEC)
> > - * NS_PER_US + ns % NS_PER_US;
> > + us = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) *
> > SECS_PER_DAY * USECS_PER_SEC);
> >
> > /* Generate an UUIDv7 */
> > - uuid = generate_uuidv7(ns);
> > + uuid = generate_uuidv7(us / US_PER_MS, (us % US_PER_MS) *
> > NS_PER_US + ns % NS_PER_US);
> >
> > Need to update comments in uuidv7_internval() such as:
> >
> > /*
> > * Shift the current timestamp by the given interval. To calculate time
> > * shift correctly, we convert the UNIX epoch to TimestampTz and use
> > * timestamptz_pl_interval(). Since this calculation is done with
> > * microsecond precision, we carry nanoseconds from original ns value to
> > * shifted ns value.
> > */
> >
> > and
> >
> > /*
> > * Convert a TimestampTz value back to an UNIX epoch and back nanoseconds.
> > */
>
> I've tried. I'm not very satisfied with comments, but could not come up with easier description.
>
Thank you for updating the patch. I had missed to track this patch.
I've updated the patch from your v4 patch. In this version, I excluded
the argument name change (from 'shift' to 'offset') as it's not
related to the bug fix and simplified the regression test case.
Please review it.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
[application/octet-stream] v5-0001-Fix-timestamp-overflow-in-UUIDv7-implementation.patch (6.9K, ../../CAD21AoAVU-VnuqvdafqUCHCAo7Msb6_2k7nSzd_fijMbbtKrCg@mail.gmail.com/2-v5-0001-Fix-timestamp-overflow-in-UUIDv7-implementation.patch)
download | inline diff:
From 82ba268b7af93a75c76cf36a85c764761e0dbeb1 Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <[email protected]>
Date: Tue, 25 Mar 2025 15:14:42 -0700
Subject: [PATCH v5] Fix timestamp overflow in UUIDv7 implementation.
Previously, the uuidv7_interval() function performed timestamp
shifting calculations using microsecond precision, but then converted
the result back to nanosecond precision. Since the millisecond and
sub-millisecond parts were extracted from this nanosecond timestamp
and stored into the UUIDv7 value, overflow occurred for timestamps
beyond the year 2262.
With this commit, the millisecond and sub-millisecond parts are stored
directly into the UUIDv7 value without being converted back to a
nanosecond precision timestamp. Following RFC 9562, the timestamp is
stored as an unsigned integer, enabling support for dates up to the
year 10889.
Reported and fixed by Andrey Borodin, with cosmetic changes and
regression tests by me.
Reported-by: Andrey Borodin <[email protected]>
Author: Andrey Borodin <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/backend/utils/adt/uuid.c | 34 +++++++++++++++---------------
src/test/regress/expected/uuid.out | 14 ++++++++++++
src/test/regress/sql/uuid.sql | 11 ++++++++++
3 files changed, 42 insertions(+), 17 deletions(-)
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 4f8402ef925..be0f0f9f1ce 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -29,6 +29,7 @@
#define NS_PER_S INT64CONST(1000000000)
#define NS_PER_MS INT64CONST(1000000)
#define NS_PER_US INT64CONST(1000)
+#define US_PER_MS INT64CONST(1000)
/*
* UUID version 7 uses 12 bits in "rand_a" to store 1/4096 (or 2^12) fractions of
@@ -69,6 +70,7 @@ static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
static inline int64 get_real_time_ns_ascending();
+static pg_uuid_t *generate_uuidv7(uint64 unix_ts_ms, uint32 sub_ms);
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -523,17 +525,17 @@ get_real_time_ns_ascending()
* described in the RFC. This method utilizes 12 bits from the "rand_a" bits
* to store a 1/4096 (or 2^12) fraction of sub-millisecond precision.
*
- * ns is a number of nanoseconds since start of the UNIX epoch. This value is
+ * unix_ts_ms is a number of milliseconds since start of the UNIX epoch,
+ * and sub_ms is a number of nanoseconds within millisecond. These values are
* used for time-dependent bits of UUID.
+ *
+ * NB: all numbers here are unsigned, unix_ts_ms cannot be negative per RFC.
*/
static pg_uuid_t *
-generate_uuidv7(int64 ns)
+generate_uuidv7(uint64 unix_ts_ms, uint32 sub_ms)
{
pg_uuid_t *uuid = palloc(UUID_LEN);
- int64 unix_ts_ms;
- int32 increased_clock_precision;
-
- unix_ts_ms = ns / NS_PER_MS;
+ uint32 increased_clock_precision;
/* Fill in time part */
uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
@@ -547,7 +549,7 @@ generate_uuidv7(int64 ns)
* sub-millisecond timestamp fraction (SUBMS_BITS bits, not
* SUBMS_MINIMAL_STEP_BITS)
*/
- increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+ increased_clock_precision = (sub_ms * (1 << SUBMS_BITS)) / NS_PER_MS;
/* Fill the increased clock precision to "rand_a" bits */
uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
@@ -586,7 +588,8 @@ generate_uuidv7(int64 ns)
Datum
uuidv7(PG_FUNCTION_ARGS)
{
- pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
+ int64 ns = get_real_time_ns_ascending();
+ pg_uuid_t *uuid = generate_uuidv7(ns / NS_PER_MS, ns % NS_PER_MS);
PG_RETURN_UUID_P(uuid);
}
@@ -601,13 +604,13 @@ uuidv7_interval(PG_FUNCTION_ARGS)
TimestampTz ts;
pg_uuid_t *uuid;
int64 ns = get_real_time_ns_ascending();
+ int64 us;
/*
* Shift the current timestamp by the given interval. To calculate time
* shift correctly, we convert the UNIX epoch to TimestampTz and use
- * timestamptz_pl_interval(). Since this calculation is done with
- * microsecond precision, we carry nanoseconds from original ns value to
- * shifted ns value.
+ * timestamptz_pl_interval(). This calculation is done with microsecond
+ * precision.
*/
ts = (TimestampTz) (ns / NS_PER_US) -
@@ -618,14 +621,11 @@ uuidv7_interval(PG_FUNCTION_ARGS)
TimestampTzGetDatum(ts),
IntervalPGetDatum(shift)));
- /*
- * Convert a TimestampTz value back to an UNIX epoch and back nanoseconds.
- */
- ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
- * NS_PER_US + ns % NS_PER_US;
+ /* Convert a TimestampTz value back to an UNIX epoch timestamp */
+ us = ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
/* Generate an UUIDv7 */
- uuid = generate_uuidv7(ns);
+ uuid = generate_uuidv7(us / US_PER_MS, (us % US_PER_MS) * NS_PER_US + ns % NS_PER_US);
PG_RETURN_UUID_P(uuid);
}
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 798633ad51e..cbd497376c4 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -233,6 +233,20 @@ SELECT array_agg(id ORDER BY guid_field) FROM guid3;
{1,2,3,4,5,6,7,8,9,10}
(1 row)
+-- Check the timestamp offsets for v7.
+--
+-- generate UUIDv7 having timestamps up to 10889 year, which is the maximum year
+-- can be stored in UUIDv7, and then check if the timestamps extracted from UUIDv7
+-- values are not overflowed.
+WITH uuidts AS (
+ SELECT y, ts as ts, lag(ts) OVER (ORDER BY y) AS prev_ts
+ FROM (SELECT y, uuid_extract_timestamp(uuidv7((y || ' years')::interval)) AS ts FROM generate_series(-50, 10889 - extract(year from now())::int) y)
+)
+SELECT y, ts, prev_ts FROM uuidts WHERE ts < prev_ts;
+ y | ts | prev_ts
+---+----+---------
+(0 rows)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 110188361d1..cd0e65d3a8b 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -119,6 +119,17 @@ SELECT count(DISTINCT guid_field) FROM guid1;
INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
SELECT array_agg(id ORDER BY guid_field) FROM guid3;
+-- Check the timestamp offsets for v7.
+--
+-- generate UUIDv7 having timestamps up to 10889 year, which is the maximum year
+-- can be stored in UUIDv7, and then check if the timestamps extracted from UUIDv7
+-- values are not overflowed.
+WITH uuidts AS (
+ SELECT y, ts as ts, lag(ts) OVER (ORDER BY y) AS prev_ts
+ FROM (SELECT y, uuid_extract_timestamp(uuidv7((y || ' years')::interval)) AS ts FROM generate_series(-50, 10889 - extract(year from now())::int) y)
+)
+SELECT y, ts, prev_ts FROM uuidts WHERE ts < prev_ts;
+
-- extract functions
-- version
--
2.43.5
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-09 17:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 04:32 ` Re: UUID v7 Masahiko Sawada <[email protected]>
@ 2025-03-26 13:00 ` Andrey Borodin <[email protected]>
2025-03-26 17:06 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-03-26 19:32 ` Re: UUID v7 Andrei Borodin <[email protected]>
0 siblings, 2 replies; 68+ messages in thread
From: Andrey Borodin @ 2025-03-26 13:00 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
> On 26 Mar 2025, at 08:32, Masahiko Sawada <[email protected]> wrote:
>
> Please review it.
The patch looks good to me except one nit.
+WITH uuidts AS (
+ SELECT y, ts as ts, lag(ts) OVER (ORDER BY y) AS prev_ts
+ FROM (SELECT y, uuid_extract_timestamp(uuidv7((y || ' years')::interval)) AS ts FROM generate_series(-50, 10889 - extract(year from now())::int) y)
+)
+SELECT y, ts, prev_ts FROM uuidts WHERE ts < prev_ts;
if "extract(year from now())::int)" runs slightly before new year and the rest of the test after - the test will fail. How about avoiding overflow by using 10888 instead of 10889?
If we are sure citizen time never will go back, IMO we can safely move other border back to -55.
Also the test is not proof to NTP time drift during New Year's edge, but it's hardly a problem. The test, NTP clock sync and New Year millisecond must coincide for a false failure.
Thank you!
Best regards, Andrey Borodin.
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-09 17:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 04:32 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-03-26 13:00 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2025-03-26 17:06 ` Masahiko Sawada <[email protected]>
1 sibling, 0 replies; 68+ messages in thread
From: Masahiko Sawada @ 2025-03-26 17:06 UTC (permalink / raw)
To: Andrey Borodin <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Wed, Mar 26, 2025 at 6:00 AM Andrey Borodin <[email protected]> wrote:
>
>
>
> > On 26 Mar 2025, at 08:32, Masahiko Sawada <[email protected]> wrote:
> >
> > Please review it.
>
> The patch looks good to me except one nit.
> +WITH uuidts AS (
> + SELECT y, ts as ts, lag(ts) OVER (ORDER BY y) AS prev_ts
> + FROM (SELECT y, uuid_extract_timestamp(uuidv7((y || ' years')::interval)) AS ts FROM generate_series(-50, 10889 - extract(year from now())::int) y)
> +)
> +SELECT y, ts, prev_ts FROM uuidts WHERE ts < prev_ts;
>
> if "extract(year from now())::int)" runs slightly before new year and the rest of the test after - the test will fail. How about avoiding overflow by using 10888 instead of 10889?
Agreed. I've done this in the attached patch.
> If we are sure citizen time never will go back, IMO we can safely move other border back to -55.
Yes. Or I think we can verify the range from 1970 to 10888 like I did
in the updated patch.
> Also the test is not proof to NTP time drift during New Year's edge, but it's hardly a problem. The test, NTP clock sync and New Year millisecond must coincide for a false failure.
Agreed.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Attachments:
[application/octet-stream] v6-0001-Fix-timestamp-overflow-in-UUIDv7-implementation.patch (7.1K, ../../CAD21AoCr1aZdEBzDtU5Lg1XssA68L0uRyWC-a4DjmaXGLK++zA@mail.gmail.com/2-v6-0001-Fix-timestamp-overflow-in-UUIDv7-implementation.patch)
download | inline diff:
From 0baca061a9ff508b81d90c514c498d9cb03a63b2 Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <[email protected]>
Date: Tue, 25 Mar 2025 15:14:42 -0700
Subject: [PATCH v6] Fix timestamp overflow in UUIDv7 implementation.
Previously, the uuidv7_interval() function performed timestamp
shifting calculations using microsecond precision, but then converted
the result back to nanosecond precision. Since the millisecond and
sub-millisecond parts were extracted from this nanosecond timestamp
and stored into the UUIDv7 value, overflow occurred for timestamps
beyond the year 2262.
With this commit, the millisecond and sub-millisecond parts are stored
directly into the UUIDv7 value without being converted back to a
nanosecond precision timestamp. Following RFC 9562, the timestamp is
stored as an unsigned integer, enabling support for dates up to the
year 10889.
Reported and fixed by Andrey Borodin, with cosmetic changes and
regression tests by me.
Reported-by: Andrey Borodin <[email protected]>
Author: Andrey Borodin <[email protected]>
Discussion: https://postgr.es/m/[email protected]
---
src/backend/utils/adt/uuid.c | 34 +++++++++++++++---------------
src/test/regress/expected/uuid.out | 15 +++++++++++++
src/test/regress/sql/uuid.sql | 12 +++++++++++
3 files changed, 44 insertions(+), 17 deletions(-)
diff --git a/src/backend/utils/adt/uuid.c b/src/backend/utils/adt/uuid.c
index 4f8402ef925..be0f0f9f1ce 100644
--- a/src/backend/utils/adt/uuid.c
+++ b/src/backend/utils/adt/uuid.c
@@ -29,6 +29,7 @@
#define NS_PER_S INT64CONST(1000000000)
#define NS_PER_MS INT64CONST(1000000)
#define NS_PER_US INT64CONST(1000)
+#define US_PER_MS INT64CONST(1000)
/*
* UUID version 7 uses 12 bits in "rand_a" to store 1/4096 (or 2^12) fractions of
@@ -69,6 +70,7 @@ static bool uuid_abbrev_abort(int memtupcount, SortSupport ssup);
static Datum uuid_abbrev_convert(Datum original, SortSupport ssup);
static inline void uuid_set_version(pg_uuid_t *uuid, unsigned char version);
static inline int64 get_real_time_ns_ascending();
+static pg_uuid_t *generate_uuidv7(uint64 unix_ts_ms, uint32 sub_ms);
Datum
uuid_in(PG_FUNCTION_ARGS)
@@ -523,17 +525,17 @@ get_real_time_ns_ascending()
* described in the RFC. This method utilizes 12 bits from the "rand_a" bits
* to store a 1/4096 (or 2^12) fraction of sub-millisecond precision.
*
- * ns is a number of nanoseconds since start of the UNIX epoch. This value is
+ * unix_ts_ms is a number of milliseconds since start of the UNIX epoch,
+ * and sub_ms is a number of nanoseconds within millisecond. These values are
* used for time-dependent bits of UUID.
+ *
+ * NB: all numbers here are unsigned, unix_ts_ms cannot be negative per RFC.
*/
static pg_uuid_t *
-generate_uuidv7(int64 ns)
+generate_uuidv7(uint64 unix_ts_ms, uint32 sub_ms)
{
pg_uuid_t *uuid = palloc(UUID_LEN);
- int64 unix_ts_ms;
- int32 increased_clock_precision;
-
- unix_ts_ms = ns / NS_PER_MS;
+ uint32 increased_clock_precision;
/* Fill in time part */
uuid->data[0] = (unsigned char) (unix_ts_ms >> 40);
@@ -547,7 +549,7 @@ generate_uuidv7(int64 ns)
* sub-millisecond timestamp fraction (SUBMS_BITS bits, not
* SUBMS_MINIMAL_STEP_BITS)
*/
- increased_clock_precision = ((ns % NS_PER_MS) * (1 << SUBMS_BITS)) / NS_PER_MS;
+ increased_clock_precision = (sub_ms * (1 << SUBMS_BITS)) / NS_PER_MS;
/* Fill the increased clock precision to "rand_a" bits */
uuid->data[6] = (unsigned char) (increased_clock_precision >> 8);
@@ -586,7 +588,8 @@ generate_uuidv7(int64 ns)
Datum
uuidv7(PG_FUNCTION_ARGS)
{
- pg_uuid_t *uuid = generate_uuidv7(get_real_time_ns_ascending());
+ int64 ns = get_real_time_ns_ascending();
+ pg_uuid_t *uuid = generate_uuidv7(ns / NS_PER_MS, ns % NS_PER_MS);
PG_RETURN_UUID_P(uuid);
}
@@ -601,13 +604,13 @@ uuidv7_interval(PG_FUNCTION_ARGS)
TimestampTz ts;
pg_uuid_t *uuid;
int64 ns = get_real_time_ns_ascending();
+ int64 us;
/*
* Shift the current timestamp by the given interval. To calculate time
* shift correctly, we convert the UNIX epoch to TimestampTz and use
- * timestamptz_pl_interval(). Since this calculation is done with
- * microsecond precision, we carry nanoseconds from original ns value to
- * shifted ns value.
+ * timestamptz_pl_interval(). This calculation is done with microsecond
+ * precision.
*/
ts = (TimestampTz) (ns / NS_PER_US) -
@@ -618,14 +621,11 @@ uuidv7_interval(PG_FUNCTION_ARGS)
TimestampTzGetDatum(ts),
IntervalPGetDatum(shift)));
- /*
- * Convert a TimestampTz value back to an UNIX epoch and back nanoseconds.
- */
- ns = (ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC)
- * NS_PER_US + ns % NS_PER_US;
+ /* Convert a TimestampTz value back to an UNIX epoch timestamp */
+ us = ts + (POSTGRES_EPOCH_JDATE - UNIX_EPOCH_JDATE) * SECS_PER_DAY * USECS_PER_SEC;
/* Generate an UUIDv7 */
- uuid = generate_uuidv7(ns);
+ uuid = generate_uuidv7(us / US_PER_MS, (us % US_PER_MS) * NS_PER_US + ns % NS_PER_US);
PG_RETURN_UUID_P(uuid);
}
diff --git a/src/test/regress/expected/uuid.out b/src/test/regress/expected/uuid.out
index 798633ad51e..2e868457d63 100644
--- a/src/test/regress/expected/uuid.out
+++ b/src/test/regress/expected/uuid.out
@@ -233,6 +233,21 @@ SELECT array_agg(id ORDER BY guid_field) FROM guid3;
{1,2,3,4,5,6,7,8,9,10}
(1 row)
+-- Check the timestamp offsets for v7.
+--
+-- generate UUIDv7 values with timestamps ranging from 1970 (the Unix epoch year)
+-- to 10888 (one year before the maximum possible year), and then verify that
+-- the extracted timestamps from these UUIDv7 values have not overflowed.
+WITH uuidts AS (
+ SELECT y, ts as ts, lag(ts) OVER (ORDER BY y) AS prev_ts
+ FROM (SELECT y, uuid_extract_timestamp(uuidv7((y || ' years')::interval)) AS ts
+ FROM generate_series(1970 - extract(year from now())::int, 10888 - extract(year from now())::int) y)
+)
+SELECT y, ts, prev_ts FROM uuidts WHERE ts < prev_ts;
+ y | ts | prev_ts
+---+----+---------
+(0 rows)
+
-- extract functions
-- version
SELECT uuid_extract_version('11111111-1111-5111-8111-111111111111'); -- 5
diff --git a/src/test/regress/sql/uuid.sql b/src/test/regress/sql/uuid.sql
index 110188361d1..241d514eb9c 100644
--- a/src/test/regress/sql/uuid.sql
+++ b/src/test/regress/sql/uuid.sql
@@ -119,6 +119,18 @@ SELECT count(DISTINCT guid_field) FROM guid1;
INSERT INTO guid3 (guid_field) SELECT uuidv7() FROM generate_series(1, 10);
SELECT array_agg(id ORDER BY guid_field) FROM guid3;
+-- Check the timestamp offsets for v7.
+--
+-- generate UUIDv7 values with timestamps ranging from 1970 (the Unix epoch year)
+-- to 10888 (one year before the maximum possible year), and then verify that
+-- the extracted timestamps from these UUIDv7 values have not overflowed.
+WITH uuidts AS (
+ SELECT y, ts as ts, lag(ts) OVER (ORDER BY y) AS prev_ts
+ FROM (SELECT y, uuid_extract_timestamp(uuidv7((y || ' years')::interval)) AS ts
+ FROM generate_series(1970 - extract(year from now())::int, 10888 - extract(year from now())::int) y)
+)
+SELECT y, ts, prev_ts FROM uuidts WHERE ts < prev_ts;
+
-- extract functions
-- version
--
2.43.5
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-09 17:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 04:32 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-03-26 13:00 ` Re: UUID v7 Andrey Borodin <[email protected]>
@ 2025-03-26 19:32 ` Andrei Borodin <[email protected]>
2025-03-28 05:31 ` Re: UUID v7 Masahiko Sawada <[email protected]>
1 sibling, 1 reply; 68+ messages in thread
From: Andrei Borodin @ 2025-03-26 19:32 UTC (permalink / raw)
To: Masahiko Sawada <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
<br /><br />26.03.2025, 21:06, "Masahiko Sawada" <[email protected]>:<br /><blockquote><p>Agreed. I've done this in the attached patch.</p></blockquote>Great! The patch looks good to me.<div><br /></div><div>Best regards, Andrey Borodin.</div>
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-09 17:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 04:32 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-03-26 13:00 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 19:32 ` Re: UUID v7 Andrei Borodin <[email protected]>
@ 2025-03-28 05:31 ` Masahiko Sawada <[email protected]>
2025-03-29 00:09 ` Re: UUID v7 Masahiko Sawada <[email protected]>
0 siblings, 1 reply; 68+ messages in thread
From: Masahiko Sawada @ 2025-03-28 05:31 UTC (permalink / raw)
To: Andrei Borodin <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Wed, Mar 26, 2025 at 12:32 PM Andrei Borodin <[email protected]> wrote:
>
>
>
> 26.03.2025, 21:06, "Masahiko Sawada" <[email protected]>:
>
> Agreed. I've done this in the attached patch.
>
> Great! The patch looks good to me.
Thank you for reviewing it. I'm going to push the fix tomorrow,
barring further comments.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 68+ messages in thread
* Re: UUID v7
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-09 17:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 04:32 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-03-26 13:00 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 19:32 ` Re: UUID v7 Andrei Borodin <[email protected]>
2025-03-28 05:31 ` Re: UUID v7 Masahiko Sawada <[email protected]>
@ 2025-03-29 00:09 ` Masahiko Sawada <[email protected]>
0 siblings, 0 replies; 68+ messages in thread
From: Masahiko Sawada @ 2025-03-29 00:09 UTC (permalink / raw)
To: Andrei Borodin <[email protected]>; +Cc: Daniel Verite <[email protected]>; Peter Eisentraut <[email protected]>; Jelte Fennema-Nio <[email protected]>; Sergey Prokhorenko <[email protected]>; Przemysław Sztoch <[email protected]>; Michael Paquier <[email protected]>; Aleksander Alekseev <[email protected]>; pgsql-hackers; David G. Johnston <[email protected]>; Mat Arye <[email protected]>; Matthias van de Meent <[email protected]>; Nikolay Samokhvalov <[email protected]>; Junwang Zhao <[email protected]>; Stepan Neretin <[email protected]>
On Thu, Mar 27, 2025 at 10:31 PM Masahiko Sawada <[email protected]> wrote:
>
> On Wed, Mar 26, 2025 at 12:32 PM Andrei Borodin <[email protected]> wrote:
> >
> >
> >
> > 26.03.2025, 21:06, "Masahiko Sawada" <[email protected]>:
> >
> > Agreed. I've done this in the attached patch.
> >
> > Great! The patch looks good to me.
>
> Thank you for reviewing it. I'm going to push the fix tomorrow,
> barring further comments.
Pushed.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 68+ messages in thread
end of thread, other threads:[~2025-03-29 00:09 UTC | newest]
Thread overview: 68+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-06-09 06:44 [PATCH v1] Common Era for dates David Fetter <[email protected]>
2024-01-18 13:17 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 20:26 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-19 02:58 ` Re: UUID v7 Lukas Fittl <[email protected]>
2024-01-18 14:20 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 15:39 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 16:21 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-18 18:28 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-18 18:31 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-18 18:59 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-19 03:24 ` Re: UUID v7 David G. Johnston <[email protected]>
2024-01-19 08:25 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-19 18:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-22 04:22 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 04:24 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-22 15:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 12:40 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:02 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 13:29 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 15:29 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 15:46 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 16:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-24 17:00 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
2024-01-24 17:51 ` Re: UUID v7 Andrey Borodin <[email protected]>
2024-01-24 20:47 ` Re: UUID v7 Marcos Pegoraro <[email protected]>
2024-01-24 21:15 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-24 21:30 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-24 13:16 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-24 21:49 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 04:40 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 04:41 ` Re: UUID v7 Nikolay Samokhvalov <[email protected]>
2024-01-25 08:09 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-25 11:14 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-25 12:06 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-25 12:31 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-01-29 12:01 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-25 17:04 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-29 11:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-29 13:58 ` Re: UUID v7 Junwang Zhao <[email protected]>
2024-01-29 18:32 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-29 20:38 ` Re: UUID v7 Jelte Fennema-Nio <[email protected]>
2024-01-30 06:54 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 07:28 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-30 09:56 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 10:33 ` Re: UUID v7 Junwang Zhao <[email protected]>
2024-01-30 13:35 ` Re: UUID v7 Andrey M. Borodin <[email protected]>
2024-01-30 18:37 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-30 00:27 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-28 12:42 ` Re: UUID v7 Sergey Prokhorenko <[email protected]>
2024-01-18 20:49 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-18 20:39 ` Re: UUID v7 Przemysław Sztoch <[email protected]>
2024-01-19 11:07 ` Re: UUID v7 Aleksander Alekseev <[email protected]>
2024-12-12 18:08 Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-30 08:59 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-30 19:54 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-01-31 07:08 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-01-31 18:49 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-02 10:15 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-02-04 22:02 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-02-09 17:07 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 04:32 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-03-26 13:00 ` Re: UUID v7 Andrey Borodin <[email protected]>
2025-03-26 17:06 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-03-26 19:32 ` Re: UUID v7 Andrei Borodin <[email protected]>
2025-03-28 05:31 ` Re: UUID v7 Masahiko Sawada <[email protected]>
2025-03-29 00:09 ` Re: UUID v7 Masahiko Sawada <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox