agora inbox for [email protected]
help / color / mirror / Atom feedChange pgarch_readyXlog() to return .history files first
35+ messages / 12 participants
[nested] [flat]
* Change pgarch_readyXlog() to return .history files first
@ 2018-12-13 16:53 David Steele <[email protected]>
0 siblings, 2 replies; 35+ messages in thread
From: David Steele @ 2018-12-13 16:53 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Michael Paquier <[email protected]>
Hackers,
The alphabetical ordering of pgarch_readyXlog() means that on promotion
000000010000000100000001.partial will be archived before 00000002.history.
This appears harmless, but the .history files are what other potential
primaries use to decide what timeline they should pick. The additional
latency of compressing/transferring the much larger partial file means
that archiving of the .history file is delayed and greatly increases the
chance that another primary will promote to the same timeline.
Teach pgarch_readyXlog() to return .history files first (and in order)
to reduce the window where this can happen. This won't prevent all
conflicts, but it is a simple change and should greatly reduce
real-world occurrences.
I also think we should consider back-patching this change. It's hard to
imagine that archive commands would have trouble with this reordering
and the current ordering causes real pain in HA clusters.
Regards,
--
-David
[email protected]
From 2279697ce828e825066065e3e40c9926633523ad Mon Sep 17 00:00:00 2001
From: David Steele <[email protected]>
Date: Thu, 13 Dec 2018 11:00:37 -0500
Subject: [PATCH] Change pgarch_readyXlog() to return .history files first
The alphabetical ordering of pgarch_readyXlog() means that on promotion
000000010000000100000001.partial will be archived before 00000002.history.
This appears harmless, but the .history files are what other potential primaries
use to decide what timeline they should pick. The additional latency of
compressing/transferring the much larger partial file means that archiving of
the .history file is delayed and greatly increases the chance that another
primary will promote to the same timeline.
Teach pgarch_readyXlog() to return .history files first (and in order) to reduce
the window where this can happen. This won't prevent all conflicts, but it is a
simple change and should greatly reduce real-world occurrences.
---
src/backend/postmaster/pgarch.c | 30 +++++++++++++++++++++++-------
1 file changed, 23 insertions(+), 7 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 0dcf609f19..5dbab6a3d7 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -704,11 +704,11 @@ pgarch_archiveXlog(char *xlog)
* 2) because the oldest ones will sooner become candidates for
* recycling at time of checkpoint
*
- * NOTE: the "oldest" comparison will presently consider all segments of
- * a timeline with a smaller ID to be older than all segments of a timeline
- * with a larger ID; the net result being that past timelines are given
- * higher priority for archiving. This seems okay, or at least not
- * obviously worth changing.
+ * NOTE: the "oldest" comparison will consider any .history file to be older
+ * than any other file except another .history file. Segments on a timeline
+ * with a smaller ID will be older than all segments on a timeline with a larger
+ * ID; the net result being that past timelines are given higher priority for
+ * archiving. This seems okay, or at least not obviously worth changing.
*/
static bool
pgarch_readyXlog(char *xlog)
@@ -724,6 +724,7 @@ pgarch_readyXlog(char *xlog)
DIR *rldir;
struct dirent *rlde;
bool found = false;
+ bool historyFound = false;
snprintf(XLogArchiveStatusDir, MAXPGPATH, XLOGDIR "/archive_status");
rldir = AllocateDir(XLogArchiveStatusDir);
@@ -737,12 +738,27 @@ pgarch_readyXlog(char *xlog)
strspn(rlde->d_name, VALID_XFN_CHARS) >= basenamelen &&
strcmp(rlde->d_name + basenamelen, ".ready") == 0)
{
- if (!found)
+ /* Is this a history file? */
+ bool history = basenamelen >= sizeof(".history") &&
+ strcmp(rlde->d_name + (basenamelen - sizeof(".history") + 1),
+ ".history.ready") == 0;
+
+ /*
+ * If this is the first file or the first history file, copy it
+ */
+ if (!found || history && !historyFound)
{
strcpy(newxlog, rlde->d_name);
found = true;
+ historyFound = history;
}
- else
+ /*
+ * Else compare and see if this file is alpabetically less than
+ * what we already have. If so, copy it. History files always get
+ * this comparison but other files only do when no history file has
+ * been found yet.
+ */
+ else if (history || !history && !historyFound)
{
if (strcmp(rlde->d_name, newxlog) < 0)
strcpy(newxlog, rlde->d_name);
--
2.17.2 (Apple Git-113)
Attachments:
[text/plain] history-files-first-v1.patch (3.4K, ../../[email protected]/2-history-files-first-v1.patch)
download | inline diff:
From 2279697ce828e825066065e3e40c9926633523ad Mon Sep 17 00:00:00 2001
From: David Steele <[email protected]>
Date: Thu, 13 Dec 2018 11:00:37 -0500
Subject: [PATCH] Change pgarch_readyXlog() to return .history files first
The alphabetical ordering of pgarch_readyXlog() means that on promotion
000000010000000100000001.partial will be archived before 00000002.history.
This appears harmless, but the .history files are what other potential primaries
use to decide what timeline they should pick. The additional latency of
compressing/transferring the much larger partial file means that archiving of
the .history file is delayed and greatly increases the chance that another
primary will promote to the same timeline.
Teach pgarch_readyXlog() to return .history files first (and in order) to reduce
the window where this can happen. This won't prevent all conflicts, but it is a
simple change and should greatly reduce real-world occurrences.
---
src/backend/postmaster/pgarch.c | 30 +++++++++++++++++++++++-------
1 file changed, 23 insertions(+), 7 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 0dcf609f19..5dbab6a3d7 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -704,11 +704,11 @@ pgarch_archiveXlog(char *xlog)
* 2) because the oldest ones will sooner become candidates for
* recycling at time of checkpoint
*
- * NOTE: the "oldest" comparison will presently consider all segments of
- * a timeline with a smaller ID to be older than all segments of a timeline
- * with a larger ID; the net result being that past timelines are given
- * higher priority for archiving. This seems okay, or at least not
- * obviously worth changing.
+ * NOTE: the "oldest" comparison will consider any .history file to be older
+ * than any other file except another .history file. Segments on a timeline
+ * with a smaller ID will be older than all segments on a timeline with a larger
+ * ID; the net result being that past timelines are given higher priority for
+ * archiving. This seems okay, or at least not obviously worth changing.
*/
static bool
pgarch_readyXlog(char *xlog)
@@ -724,6 +724,7 @@ pgarch_readyXlog(char *xlog)
DIR *rldir;
struct dirent *rlde;
bool found = false;
+ bool historyFound = false;
snprintf(XLogArchiveStatusDir, MAXPGPATH, XLOGDIR "/archive_status");
rldir = AllocateDir(XLogArchiveStatusDir);
@@ -737,12 +738,27 @@ pgarch_readyXlog(char *xlog)
strspn(rlde->d_name, VALID_XFN_CHARS) >= basenamelen &&
strcmp(rlde->d_name + basenamelen, ".ready") == 0)
{
- if (!found)
+ /* Is this a history file? */
+ bool history = basenamelen >= sizeof(".history") &&
+ strcmp(rlde->d_name + (basenamelen - sizeof(".history") + 1),
+ ".history.ready") == 0;
+
+ /*
+ * If this is the first file or the first history file, copy it
+ */
+ if (!found || history && !historyFound)
{
strcpy(newxlog, rlde->d_name);
found = true;
+ historyFound = history;
}
- else
+ /*
+ * Else compare and see if this file is alpabetically less than
+ * what we already have. If so, copy it. History files always get
+ * this comparison but other files only do when no history file has
+ * been found yet.
+ */
+ else if (history || !history && !historyFound)
{
if (strcmp(rlde->d_name, newxlog) < 0)
strcpy(newxlog, rlde->d_name);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-13 18:45 David Steele <[email protected]>
parent: David Steele <[email protected]>
1 sibling, 0 replies; 35+ messages in thread
From: David Steele @ 2018-12-13 18:45 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Michael Paquier <[email protected]>
On 12/13/18 11:53 AM, David Steele wrote:
> Hackers,
>
> The alphabetical ordering of pgarch_readyXlog() means that on promotion
> 000000010000000100000001.partial will be archived before 00000002.history.
>
> This appears harmless, but the .history files are what other potential
> primaries use to decide what timeline they should pick. The additional
> latency of compressing/transferring the much larger partial file means
> that archiving of the .history file is delayed and greatly increases the
> chance that another primary will promote to the same timeline.
>
> Teach pgarch_readyXlog() to return .history files first (and in order)
> to reduce the window where this can happen. This won't prevent all
> conflicts, but it is a simple change and should greatly reduce
> real-world occurrences.
>
> I also think we should consider back-patching this change. It's hard to
> imagine that archive commands would have trouble with this reordering
> and the current ordering causes real pain in HA clusters.
Some gcc versions wanted more parens, so updated in attached.
--
-David
[email protected]
From d5fd60b4e329fe99b2f6565355014040dcdcee26 Mon Sep 17 00:00:00 2001
From: David Steele <[email protected]>
Date: Thu, 13 Dec 2018 12:58:27 -0500
Subject: [PATCH] Change pgarch_readyXlog() to return .history files first.
The alphabetical ordering of pgarch_readyXlog() means that on promotion
000000010000000100000001.partial will be archived before 00000002.history.
This appears harmless, but the .history files are what other potential primaries
use to decide what timeline they should pick. The additional latency of
compressing/transferring the much larger partial file means that archiving of
the .history file is delayed and greatly increases the chance that another
primary will promote to the same timeline.
Teach pgarch_readyXlog() to return .history files first (and in order) to reduce
the window where this can happen. This won't prevent all conflicts, but it is a
simple change and should greatly reduce real-world occurrences.
---
src/backend/postmaster/pgarch.c | 30 +++++++++++++++++++++++-------
1 file changed, 23 insertions(+), 7 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 0dcf609f19..95898080d7 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -704,11 +704,11 @@ pgarch_archiveXlog(char *xlog)
* 2) because the oldest ones will sooner become candidates for
* recycling at time of checkpoint
*
- * NOTE: the "oldest" comparison will presently consider all segments of
- * a timeline with a smaller ID to be older than all segments of a timeline
- * with a larger ID; the net result being that past timelines are given
- * higher priority for archiving. This seems okay, or at least not
- * obviously worth changing.
+ * NOTE: the "oldest" comparison will consider any .history file to be older
+ * than any other file except another .history file. Segments on a timeline
+ * with a smaller ID will be older than all segments on a timeline with a larger
+ * ID; the net result being that past timelines are given higher priority for
+ * archiving. This seems okay, or at least not obviously worth changing.
*/
static bool
pgarch_readyXlog(char *xlog)
@@ -724,6 +724,7 @@ pgarch_readyXlog(char *xlog)
DIR *rldir;
struct dirent *rlde;
bool found = false;
+ bool historyFound = false;
snprintf(XLogArchiveStatusDir, MAXPGPATH, XLOGDIR "/archive_status");
rldir = AllocateDir(XLogArchiveStatusDir);
@@ -737,12 +738,27 @@ pgarch_readyXlog(char *xlog)
strspn(rlde->d_name, VALID_XFN_CHARS) >= basenamelen &&
strcmp(rlde->d_name + basenamelen, ".ready") == 0)
{
- if (!found)
+ /* Is this a history file? */
+ bool history = basenamelen >= sizeof(".history") &&
+ strcmp(rlde->d_name + (basenamelen - sizeof(".history") + 1),
+ ".history.ready") == 0;
+
+ /*
+ * If this is the first file or the first history file, copy it
+ */
+ if (!found || (history && !historyFound))
{
strcpy(newxlog, rlde->d_name);
found = true;
+ historyFound = history;
}
- else
+ /*
+ * Else compare and see if this file is alpabetically less than
+ * what we already have. If so, copy it. History files always get
+ * this comparison but other files only do when no history file has
+ * been found yet.
+ */
+ else if (history || (!history && !historyFound))
{
if (strcmp(rlde->d_name, newxlog) < 0)
strcpy(newxlog, rlde->d_name);
--
2.17.2 (Apple Git-113)
Attachments:
[text/plain] history-files-first-v2.patch (3.4K, ../../[email protected]/2-history-files-first-v2.patch)
download | inline diff:
From d5fd60b4e329fe99b2f6565355014040dcdcee26 Mon Sep 17 00:00:00 2001
From: David Steele <[email protected]>
Date: Thu, 13 Dec 2018 12:58:27 -0500
Subject: [PATCH] Change pgarch_readyXlog() to return .history files first.
The alphabetical ordering of pgarch_readyXlog() means that on promotion
000000010000000100000001.partial will be archived before 00000002.history.
This appears harmless, but the .history files are what other potential primaries
use to decide what timeline they should pick. The additional latency of
compressing/transferring the much larger partial file means that archiving of
the .history file is delayed and greatly increases the chance that another
primary will promote to the same timeline.
Teach pgarch_readyXlog() to return .history files first (and in order) to reduce
the window where this can happen. This won't prevent all conflicts, but it is a
simple change and should greatly reduce real-world occurrences.
---
src/backend/postmaster/pgarch.c | 30 +++++++++++++++++++++++-------
1 file changed, 23 insertions(+), 7 deletions(-)
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 0dcf609f19..95898080d7 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -704,11 +704,11 @@ pgarch_archiveXlog(char *xlog)
* 2) because the oldest ones will sooner become candidates for
* recycling at time of checkpoint
*
- * NOTE: the "oldest" comparison will presently consider all segments of
- * a timeline with a smaller ID to be older than all segments of a timeline
- * with a larger ID; the net result being that past timelines are given
- * higher priority for archiving. This seems okay, or at least not
- * obviously worth changing.
+ * NOTE: the "oldest" comparison will consider any .history file to be older
+ * than any other file except another .history file. Segments on a timeline
+ * with a smaller ID will be older than all segments on a timeline with a larger
+ * ID; the net result being that past timelines are given higher priority for
+ * archiving. This seems okay, or at least not obviously worth changing.
*/
static bool
pgarch_readyXlog(char *xlog)
@@ -724,6 +724,7 @@ pgarch_readyXlog(char *xlog)
DIR *rldir;
struct dirent *rlde;
bool found = false;
+ bool historyFound = false;
snprintf(XLogArchiveStatusDir, MAXPGPATH, XLOGDIR "/archive_status");
rldir = AllocateDir(XLogArchiveStatusDir);
@@ -737,12 +738,27 @@ pgarch_readyXlog(char *xlog)
strspn(rlde->d_name, VALID_XFN_CHARS) >= basenamelen &&
strcmp(rlde->d_name + basenamelen, ".ready") == 0)
{
- if (!found)
+ /* Is this a history file? */
+ bool history = basenamelen >= sizeof(".history") &&
+ strcmp(rlde->d_name + (basenamelen - sizeof(".history") + 1),
+ ".history.ready") == 0;
+
+ /*
+ * If this is the first file or the first history file, copy it
+ */
+ if (!found || (history && !historyFound))
{
strcpy(newxlog, rlde->d_name);
found = true;
+ historyFound = history;
}
- else
+ /*
+ * Else compare and see if this file is alpabetically less than
+ * what we already have. If so, copy it. History files always get
+ * this comparison but other files only do when no history file has
+ * been found yet.
+ */
+ else if (history || (!history && !historyFound))
{
if (strcmp(rlde->d_name, newxlog) < 0)
strcpy(newxlog, rlde->d_name);
--
2.17.2 (Apple Git-113)
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-14 00:15 Michael Paquier <[email protected]>
parent: David Steele <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Michael Paquier @ 2018-12-14 00:15 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: pgsql-hackers
On Thu, Dec 13, 2018 at 11:53:53AM -0500, David Steele wrote:
> I also think we should consider back-patching this change. It's hard to
> imagine that archive commands would have trouble with this reordering
> and the current ordering causes real pain in HA clusters.
I would like to hear opinion from other though if we should consider
that as an improvement or an actual bug fix. Changing the order of the
files to map with what the startup process does when promoting does not
sound like a bug fix to me, still this is not really invasive, so we
could really consider it worth back-patching to reduce common pain from
users when it comes to timeline handling.
> - if (!found)
> + /* Is this a history file? */
> + bool history = basenamelen >= sizeof(".history") &&
> + strcmp(rlde->d_name + (basenamelen - sizeof(".history") + 1),
> + ".history.ready") == 0;
Or you could just use IsTLHistoryFileName here?
If one wants to simply check this code, you can just create dummy orphan
files in archive_status and see in which order they get removed.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-14 13:43 David Steele <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: David Steele @ 2018-12-14 13:43 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: pgsql-hackers
On 12/13/18 7:15 PM, Michael Paquier wrote:
> On Thu, Dec 13, 2018 at 11:53:53AM -0500, David Steele wrote:
>> I also think we should consider back-patching this change. It's hard to
>> imagine that archive commands would have trouble with this reordering
>> and the current ordering causes real pain in HA clusters.
>
> I would like to hear opinion from other though if we should consider
> that as an improvement or an actual bug fix. Changing the order of the
> files to map with what the startup process does when promoting does not
> sound like a bug fix to me, still this is not really invasive, so we
> could really consider it worth back-patching to reduce common pain from
> users when it comes to timeline handling.
I think an argument can be made that it is a bug (ish). Postgres
generates the files in one order, and they get archived in a different
order.
>> - if (!found)
>> + /* Is this a history file? */
>> + bool history = basenamelen >= sizeof(".history") &&
>> + strcmp(rlde->d_name + (basenamelen - sizeof(".history") + 1),
>> + ".history.ready") == 0;
>
> Or you could just use IsTLHistoryFileName here?
We'd have to truncate .ready off the string to make that work, which
seems easy enough. Is that what you were thinking?
One thing to consider is the check above is more efficient than
IsTLHistoryFileName() and it potentially gets run a lot.
> If one wants to simply check this code, you can just create dummy orphan
> files in archive_status and see in which order they get removed.
Seems awfully racy. Are there currently any tests like this for the
archiver that I can look at extending?
Regards,
--
-David
[email protected]
Attachments:
[application/pgp-signature] signature.asc (874B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-15 00:10 Michael Paquier <[email protected]>
parent: David Steele <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Michael Paquier @ 2018-12-15 00:10 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: pgsql-hackers
On Fri, Dec 14, 2018 at 08:43:20AM -0500, David Steele wrote:
> On 12/13/18 7:15 PM, Michael Paquier wrote:
>> I would like to hear opinion from other though if we should consider
>> that as an improvement or an actual bug fix. Changing the order of the
>> files to map with what the startup process does when promoting does not
>> sound like a bug fix to me, still this is not really invasive, so we
>> could really consider it worth back-patching to reduce common pain from
>> users when it comes to timeline handling.
>
> I think an argument can be made that it is a bug (ish). Postgres
> generates the files in one order, and they get archived in a different
> order.
I am not completely sure either. In my experience, if there is any
doubt on such definitions the best answer is to not backpatch.
>> Or you could just use IsTLHistoryFileName here?
>
> We'd have to truncate .ready off the string to make that work, which
> seems easy enough. Is that what you were thinking?
Yes, that's the idea. pgarch_readyXlog returns the segment name which
should be archived, so you could just compute it after detecting a
.ready file.
> One thing to consider is the check above is more efficient than
> IsTLHistoryFileName() and it potentially gets run a lot.
This check misses strspn(), so any file finishing with .history would
get eaten even if that's unlikely to happen.
>> If one wants to simply check this code, you can just create dummy orphan
>> files in archive_status and see in which order they get removed.
>
> Seems awfully racy. Are there currently any tests like this for the
> archiver that I can look at extending?
Sorry for the confusion, I was referring to manual testing here.
Thinking about it, we could have an automatic test to check for the file
order pattern by creating dummy files, starting the server with archiver
enabled, and then parse the logs as orphan .ready files would get
removed in the order their archiving is attempted with one WARNING entry
generated for each. I am not sure if that is worth a test though.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-20 11:57 David Steele <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: David Steele @ 2018-12-20 11:57 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: pgsql-hackers
On 12/15/18 2:10 AM, Michael Paquier wrote:
> On Fri, Dec 14, 2018 at 08:43:20AM -0500, David Steele wrote:
>> On 12/13/18 7:15 PM, Michael Paquier wrote:
>>> Or you could just use IsTLHistoryFileName here?
>>
>> We'd have to truncate .ready off the string to make that work, which
>> seems easy enough. Is that what you were thinking?
>
> Yes, that's the idea. pgarch_readyXlog returns the segment name which
> should be archived, so you could just compute it after detecting a
> .ready file.
>
>> One thing to consider is the check above is more efficient than
>> IsTLHistoryFileName() and it potentially gets run a lot.
>
> This check misses strspn(), so any file finishing with .history would
> get eaten even if that's unlikely to happen.
Good point. The new patch uses IsTLHistoryFileName().
>>> If one wants to simply check this code, you can just create dummy orphan
>>> files in archive_status and see in which order they get removed.
>>
>> Seems awfully racy. Are there currently any tests like this for the
>> archiver that I can look at extending?
>
> Sorry for the confusion, I was referring to manual testing here.
Ah, I see. Yes, that's exactly how I tested it, in addition to doing
real promotions.
> Thinking about it, we could have an automatic test to check for the file
> order pattern by creating dummy files, starting the server with archiver
> enabled, and then parse the logs as orphan .ready files would get
> removed in the order their archiving is attempted with one WARNING entry
> generated for each. I am not sure if that is worth a test though.
Yes, parsing the logs was the best thing I could think of, too.
--
-David
[email protected]
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-21 03:19 Michael Paquier <[email protected]>
parent: David Steele <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Michael Paquier @ 2018-12-21 03:19 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: pgsql-hackers
On Thu, Dec 20, 2018 at 01:57:30PM +0200, David Steele wrote:
> Good point. The new patch uses IsTLHistoryFileName().
OK, I have been reviewing the patch and the logic is correct, still I
could not resist reducing the number of inner if's for readability. I
also did not like the high-jacking of rlde->d_name so instead let's use
an intermediate variable to store the basename of a scanned entry. The
format of the if/elif with comments in-between was not really consistent
with the common practice as well. pg_indent has also been applied.
> Ah, I see. Yes, that's exactly how I tested it, in addition to doing real
> promotions.
OK, so am I doing.
Attached is an updated patch. Does that look fine to you? The base
logic is unchanged, and after a promotion history files get archived
before the last partial segment.
--
Michael
Attachments:
[text/x-diff] history-files-first-v4.patch (3.3K, ../../[email protected]/2-history-files-first-v4.patch)
download | inline diff:
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index e88d545d65..bbd6311b35 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -695,11 +695,12 @@ pgarch_archiveXlog(char *xlog)
* 2) because the oldest ones will sooner become candidates for
* recycling at time of checkpoint
*
- * NOTE: the "oldest" comparison will presently consider all segments of
- * a timeline with a smaller ID to be older than all segments of a timeline
- * with a larger ID; the net result being that past timelines are given
- * higher priority for archiving. This seems okay, or at least not
- * obviously worth changing.
+ * NOTE: the "oldest" comparison will consider any .history file to be older
+ * than any other file except another .history file. Segments on a timeline
+ * with a smaller ID will be older than all segments on a timeline with a
+ * larger ID; the net result being that past timelines are given higher
+ * priority for archiving. This seems okay, or at least not obviously worth
+ * changing.
*/
static bool
pgarch_readyXlog(char *xlog)
@@ -715,6 +716,7 @@ pgarch_readyXlog(char *xlog)
DIR *rldir;
struct dirent *rlde;
bool found = false;
+ bool historyFound = false;
snprintf(XLogArchiveStatusDir, MAXPGPATH, XLOGDIR "/archive_status");
rldir = AllocateDir(XLogArchiveStatusDir);
@@ -722,32 +724,54 @@ pgarch_readyXlog(char *xlog)
while ((rlde = ReadDir(rldir, XLogArchiveStatusDir)) != NULL)
{
int basenamelen = (int) strlen(rlde->d_name) - 6;
+ char basename[MAX_XFN_CHARS + 1];
+ bool ishistory;
- if (basenamelen >= MIN_XFN_CHARS &&
- basenamelen <= MAX_XFN_CHARS &&
- strspn(rlde->d_name, VALID_XFN_CHARS) >= basenamelen &&
- strcmp(rlde->d_name + basenamelen, ".ready") == 0)
+ /* Ignore entries with unexpected number of characters */
+ if (basenamelen < MIN_XFN_CHARS ||
+ basenamelen > MAX_XFN_CHARS)
+ continue;
+
+ /* Ignore entries with unexpected characters */
+ if (strspn(rlde->d_name, VALID_XFN_CHARS) < basenamelen)
+ continue;
+
+ /* Ignore anything not suffixed with .ready */
+ if (strcmp(rlde->d_name + basenamelen, ".ready") != 0)
+ continue;
+
+ /* Truncate off the .ready */
+ memcpy(basename, rlde->d_name, basenamelen);
+ basename[basenamelen] = '\0';
+
+ /* Is this a history file? */
+ ishistory = IsTLHistoryFileName(basename);
+
+ /*
+ * Consume the file to archive. History files have the highest
+ * priority. If this is the first file or the first history file
+ * ever, copy it. In the presence of a history file already chosen as
+ * target, ignore all other files except history files which have been
+ * generated for an older timeline than what is already chosen as
+ * target to archive.
+ */
+ if (!found || (ishistory && !historyFound))
{
- if (!found)
- {
- strcpy(newxlog, rlde->d_name);
- found = true;
- }
- else
- {
- if (strcmp(rlde->d_name, newxlog) < 0)
- strcpy(newxlog, rlde->d_name);
- }
+ strcpy(newxlog, basename);
+ found = true;
+ historyFound = ishistory;
+ }
+ else if (ishistory || (!ishistory && !historyFound))
+ {
+ if (strcmp(basename, newxlog) < 0)
+ strcpy(newxlog, basename);
}
}
FreeDir(rldir);
if (found)
- {
- /* truncate off the .ready */
- newxlog[strlen(newxlog) - 6] = '\0';
strcpy(xlog, newxlog);
- }
+
return found;
}
[application/pgp-signature] signature.asc (833B, ../../[email protected]/3-signature.asc)
download
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-21 04:49 Kyotaro HORIGUCHI <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 2 replies; 35+ messages in thread
From: Kyotaro HORIGUCHI @ 2018-12-21 04:49 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; pgsql-hackers
Hello.
FWIW it seems to me a bug that making an inconsistent set of
files in archive directory.
At Fri, 21 Dec 2018 12:19:21 +0900, Michael Paquier <[email protected]> wrote in <[email protected]>
> On Thu, Dec 20, 2018 at 01:57:30PM +0200, David Steele wrote:
> > Good point. The new patch uses IsTLHistoryFileName().
>
> OK, I have been reviewing the patch and the logic is correct, still I
> could not resist reducing the number of inner if's for readability. I
+1 basically. But I think that tail(name, 6) != ".ready" can
happen with a certain frequency but strspn(name) < basenamelen
rarely in the normal case.
> also did not like the high-jacking of rlde->d_name so instead let's use
> an intermediate variable to store the basename of a scanned entry. The
> format of the if/elif with comments in-between was not really consistent
> with the common practice as well. pg_indent has also been applied.
>
> > Ah, I see. Yes, that's exactly how I tested it, in addition to doing real
> > promotions.
>
> OK, so am I doing.
>
> Attached is an updated patch. Does that look fine to you? The base
> logic is unchanged, and after a promotion history files get archived
> before the last partial segment.
Renaming history to ishistory looks good.
if (!found || (ishistory && !historyFound))
{
/* init */
found = true;
historyFound = ishistory;
}
else if (ishistory || (!ishstory && !historyFound))
/* compare/replace */
In the else if condition, ishisotry must be false in the right
hand of ||. What we do here is ignoring non-history files once
history file found. (Just a logic condensing and it would be done
by compiler, though)
"else if (!historyFound || ishistory)"
> strcpy(xlog, newxlog);
The caller prepares sufficient memory for basename, and we no
longer copy ".ready" into newxlog. Douldn't we work directly on
xlog instead of allocating newxlog?
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-21 05:17 Michael Paquier <[email protected]>
parent: Kyotaro HORIGUCHI <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Michael Paquier @ 2018-12-21 05:17 UTC (permalink / raw)
To: Kyotaro HORIGUCHI <[email protected]>; +Cc: [email protected]; pgsql-hackers
On Fri, Dec 21, 2018 at 01:49:18PM +0900, Kyotaro HORIGUCHI wrote:
> FWIW it seems to me a bug that making an inconsistent set of
> files in archive directory.
Okay, point taken! FWIW, I have no actual objections in not
back-patching that.
> At Fri, 21 Dec 2018 12:19:21 +0900, Michael Paquier <[email protected]> wrote in <[email protected]>
>> On Thu, Dec 20, 2018 at 01:57:30PM +0200, David Steele wrote:
>> > Good point. The new patch uses IsTLHistoryFileName().
>>
>> OK, I have been reviewing the patch and the logic is correct, still I
>> could not resist reducing the number of inner if's for readability. I
>
> +1 basically. But I think that tail(name, 6) != ".ready" can
> happen with a certain frequency but strspn(name) < basenamelen
> rarely in the normal case.
So that +0.5 if "basically" means a partial agreement? :p
> In the else if condition, ishisotry must be false in the right
> hand of ||. What we do here is ignoring non-history files once
> history file found. (Just a logic condensing and it would be done
> by compiler, though)
Yes, this can be simplified. So let's do so.
> The caller prepares sufficient memory for basename, and we no
> longer copy ".ready" into newxlog. Wouldn't we work directly on
> xlog instead of allocating newxlog?
Okay, let's simplify that as you suggest.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-21 05:23 Kyotaro HORIGUCHI <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Kyotaro HORIGUCHI @ 2018-12-21 05:23 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; pgsql-hackers
At Fri, 21 Dec 2018 14:17:25 +0900, Michael Paquier <[email protected]> wrote in <[email protected]>
> On Fri, Dec 21, 2018 at 01:49:18PM +0900, Kyotaro HORIGUCHI wrote:
> > FWIW it seems to me a bug that making an inconsistent set of
> > files in archive directory.
>
> Okay, point taken! FWIW, I have no actual objections in not
> back-patching that.
I maybe(?) know.
> > At Fri, 21 Dec 2018 12:19:21 +0900, Michael Paquier <[email protected]> wrote in <[email protected]>
> >> On Thu, Dec 20, 2018 at 01:57:30PM +0200, David Steele wrote:
> >> > Good point. The new patch uses IsTLHistoryFileName().
> >>
> >> OK, I have been reviewing the patch and the logic is correct, still I
> >> could not resist reducing the number of inner if's for readability. I
> >
> > +1 basically. But I think that tail(name, 6) != ".ready" can
> > happen with a certain frequency but strspn(name) < basenamelen
> > rarely in the normal case.
>
> So that +0.5 if "basically" means a partial agreement? :p
Mmm. No, +0.9.
> > In the else if condition, ishisotry must be false in the right
> > hand of ||. What we do here is ignoring non-history files once
> > history file found. (Just a logic condensing and it would be done
> > by compiler, though)
>
> Yes, this can be simplified. So let's do so.
>
> > The caller prepares sufficient memory for basename, and we no
> > longer copy ".ready" into newxlog. Wouldn't we work directly on
Sorry for silly typo, but the 'W' was 'C', I meant:p
> > xlog instead of allocating newxlog?
>
> Okay, let's simplify that as you suggest.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-21 06:17 David Steele <[email protected]>
parent: Kyotaro HORIGUCHI <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: David Steele @ 2018-12-21 06:17 UTC (permalink / raw)
To: Kyotaro HORIGUCHI <[email protected]>; [email protected]; +Cc: pgsql-hackers
On 12/21/18 6:49 AM, Kyotaro HORIGUCHI wrote:
> "else if (!historyFound || ishistory)"
>
>
>
>> strcpy(xlog, newxlog);
>
> The caller prepares sufficient memory for basename, and we no
> longer copy ".ready" into newxlog. Douldn't we work directly on
> xlog instead of allocating newxlog?
I thought about doing that, but wanted to focus on the task at hand. It
does save a strcpy and a bit of stack space, so seems like a win.
Overall, the patch looks good to me. I think breaking up the if does
make the code more readable.
Regards,
--
-David
[email protected]
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-21 23:55 Michael Paquier <[email protected]>
parent: David Steele <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Michael Paquier @ 2018-12-21 23:55 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Kyotaro HORIGUCHI <[email protected]>; pgsql-hackers
On Fri, Dec 21, 2018 at 08:17:12AM +0200, David Steele wrote:
> I thought about doing that, but wanted to focus on the task at hand. It
> does save a strcpy and a bit of stack space, so seems like a win.
>
> Overall, the patch looks good to me. I think breaking up the if does make
> the code more readable.
Thanks for the lookups. I can see that the patch applies without
conflicts down to 9.4, and based on the opinions gathered on this
thread back-patching this stuff is the consensus, based on input from
Kyotaro Horiguchi and David Steele (I don't mind much myself). So,
any objections from others in doing so?
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-24 11:31 Michael Paquier <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Michael Paquier @ 2018-12-24 11:31 UTC (permalink / raw)
To: David Steele <[email protected]>; +Cc: Kyotaro HORIGUCHI <[email protected]>; pgsql-hackers
On Sat, Dec 22, 2018 at 08:55:14AM +0900, Michael Paquier wrote:
> Thanks for the lookups. I can see that the patch applies without
> conflicts down to 9.4, and based on the opinions gathered on this
> thread back-patching this stuff is the consensus, based on input from
> Kyotaro Horiguchi and David Steele (I don't mind much myself). So,
> any objections from others in doing so?
On REL9_4_STABLE, IsTLHistoryFileName() goes missing, so committed
"only" down to 9.5. Thanks David for the patch and Horiguchi-san for
the review.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Change pgarch_readyXlog() to return .history files first
@ 2018-12-26 14:45 David Steele <[email protected]>
parent: Michael Paquier <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: David Steele @ 2018-12-26 14:45 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Kyotaro HORIGUCHI <[email protected]>; pgsql-hackers
On 12/24/18 1:31 PM, Michael Paquier wrote:
> On Sat, Dec 22, 2018 at 08:55:14AM +0900, Michael Paquier wrote:
>> Thanks for the lookups. I can see that the patch applies without
>> conflicts down to 9.4, and based on the opinions gathered on this
>> thread back-patching this stuff is the consensus, based on input from
>> Kyotaro Horiguchi and David Steele (I don't mind much myself). So,
>> any objections from others in doing so?
>
> On REL9_4_STABLE, IsTLHistoryFileName() goes missing, so committed
> "only" down to 9.5. Thanks David for the patch and Horiguchi-san for
> the review.
Thanks, Michael!
I'm not too worried about 9.4 since it is currently the oldest supported
version. HA users tend to be on the leading edge of the upgrade curve
and others have the opportunity to upgrade if the reordering will help them.
--
-David
[email protected]
^ permalink raw reply [nested|flat] 35+ messages in thread
* [PATCH v11 3/3] Rework wal_receiver_create_temp_slot using new infrastructure
@ 2020-03-25 23:48 Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Alvaro Herrera @ 2020-03-25 23:48 UTC (permalink / raw)
---
doc/src/sgml/config.sgml | 10 ++---
src/backend/access/transam/xlog.c | 3 +-
src/backend/postmaster/startup.c | 10 ++++-
src/backend/replication/walreceiver.c | 40 +++++--------------
src/backend/replication/walreceiverfuncs.c | 28 ++++++++++---
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/postgresql.conf.sample | 3 +-
src/include/access/xlog.h | 1 +
src/include/replication/walreceiver.h | 4 +-
9 files changed, 55 insertions(+), 46 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 1cf88e953d..c5160fe907 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4173,11 +4173,11 @@ ANY <replaceable class="parameter">num_sync</replaceable> ( <replaceable class="
Specifies whether a WAL receiver should create a temporary replication
slot on the remote instance when no permanent replication slot to use
has been configured (using <xref linkend="guc-primary-slot-name"/>).
- The default is on. The only reason to turn this off would be if the
- remote instance is currently out of available replication slots. This
- parameter can only be set in the <filename>postgresql.conf</filename>
- file or on the server command line. Changes only take effect when the
- WAL receiver process starts a new connection.
+ The default is off. This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server command line.
+ </para>
+ <para>
+ The WAL receiver is restarted after an update of <varname>wal_receiver_create_temp_slot</varname>.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index bbf8d4eee5..2ecb319438 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -290,6 +290,7 @@ bool StandbyModeRequested = false;
char *PrimaryConnInfo = NULL;
char *PrimarySlotName = NULL;
char *PromoteTriggerFile = NULL;
+bool wal_receiver_create_temp_slot = false;
/* are we currently in standby mode? */
bool StandbyMode = false;
@@ -12157,7 +12158,7 @@ WaitForWALToBecomeAvailable(XLogRecPtr RecPtr, bool randAccess,
}
curFileTLI = tli;
RequestXLogStreaming(tli, ptr, PrimaryConnInfo,
- PrimarySlotName);
+ PrimarySlotName, wal_receiver_create_temp_slot);
receivedUpto = 0;
}
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index f35b5adf7d..fd9ac35dac 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -107,18 +107,26 @@ StartupRereadConfig(void)
{
char *conninfo = pstrdup(PrimaryConnInfo);
char *slotname = pstrdup(PrimarySlotName);
+ bool tempSlot = wal_receiver_create_temp_slot;
bool conninfoChanged;
bool slotnameChanged;
+ bool tempSlotChanged = false;
ProcessConfigFile(PGC_SIGHUP);
conninfoChanged = strcmp(conninfo, PrimaryConnInfo) != 0;
slotnameChanged = strcmp(slotname, PrimarySlotName) != 0;
+ /*
+ * wal_receiver_create_temp_slot is used only when we have no slot
+ * configured. We do not need to track this change if it has no effect.
+ */
+ if (!slotnameChanged && strcmp(PrimarySlotName, "") == 0)
+ tempSlotChanged = tempSlot != wal_receiver_create_temp_slot;
pfree(conninfo);
pfree(slotname);
- if (conninfoChanged || slotnameChanged)
+ if (conninfoChanged || slotnameChanged || tempSlotChanged)
StartupRequestWalReceiverRestart();
}
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index b471a49d12..0aa0c52c49 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -80,7 +80,6 @@
/* GUC variables */
-bool wal_receiver_create_temp_slot;
int wal_receiver_status_interval;
int wal_receiver_timeout;
bool hot_standby_feedback;
@@ -355,42 +354,23 @@ WalReceiverMain(void)
WalRcvFetchTimeLineHistoryFiles(startpointTLI, primaryTLI);
/*
- * Create temporary replication slot if no slot name is configured or
- * the slot from the previous run was temporary, unless
- * wal_receiver_create_temp_slot is disabled. We also need to handle
- * the case where the previous run used a temporary slot but
- * wal_receiver_create_temp_slot was changed in the meantime. In that
- * case, we delete the old slot name in shared memory. (This would
+ * Create temporary replication slot if requested. In that
+ * case, we update slot name in shared memory. (This would
* all be a bit easier if we just didn't copy the slot name into
* shared memory, since we won't need it again later, but then we
* can't see the slot name in the stats views.)
*/
- if (slotname[0] == '\0' || is_temp_slot)
+ if (is_temp_slot)
{
- bool changed = false;
+ snprintf(slotname, sizeof(slotname),
+ "pg_walreceiver_%lld",
+ (long long int) walrcv_get_backend_pid(wrconn));
- if (wal_receiver_create_temp_slot)
- {
- snprintf(slotname, sizeof(slotname),
- "pg_walreceiver_%lld",
- (long long int) walrcv_get_backend_pid(wrconn));
+ walrcv_create_slot(wrconn, slotname, true, 0, NULL);
- walrcv_create_slot(wrconn, slotname, true, 0, NULL);
- changed = true;
- }
- else if (slotname[0] != '\0')
- {
- slotname[0] = '\0';
- changed = true;
- }
-
- if (changed)
- {
- SpinLockAcquire(&walrcv->mutex);
- strlcpy(walrcv->slotname, slotname, NAMEDATALEN);
- walrcv->is_temp_slot = wal_receiver_create_temp_slot;
- SpinLockRelease(&walrcv->mutex);
- }
+ SpinLockAcquire(&walrcv->mutex);
+ strlcpy(walrcv->slotname, slotname, NAMEDATALEN);
+ SpinLockRelease(&walrcv->mutex);
}
/*
diff --git a/src/backend/replication/walreceiverfuncs.c b/src/backend/replication/walreceiverfuncs.c
index 89c903e45a..21d1823607 100644
--- a/src/backend/replication/walreceiverfuncs.c
+++ b/src/backend/replication/walreceiverfuncs.c
@@ -215,13 +215,19 @@ ShutdownWalRcv(void)
/*
* Request postmaster to start walreceiver.
*
- * recptr indicates the position where streaming should begin, conninfo
- * is a libpq connection string to use, and slotname is, optionally, the name
- * of a replication slot to acquire.
+ * "recptr" indicates the position where streaming should begin. "conninfo"
+ * is a libpq connection string to use. "slotname" is, optionally, the name
+ * of a replication slot to acquire. "create_temp_slot" indicates to create
+ * a temporary slot when no "slotname" is given.
+ *
+ * WAL receivers do not directly load GUC parameters used for the connection
+ * to the primary, and rely on the values passed down by the caller of this
+ * routine instead. Hence, the addition of any new parameters should happen
+ * through this code path.
*/
void
RequestXLogStreaming(TimeLineID tli, XLogRecPtr recptr, const char *conninfo,
- const char *slotname)
+ const char *slotname, bool create_temp_slot)
{
WalRcvData *walrcv = WalRcv;
bool launch = false;
@@ -248,10 +254,22 @@ RequestXLogStreaming(TimeLineID tli, XLogRecPtr recptr, const char *conninfo,
else
walrcv->conninfo[0] = '\0';
- if (slotname != NULL)
+ /*
+ * Use configured replication slot if present, and ignore the value
+ * of create_temp_slot as the slot name should be persistent. Otherwise,
+ * use create_temp_slot to determine whether this WAL receiver should
+ * create a temporary slot by itself and use it, or not.
+ */
+ if (slotname != NULL && slotname[0] != '\0')
+ {
strlcpy((char *) walrcv->slotname, slotname, NAMEDATALEN);
+ walrcv->is_temp_slot = false;
+ }
else
+ {
walrcv->slotname[0] = '\0';
+ walrcv->is_temp_slot = create_temp_slot;
+ }
if (walrcv->walRcvState == WALRCV_STOPPED)
{
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index c0b2b75407..53665971f5 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2054,7 +2054,7 @@ static struct config_bool ConfigureNamesBool[] =
gettext_noop("Sets whether a WAL receiver should create a temporary replication slot if no permanent slot is configured."),
},
&wal_receiver_create_temp_slot,
- true,
+ false,
NULL, NULL, NULL
},
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index c4f2f881c9..f01e43b818 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -319,7 +319,8 @@
#max_standby_streaming_delay = 30s # max delay before canceling queries
# when reading streaming WAL;
# -1 allows indefinite delay
-#wal_receiver_create_temp_slot = on # create temp slot if primary_slot_name not set
+#wal_receiver_create_temp_slot = off # Create temp slot if primary_slot_name
+ # is not set.
#wal_receiver_status_interval = 10s # send replies at least this often
# 0 disables
#hot_standby_feedback = off # send info from standby to prevent
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 42c4df0262..997db214cc 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -129,6 +129,7 @@ extern int recoveryTargetAction;
extern int recovery_min_apply_delay;
extern char *PrimaryConnInfo;
extern char *PrimarySlotName;
+extern bool wal_receiver_create_temp_slot;
/* indirectly set via GUC system */
extern TransactionId recoveryTargetXid;
diff --git a/src/include/replication/walreceiver.h b/src/include/replication/walreceiver.h
index e08afc6548..cf3e43128c 100644
--- a/src/include/replication/walreceiver.h
+++ b/src/include/replication/walreceiver.h
@@ -23,7 +23,6 @@
#include "utils/tuplestore.h"
/* user-settable parameters */
-extern bool wal_receiver_create_temp_slot;
extern int wal_receiver_status_interval;
extern int wal_receiver_timeout;
extern bool hot_standby_feedback;
@@ -321,7 +320,8 @@ extern void ShutdownWalRcv(void);
extern bool WalRcvStreaming(void);
extern bool WalRcvRunning(void);
extern void RequestXLogStreaming(TimeLineID tli, XLogRecPtr recptr,
- const char *conninfo, const char *slotname);
+ const char *conninfo, const char *slotname,
+ bool create_temp_slot);
extern XLogRecPtr GetWalRcvWriteRecPtr(XLogRecPtr *latestChunkStart, TimeLineID *receiveTLI);
extern int GetReplicationApplyDelay(void);
extern int GetReplicationTransferLatency(void);
--
2.20.1
--ibTvN161/egqYuK8--
^ permalink raw reply [nested|flat] 35+ messages in thread
* MERGE ... WHEN NOT MATCHED BY SOURCE
@ 2022-12-30 16:56 Dean Rasheed <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Dean Rasheed @ 2022-12-30 16:56 UTC (permalink / raw)
To: PostgreSQL Hackers <[email protected]>
This allows MERGE to UPDATE or DELETE target rows where there is no
matching source row. In addition, it allows the existing "WHEN NOT
MATCHED" syntax to include an optional "BY TARGET" to make its meaning
more explicit. E.g.,
MERGE INTO tgt USING src ON ...
WHEN NOT MATCHED BY SOURCE THEN UPDATE/DELETE ...
WHEN NOT MATCHED BY TARGET THEN INSERT ...
AFAIK, this is not part of the standard (though I only have a very old
draft copy). It is supported by at least 2 other major DB vendors
though, and I think it usefully rounds off the set of possible MERGE
actions.
Attached is a WIP patch. I haven't updated the docs yet, and there are
probably a few other things to tidy up and test, but the basic
functionality is there.
Regards,
Dean
Attachments:
[text/x-patch] support-merge-when-not-matched-by-source-v1.patch (24.8K, ../../CAEZATCWqnKGc57Y_JanUBHQXNKcXd7r=0R4NEZUVwP+syRkWbA@mail.gmail.com/2-support-merge-when-not-matched-by-source-v1.patch)
download | inline diff:
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
new file mode 100644
index 4cec12a..ed538e5
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -142,7 +142,6 @@ transform_MERGE_to_join(Query *parse)
{
RangeTblEntry *joinrte;
JoinExpr *joinexpr;
- JoinType jointype;
int joinrti;
List *vars;
@@ -152,20 +151,10 @@ transform_MERGE_to_join(Query *parse)
/* XXX probably bogus */
vars = NIL;
- /*
- * When any WHEN NOT MATCHED THEN INSERT clauses exist, we need to use an
- * outer join so that we process all unmatched tuples from the source
- * relation. If none exist, we can use an inner join.
- */
- if (parse->mergeUseOuterJoin)
- jointype = JOIN_RIGHT;
- else
- jointype = JOIN_INNER;
-
/* Manufacture a join RTE to use. */
joinrte = makeNode(RangeTblEntry);
joinrte->rtekind = RTE_JOIN;
- joinrte->jointype = jointype;
+ joinrte->jointype = parse->mergeJoinType;
joinrte->joinmergedcols = 0;
joinrte->joinaliasvars = vars;
joinrte->joinleftcols = NIL; /* MERGE does not allow JOIN USING */
@@ -189,7 +178,7 @@ transform_MERGE_to_join(Query *parse)
* Create a JOIN between the target and the source relation.
*/
joinexpr = makeNode(JoinExpr);
- joinexpr->jointype = jointype;
+ joinexpr->jointype = parse->mergeJoinType;
joinexpr->isNatural = false;
joinexpr->larg = (Node *) makeNode(RangeTblRef);
((RangeTblRef *) joinexpr->larg)->rtindex = parse->resultRelation;
diff --git a/src/backend/optimizer/prep/preptlist.c b/src/backend/optimizer/prep/preptlist.c
new file mode 100644
index 137b283..1a4e13e
--- a/src/backend/optimizer/prep/preptlist.c
+++ b/src/backend/optimizer/prep/preptlist.c
@@ -157,15 +157,14 @@ preprocess_targetlist(PlannerInfo *root)
/*
* Add resjunk entries for any Vars used in each action's
* targetlist and WHEN condition that belong to relations other
- * than target. Note that aggregates, window functions and
- * placeholder vars are not possible anywhere in MERGE's WHEN
- * clauses. (PHVs may be added later, but they don't concern us
- * here.)
+ * than target. Note that aggregates and window functions are not
+ * possible anywhere in MERGE's WHEN clauses, but PlaceHolderVars
+ * may have been added by subquery pullup.
*/
vars = pull_var_clause((Node *)
list_concat_copy((List *) action->qual,
action->targetList),
- 0);
+ PVC_INCLUDE_PLACEHOLDERS);
foreach(l2, vars)
{
Var *var = (Var *) lfirst(l2);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
new file mode 100644
index 63b4baa..95531ab
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -273,6 +273,7 @@ static Node *makeRecursiveViewSelect(cha
struct SelectLimit *selectlimit;
SetQuantifier setquantifier;
struct GroupClause *groupclause;
+ MergeMatchKind mergematch;
MergeWhenClause *mergewhen;
struct KeyActions *keyactions;
struct KeyAction *keyaction;
@@ -512,6 +513,7 @@ static Node *makeRecursiveViewSelect(cha
%type <onconflict> opt_on_conflict
%type <mergewhen> merge_insert merge_update merge_delete
+%type <mergematch> merge_when_tgt_matched merge_when_tgt_not_matched
%type <node> merge_when_clause opt_merge_when_condition
%type <list> merge_when_list
@@ -740,11 +742,11 @@ static Node *makeRecursiveViewSelect(cha
SAVEPOINT SCHEMA SCHEMAS SCROLL SEARCH SECOND_P SECURITY SELECT SEQUENCE SEQUENCES
SERIALIZABLE SERVER SESSION SESSION_USER SET SETS SETOF SHARE SHOW
- SIMILAR SIMPLE SKIP SMALLINT SNAPSHOT SOME SQL_P STABLE STANDALONE_P
+ SIMILAR SIMPLE SKIP SMALLINT SNAPSHOT SOME SOURCE SQL_P STABLE STANDALONE_P
START STATEMENT STATISTICS STDIN STDOUT STORAGE STORED STRICT_P STRIP_P
SUBSCRIPTION SUBSTRING SUPPORT SYMMETRIC SYSID SYSTEM_P SYSTEM_USER
- TABLE TABLES TABLESAMPLE TABLESPACE TEMP TEMPLATE TEMPORARY TEXT_P THEN
+ TABLE TABLES TABLESAMPLE TABLESPACE TARGET TEMP TEMPLATE TEMPORARY TEXT_P THEN
TIES TIME TIMESTAMP TO TRAILING TRANSACTION TRANSFORM
TREAT TRIGGER TRIM TRUE_P
TRUNCATE TRUSTED TYPE_P TYPES_P
@@ -12260,49 +12262,59 @@ merge_when_list:
;
merge_when_clause:
- WHEN MATCHED opt_merge_when_condition THEN merge_update
+ merge_when_tgt_matched opt_merge_when_condition THEN merge_update
{
- $5->matched = true;
- $5->condition = $3;
+ $4->matchKind = $1;
+ $4->condition = $2;
- $$ = (Node *) $5;
+ $$ = (Node *) $4;
}
- | WHEN MATCHED opt_merge_when_condition THEN merge_delete
+ | merge_when_tgt_matched opt_merge_when_condition THEN merge_delete
{
- $5->matched = true;
- $5->condition = $3;
+ $4->matchKind = $1;
+ $4->condition = $2;
- $$ = (Node *) $5;
+ $$ = (Node *) $4;
}
- | WHEN NOT MATCHED opt_merge_when_condition THEN merge_insert
+ | merge_when_tgt_not_matched opt_merge_when_condition THEN merge_insert
{
- $6->matched = false;
- $6->condition = $4;
+ $4->matchKind = $1;
+ $4->condition = $2;
- $$ = (Node *) $6;
+ $$ = (Node *) $4;
}
- | WHEN MATCHED opt_merge_when_condition THEN DO NOTHING
+ | merge_when_tgt_matched opt_merge_when_condition THEN DO NOTHING
{
MergeWhenClause *m = makeNode(MergeWhenClause);
- m->matched = true;
+ m->matchKind = $1;
m->commandType = CMD_NOTHING;
- m->condition = $3;
+ m->condition = $2;
$$ = (Node *) m;
}
- | WHEN NOT MATCHED opt_merge_when_condition THEN DO NOTHING
+ | merge_when_tgt_not_matched opt_merge_when_condition THEN DO NOTHING
{
MergeWhenClause *m = makeNode(MergeWhenClause);
- m->matched = false;
+ m->matchKind = $1;
m->commandType = CMD_NOTHING;
- m->condition = $4;
+ m->condition = $2;
$$ = (Node *) m;
}
;
+merge_when_tgt_matched:
+ WHEN MATCHED { $$ = MERGE_WHEN_MATCHED; }
+ | WHEN NOT MATCHED BY SOURCE { $$ = MERGE_WHEN_NOT_MATCHED_BY_SOURCE; }
+ ;
+
+merge_when_tgt_not_matched:
+ WHEN NOT MATCHED { $$ = MERGE_WHEN_NOT_MATCHED_BY_TARGET; }
+ | WHEN NOT MATCHED BY TARGET { $$ = MERGE_WHEN_NOT_MATCHED_BY_TARGET; }
+ ;
+
opt_merge_when_condition:
AND a_expr { $$ = $2; }
| { $$ = NULL; }
@@ -16961,6 +16973,7 @@ unreserved_keyword:
| SIMPLE
| SKIP
| SNAPSHOT
+ | SOURCE
| SQL_P
| STABLE
| STANDALONE_P
@@ -16979,6 +16992,7 @@ unreserved_keyword:
| SYSTEM_P
| TABLES
| TABLESPACE
+ | TARGET
| TEMP
| TEMPLATE
| TEMPORARY
@@ -17555,6 +17569,7 @@ bare_label_keyword:
| SMALLINT
| SNAPSHOT
| SOME
+ | SOURCE
| SQL_P
| STABLE
| STANDALONE_P
@@ -17578,6 +17593,7 @@ bare_label_keyword:
| TABLES
| TABLESAMPLE
| TABLESPACE
+ | TARGET
| TEMP
| TEMPLATE
| TEMPORARY
diff --git a/src/backend/parser/parse_merge.c b/src/backend/parser/parse_merge.c
new file mode 100644
index 3844f2b..f565072
--- a/src/backend/parser/parse_merge.c
+++ b/src/backend/parser/parse_merge.c
@@ -43,9 +43,9 @@ static void setNamespaceVisibilityForRTE
* Make appropriate changes to the namespace visibility while transforming
* individual action's quals and targetlist expressions. In particular, for
* INSERT actions we must only see the source relation (since INSERT action is
- * invoked for NOT MATCHED tuples and hence there is no target tuple to deal
- * with). On the other hand, UPDATE and DELETE actions can see both source and
- * target relations.
+ * invoked for NOT MATCHED [BY TARGET] tuples and hence there is no target
+ * tuple to deal with). On the other hand, UPDATE and DELETE actions can see
+ * both source and target relations, unless invoked for NOT MATCH BY SOURCE.
*
* Also, since the internal join node can hide the source and target
* relations, we must explicitly make the respective relation as visible so
@@ -61,7 +61,7 @@ setNamespaceForMergeWhen(ParseState *pst
targetRelRTE = rt_fetch(targetRTI, pstate->p_rtable);
sourceRelRTE = rt_fetch(sourceRTI, pstate->p_rtable);
- if (mergeWhenClause->matched)
+ if (mergeWhenClause->matchKind == MERGE_WHEN_MATCHED)
{
Assert(mergeWhenClause->commandType == CMD_UPDATE ||
mergeWhenClause->commandType == CMD_DELETE ||
@@ -73,11 +73,25 @@ setNamespaceForMergeWhen(ParseState *pst
setNamespaceVisibilityForRTE(pstate->p_namespace,
sourceRelRTE, true, true);
}
- else
+ else if (mergeWhenClause->matchKind == MERGE_WHEN_NOT_MATCHED_BY_SOURCE)
{
/*
- * NOT MATCHED actions can't see target relation, but they can see
- * source relation.
+ * NOT MATCHED BY SOURCE actions can see the target relation, but they
+ * can't see the source relation.
+ */
+ Assert(mergeWhenClause->commandType == CMD_UPDATE ||
+ mergeWhenClause->commandType == CMD_DELETE ||
+ mergeWhenClause->commandType == CMD_NOTHING);
+ setNamespaceVisibilityForRTE(pstate->p_namespace,
+ targetRelRTE, true, true);
+ setNamespaceVisibilityForRTE(pstate->p_namespace,
+ sourceRelRTE, false, false);
+ }
+ else /* mergeWhenClause->matchKind == MERGE_WHEN_NOT_MATCHED_BY_TARGET */
+ {
+ /*
+ * NOT MATCHED [BY TARGET] actions can't see target relation, but they
+ * can see source relation.
*/
Assert(mergeWhenClause->commandType == CMD_INSERT ||
mergeWhenClause->commandType == CMD_NOTHING);
@@ -98,7 +112,9 @@ transformMergeStmt(ParseState *pstate, M
Query *qry = makeNode(Query);
ListCell *l;
AclMode targetPerms = ACL_NO_RIGHTS;
- bool is_terminal[2];
+ bool is_terminal[3];
+ bool left_join;
+ bool right_join;
Index sourceRTI;
List *mergeActionList;
Node *joinExpr;
@@ -127,10 +143,12 @@ transformMergeStmt(ParseState *pstate, M
*/
is_terminal[0] = false;
is_terminal[1] = false;
+ is_terminal[2] = false;
+ left_join = false;
+ right_join = false;
foreach(l, stmt->mergeWhenClauses)
{
MergeWhenClause *mergeWhenClause = (MergeWhenClause *) lfirst(l);
- int when_type = (mergeWhenClause->matched ? 0 : 1);
/*
* Collect action types so we can check target permissions
@@ -156,13 +174,32 @@ transformMergeStmt(ParseState *pstate, M
* Check for unreachable WHEN clauses
*/
if (mergeWhenClause->condition == NULL)
- is_terminal[when_type] = true;
- else if (is_terminal[when_type])
+ is_terminal[mergeWhenClause->matchKind] = true;
+ else if (is_terminal[mergeWhenClause->matchKind])
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unreachable WHEN clause specified after unconditional WHEN clause")));
+
+ /* Is left/right/full outer join required? */
+ if (mergeWhenClause->commandType != CMD_NOTHING)
+ {
+ if (mergeWhenClause->matchKind == MERGE_WHEN_NOT_MATCHED_BY_SOURCE)
+ left_join = true;
+ if (mergeWhenClause->matchKind == MERGE_WHEN_NOT_MATCHED_BY_TARGET)
+ right_join = true;
+ }
}
+ /* Join type required */
+ if (left_join && right_join)
+ qry->mergeJoinType = JOIN_FULL;
+ else if (left_join)
+ qry->mergeJoinType = JOIN_LEFT;
+ else if (right_join)
+ qry->mergeJoinType = JOIN_RIGHT;
+ else
+ qry->mergeJoinType = JOIN_INNER;
+
/*
* Set up the MERGE target table. The target table is added to the
* namespace below and to joinlist in transform_MERGE_to_join, so don't
@@ -251,15 +288,12 @@ transformMergeStmt(ParseState *pstate, M
foreach(l, stmt->mergeWhenClauses)
{
MergeWhenClause *mergeWhenClause = lfirst_node(MergeWhenClause, l);
+ MergeMatchKind matchKind = mergeWhenClause->matchKind;
MergeAction *action;
action = makeNode(MergeAction);
action->commandType = mergeWhenClause->commandType;
- action->matched = mergeWhenClause->matched;
-
- /* Use an outer join if any INSERT actions exist in the command. */
- if (action->commandType == CMD_INSERT)
- qry->mergeUseOuterJoin = true;
+ action->matched = matchKind != MERGE_WHEN_NOT_MATCHED_BY_TARGET;
/*
* Set namespace for the specific action. This must be done before
@@ -280,6 +314,47 @@ transformMergeStmt(ParseState *pstate, M
EXPR_KIND_MERGE_WHEN, "WHEN");
/*
+ * If we are doing a left or full join that includes target-only rows
+ * (NOT MATCHED BY SOURCE), add additional WHEN conditions to each
+ * MATCHED and NOT MATCHED BY SOURCE action to distinguish them from
+ * one another.
+ *
+ * MATCHED rows are identified by a "source IS DISTINCT FROM NULL"
+ * clause, and NOT MATCHED BY SOURCE rows by a "source IS NOT DISTINCT
+ * FROM NULL" clause.
+ *
+ * Nothing is needed for NOT MATCHED BY TARGET rows, since they are
+ * identified in the executor by checking the target identity column.
+ */
+ if (left_join && action->matched)
+ {
+ NullTest *ntest;
+
+ /* Make a "source IS [NOT] DISTINCT FROM NULL" clause */
+ ntest = makeNode(NullTest);
+
+ ntest->arg = (Expr *) makeVar(sourceRTI,
+ InvalidAttrNumber,
+ RECORDOID,
+ -1,
+ InvalidOid,
+ 0);
+ ntest->nulltesttype =
+ matchKind == MERGE_WHEN_MATCHED ? IS_NOT_NULL : IS_NULL;
+ ntest->argisrow = false;
+ ntest->location = -1;
+
+ /* Combine it with the action's WHEN condition */
+ if (action->qual == NULL)
+ action->qual = (Node *) ntest;
+ else
+ action->qual =
+ (Node *) makeBoolExpr(AND_EXPR,
+ list_make2(ntest, action->qual),
+ -1);
+ }
+
+ /*
* Transform target lists for each INSERT and UPDATE action stmt
*/
switch (action->commandType)
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
new file mode 100644
index 34bc640..887a5d1
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -159,7 +159,7 @@ typedef struct Query
* also USING clause for MERGE */
List *mergeActionList; /* list of actions for MERGE (only) */
- bool mergeUseOuterJoin; /* whether to use outer join */
+ JoinType mergeJoinType; /* target to source join type */
List *targetList; /* target list (of TargetEntry) */
@@ -1586,6 +1586,14 @@ typedef struct CommonTableExpr
((Query *) (cte)->ctequery)->targetList : \
((Query *) (cte)->ctequery)->returningList)
+/* MATCHED/NOT MATCHED specification for MergeWhenClause */
+typedef enum MergeMatchKind
+{
+ MERGE_WHEN_MATCHED,
+ MERGE_WHEN_NOT_MATCHED_BY_SOURCE,
+ MERGE_WHEN_NOT_MATCHED_BY_TARGET
+} MergeMatchKind;
+
/*
* MergeWhenClause -
* raw parser representation of a WHEN clause in a MERGE statement
@@ -1595,7 +1603,7 @@ typedef struct CommonTableExpr
typedef struct MergeWhenClause
{
NodeTag type;
- bool matched; /* true=MATCHED, false=NOT MATCHED */
+ MergeMatchKind matchKind; /* MATCHED/NOT MATCHED BY SOURCE/TARGET */
CmdType commandType; /* INSERT/UPDATE/DELETE/DO NOTHING */
OverridingKind override; /* OVERRIDING clause */
Node *condition; /* WHEN conditions (raw parser) */
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
new file mode 100644
index 957ee18..fd94b0b
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -391,6 +391,7 @@ PG_KEYWORD("skip", SKIP, UNRESERVED_KEYW
PG_KEYWORD("smallint", SMALLINT, COL_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("snapshot", SNAPSHOT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("some", SOME, RESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("source", SOURCE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("sql", SQL_P, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("stable", STABLE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("standalone", STANDALONE_P, UNRESERVED_KEYWORD, BARE_LABEL)
@@ -414,6 +415,7 @@ PG_KEYWORD("table", TABLE, RESERVED_KEYW
PG_KEYWORD("tables", TABLES, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("tablesample", TABLESAMPLE, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("tablespace", TABLESPACE, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("target", TARGET, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("temp", TEMP, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("template", TEMPLATE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("temporary", TEMPORARY, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
new file mode 100644
index 6c8a18f..e99623a
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -71,6 +71,15 @@ WHEN MATCHED THEN
ERROR: syntax error at or near "INSERT"
LINE 5: INSERT DEFAULT VALUES;
^
+-- NOT MATCHED BY SOURCE/INSERT error
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE THEN
+ INSERT DEFAULT VALUES;
+ERROR: syntax error at or near "INSERT"
+LINE 5: INSERT DEFAULT VALUES;
+ ^
-- incorrectly specifying INTO target
MERGE INTO target t
USING source AS s
@@ -107,6 +116,15 @@ WHEN NOT MATCHED THEN
ERROR: syntax error at or near "UPDATE"
LINE 5: UPDATE SET balance = 0;
^
+-- NOT MATCHED BY TARGET/UPDATE
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY TARGET THEN
+ UPDATE SET balance = 0;
+ERROR: syntax error at or near "UPDATE"
+LINE 5: UPDATE SET balance = 0;
+ ^
-- UPDATE tablename
MERGE INTO target t
USING source AS s
@@ -271,6 +289,22 @@ SELECT * FROM target ORDER BY tid;
(4 rows)
ROLLBACK;
+-- DELETE/INSERT not matched by source/target
+BEGIN;
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE
+WHEN NOT MATCHED BY TARGET THEN
+ INSERT VALUES (s.sid, s.delta);
+SELECT * FROM target ORDER BY tid;
+ tid | balance
+-----+---------
+ 4 | 40
+(1 row)
+
+ROLLBACK;
-- index plans
INSERT INTO target SELECT generate_series(1000,2500), 0;
ALTER TABLE target ADD PRIMARY KEY (tid);
@@ -738,6 +772,19 @@ SELECT * FROM wq_target;
1 | -1
(1 row)
+-- conditions in NOT MATCHED BY SOURCE clause can only refer to target columns
+MERGE INTO wq_target t
+USING wq_source s ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE AND s.balance = 100 THEN
+ DELETE;
+ERROR: invalid reference to FROM-clause entry for table "s"
+LINE 3: WHEN NOT MATCHED BY SOURCE AND s.balance = 100 THEN
+ ^
+DETAIL: There is an entry for table "s", but it cannot be referenced from this part of the query.
+MERGE INTO wq_target t
+USING wq_source s ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE AND t.balance = 100 THEN
+ DELETE;
-- conditions in MATCHED clause can refer to both source and target
SELECT * FROM wq_source;
balance | sid
@@ -923,6 +970,45 @@ SELECT * FROM target ORDER BY tid;
(3 rows)
ROLLBACK;
+-- UPSERT with UPDATE/DELETE not matched
+BEGIN;
+DELETE FROM SOURCE WHERE sid = 2;
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN MATCHED AND t.balance > s.delta THEN
+ UPDATE SET balance = t.balance - s.delta
+WHEN MATCHED THEN
+ UPDATE SET balance = 0
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.sid, s.delta)
+WHEN NOT MATCHED BY SOURCE AND tid = 1 THEN
+ UPDATE SET balance = 0
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE;
+NOTICE: BEFORE INSERT STATEMENT trigger
+NOTICE: BEFORE UPDATE STATEMENT trigger
+NOTICE: BEFORE DELETE STATEMENT trigger
+NOTICE: BEFORE UPDATE ROW trigger row: (3,30) -> (3,10)
+NOTICE: BEFORE INSERT ROW trigger row: (4,40)
+NOTICE: BEFORE DELETE ROW trigger row: (2,20)
+NOTICE: BEFORE UPDATE ROW trigger row: (1,10) -> (1,0)
+NOTICE: AFTER UPDATE ROW trigger row: (3,30) -> (3,10)
+NOTICE: AFTER INSERT ROW trigger row: (4,40)
+NOTICE: AFTER DELETE ROW trigger row: (2,20)
+NOTICE: AFTER UPDATE ROW trigger row: (1,10) -> (1,0)
+NOTICE: AFTER DELETE STATEMENT trigger
+NOTICE: AFTER UPDATE STATEMENT trigger
+NOTICE: AFTER INSERT STATEMENT trigger
+SELECT * FROM target ORDER BY tid;
+ tid | balance
+-----+---------
+ 1 | 0
+ 3 | 10
+ 4 | 40
+(3 rows)
+
+ROLLBACK;
-- Test behavior of triggers that turn UPDATE/DELETE into no-ops
create or replace function skip_merge_op() returns trigger
language plpgsql as
@@ -1436,6 +1522,50 @@ WHEN NOT MATCHED AND s.a < 20 THEN
-> Seq Scan on ex_mtarget t (actual rows=49 loops=1)
(12 rows)
+-- not matched by source
+SELECT explain_merge('
+MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a
+WHEN NOT MATCHED BY SOURCE and t.a < 10 THEN
+ DELETE');
+ explain_merge
+----------------------------------------------------------------------
+ Merge on ex_mtarget t (actual rows=0 loops=1)
+ Tuples: skipped=54
+ -> Merge Left Join (actual rows=54 loops=1)
+ Merge Cond: (t.a = s.a)
+ -> Sort (actual rows=54 loops=1)
+ Sort Key: t.a
+ Sort Method: quicksort Memory: xxx
+ -> Seq Scan on ex_mtarget t (actual rows=54 loops=1)
+ -> Sort (actual rows=100 loops=1)
+ Sort Key: s.a
+ Sort Method: quicksort Memory: xxx
+ -> Seq Scan on ex_msource s (actual rows=100 loops=1)
+(12 rows)
+
+-- not matched by source and target
+SELECT explain_merge('
+MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a
+WHEN NOT MATCHED BY SOURCE AND t.a < 10 THEN
+ DELETE
+WHEN NOT MATCHED BY TARGET AND s.a < 20 THEN
+ INSERT VALUES (a, b)');
+ explain_merge
+----------------------------------------------------------------------
+ Merge on ex_mtarget t (actual rows=0 loops=1)
+ Tuples: skipped=100
+ -> Merge Full Join (actual rows=100 loops=1)
+ Merge Cond: (t.a = s.a)
+ -> Sort (actual rows=54 loops=1)
+ Sort Key: t.a
+ Sort Method: quicksort Memory: xxx
+ -> Seq Scan on ex_mtarget t (actual rows=54 loops=1)
+ -> Sort (actual rows=100 loops=1)
+ Sort Key: s.a
+ Sort Method: quicksort Memory: xxx
+ -> Seq Scan on ex_msource s (actual rows=100 loops=1)
+(12 rows)
+
-- nothing
SELECT explain_merge('
MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a AND t.a < -1000
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
new file mode 100644
index 98fe104..c641214
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -51,6 +51,12 @@ USING source AS s
ON t.tid = s.sid
WHEN MATCHED THEN
INSERT DEFAULT VALUES;
+-- NOT MATCHED BY SOURCE/INSERT error
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE THEN
+ INSERT DEFAULT VALUES;
-- incorrectly specifying INTO target
MERGE INTO target t
USING source AS s
@@ -75,6 +81,12 @@ USING source AS s
ON t.tid = s.sid
WHEN NOT MATCHED THEN
UPDATE SET balance = 0;
+-- NOT MATCHED BY TARGET/UPDATE
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY TARGET THEN
+ UPDATE SET balance = 0;
-- UPDATE tablename
MERGE INTO target t
USING source AS s
@@ -213,6 +225,18 @@ WHEN NOT MATCHED THEN
SELECT * FROM target ORDER BY tid;
ROLLBACK;
+-- DELETE/INSERT not matched by source/target
+BEGIN;
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE
+WHEN NOT MATCHED BY TARGET THEN
+ INSERT VALUES (s.sid, s.delta);
+SELECT * FROM target ORDER BY tid;
+ROLLBACK;
+
-- index plans
INSERT INTO target SELECT generate_series(1000,2500), 0;
ALTER TABLE target ADD PRIMARY KEY (tid);
@@ -497,6 +521,17 @@ WHEN NOT MATCHED AND s.balance = 100 THE
INSERT (tid) VALUES (s.sid);
SELECT * FROM wq_target;
+-- conditions in NOT MATCHED BY SOURCE clause can only refer to target columns
+MERGE INTO wq_target t
+USING wq_source s ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE AND s.balance = 100 THEN
+ DELETE;
+
+MERGE INTO wq_target t
+USING wq_source s ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE AND t.balance = 100 THEN
+ DELETE;
+
-- conditions in MATCHED clause can refer to both source and target
SELECT * FROM wq_source;
MERGE INTO wq_target t
@@ -624,6 +659,25 @@ WHEN NOT MATCHED THEN
SELECT * FROM target ORDER BY tid;
ROLLBACK;
+-- UPSERT with UPDATE/DELETE not matched
+BEGIN;
+DELETE FROM SOURCE WHERE sid = 2;
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN MATCHED AND t.balance > s.delta THEN
+ UPDATE SET balance = t.balance - s.delta
+WHEN MATCHED THEN
+ UPDATE SET balance = 0
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.sid, s.delta)
+WHEN NOT MATCHED BY SOURCE AND tid = 1 THEN
+ UPDATE SET balance = 0
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE;
+SELECT * FROM target ORDER BY tid;
+ROLLBACK;
+
-- Test behavior of triggers that turn UPDATE/DELETE into no-ops
create or replace function skip_merge_op() returns trigger
language plpgsql as
@@ -916,6 +970,20 @@ WHEN MATCHED AND t.a >= 30 AND t.a <= 40
WHEN NOT MATCHED AND s.a < 20 THEN
INSERT VALUES (a, b)');
+-- not matched by source
+SELECT explain_merge('
+MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a
+WHEN NOT MATCHED BY SOURCE and t.a < 10 THEN
+ DELETE');
+
+-- not matched by source and target
+SELECT explain_merge('
+MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a
+WHEN NOT MATCHED BY SOURCE AND t.a < 10 THEN
+ DELETE
+WHEN NOT MATCHED BY TARGET AND s.a < 20 THEN
+ INSERT VALUES (a, b)');
+
-- nothing
SELECT explain_merge('
MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a AND t.a < -1000
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-06-10 05:26 Andres Freund <[email protected]>
0 siblings, 2 replies; 35+ messages in thread
From: Andres Freund @ 2023-06-10 05:26 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: pgsql-hackers
Hi,
On 2023-06-10 14:23:52 +1200, Thomas Munro wrote:
> 1. We should completely remove --disable-thread-safety and the
> various !defined(ENABLE_THREAD_SAFETY) code paths.
Yes, please!
> 2. I don't like the way we have to deal with POSIX vs Windows at
> every site where we use threads, and each place has a different style
> of wrappers. I considered a few different approaches to cleaning this
> up:
>
> * provide centralised and thorough pthread emulation for Windows; I
> don't like this, I don't even like all of pthreads and there are many
> details to get lost in
> * adopt C11 <threads.h>; unfortunately it is too early, so you'd need
> to write/borrow replacements for at least 3 of our 11 target systems
> * invent our own mini-abstraction for a carefully controlled subset of stuff
>
> Here is an attempt at that last thing. Since I don't really like
> making up new names for things, I just used all the C11 names but with
> pg_ in front, and declared it all in "port/pg_threads.h" (actually I
> tried to write C11 replacements and then noped out and added the pg_
> prefixes). I suppose the idea is that it and the prefixes might
> eventually go away. Note: here I am talking only about very basic
> operations like create, exit, join, explicit thread locals -- the
> stuff that we actually use today in frontend code.
Unsurprisingly, I like this.
> I'm not talking
> about other stuff like C11 atomics, memory models, and the
> thread_local storage class, which are all very good and interesting
> topics for another day.
Hm. I agree on C11 atomics and memory models, but I don't see a good reason to
not add support for thread_local?
In fact, I'd rather add support for thread_local and not add support for
"thread keys" than the other way round. Afaict most, if not all, the places
using keys would look simpler with thread_local than with keys.
> One mystery still eludes me on Windows: while trying to fix the
> ancient bug that ECPG leaks memory on Windows, because it doesn't call
> thread-local destructors, I discovered that if you use FlsAlloc
> instead of TlsAlloc you can pass in a destructor (that's
> "fibre-local", but we're not using fibres so it's works out the same
> as thread local AFAICT). It seems to crash in strange ways if you
> uncomment the line FlsAlloc(destructor).
Do you have a backtrace available?
> Subject: [PATCH 1/5] Remove obsolete comments and code from fe-auth.c.
> Subject: [PATCH 2/5] Rename port/thread.c to port/user.c.
LGTM
> -###############################################################
> -# Threading
> -###############################################################
> -
> -# XXX: About to rely on thread safety in the autoconf build, so not worth
> -# implementing a fallback.
> -cdata.set('ENABLE_THREAD_SAFETY', 1)
I wonder if we should just unconditionally set that in c.h or such? It'd not
be crazy for external projects to rely on that being set.
> From ca74df4ff11ce0fd1e51786eccaeca810921fc6d Mon Sep 17 00:00:00 2001
> From: Thomas Munro <[email protected]>
> Date: Sat, 10 Jun 2023 09:14:07 +1200
> Subject: [PATCH 4/5] Add port/pg_threads.h for a common threading API.
>
> Loosely based on C11's <threads.h>, but with pg_ prefixes, this will
> allow us to clean up many places that have to cope with POSIX and
> Windows threads.
> ---
> src/include/port/pg_threads.h | 252 +++++++++++++++++++++++++++++++
> src/port/Makefile | 1 +
> src/port/meson.build | 1 +
> src/port/pg_threads.c | 117 ++++++++++++++
> src/tools/pgindent/typedefs.list | 7 +
> 5 files changed, 378 insertions(+)
> create mode 100644 src/include/port/pg_threads.h
> create mode 100644 src/port/pg_threads.c
>
> diff --git a/src/include/port/pg_threads.h b/src/include/port/pg_threads.h
> new file mode 100644
> index 0000000000..1706709994
> --- /dev/null
> +++ b/src/include/port/pg_threads.h
> @@ -0,0 +1,252 @@
> +/*
> + * A multi-threading API abstraction loosely based on the C11 standard's
> + * <threads.h> header. The identifiers have a pg_ prefix. Perhaps one day
> + * we'll use standard C threads directly, and we'll drop the prefixes.
> + *
> + * Exceptions:
> + * - pg_thrd_barrier_t is not based on C11
> + */
> +
> +#ifndef PG_THREADS_H
> +#define PG_THREADS_H
> +
> +#ifdef WIN32
I guess we could try to use C11 thread.h style threading on msvc 2022:
https://developercommunity.visualstudio.com/t/finalize-c11-support-for-threading/1629487
> +#define WIN32_LEAN_AND_MEAN
Why do we need this again here - shouldn't the define in
src/include/port/win32_port.h already take care of this?
> +#include <windows.h>
> +#include <processthreadsapi.h>
> +#include <fibersapi.h>
> +#include <synchapi.h>
> +#else
> +#include <errno.h>
> +#include "port/pg_pthread.h"
> +#endif
Seems somewhat odd to have port pg_threads.h and pg_pthread.h - why not merge
these?
> +#include <stdint.h>
I think we widely rely stdint.h, errno.h to be included via c.h.
> +#ifdef WIN32
> +typedef HANDLE pg_thrd_t;
> +typedef CRITICAL_SECTION pg_mtx_t;
> +typedef CONDITION_VARIABLE pg_cnd_t;
> +typedef SYNCHRONIZATION_BARRIER pg_thrd_barrier_t;
> +typedef DWORD pg_tss_t;
> +typedef INIT_ONCE pg_once_flag;
> +#define PG_ONCE_FLAG_INIT INIT_ONCE_STATIC_INIT
> +#else
> +typedef pthread_t pg_thrd_t;
> +typedef pthread_mutex_t pg_mtx_t;
> +typedef pthread_cond_t pg_cnd_t;
> +typedef pthread_barrier_t pg_thrd_barrier_t;
> +typedef pthread_key_t pg_tss_t;
> +typedef pthread_once_t pg_once_flag;
> +#define PG_ONCE_FLAG_INIT PTHREAD_ONCE_INIT
> +#endif
> +
> +typedef int (*pg_thrd_start_t) (void *);
> +typedef void (*pg_tss_dtor_t) (void *);
> +typedef void (*pg_call_once_function_t) (void);
> +
> +enum
> +{
> + pg_thrd_success = 0,
> + pg_thrd_nomem = 1,
> + pg_thrd_timedout = 2,
> + pg_thrd_busy = 3,
> + pg_thrd_error = 4
> +};
> +
> +enum
> +{
> + pg_mtx_plain = 0
> +};
I think we add typedefs for enums as well.
> +#ifndef WIN32
> +static inline int
> +pg_thrd_maperror(int error)
Seems like that should have pthread in the name?
> +{
> + if (error == 0)
> + return pg_thrd_success;
> + if (error == ENOMEM)
> + return pg_thrd_nomem;
> + return pg_thrd_error;
> +}
> +#endif
> +
> +#ifdef WIN32
> +BOOL pg_call_once_trampoline(pg_once_flag *flag, void *parameter, void **context);
> +#endif
> +
> +static inline void
> +pg_call_once(pg_once_flag *flag, pg_call_once_function_t function)
> +{
> +#ifdef WIN32
> + InitOnceExecuteOnce(flag, pg_call_once_trampoline, (void *) function, NULL);
> +#else
> + pthread_once(flag, function);
> +#endif
> +}
> +
> +static inline int
> +pg_thrd_equal(pg_thrd_t lhs, pg_thrd_t rhs)
> +{
> +#ifdef WIN32
> + return lhs == rhs;
> +#else
> + return pthread_equal(lhs, rhs);
> +#endif
> +}
> +
> +static inline int
> +pg_tss_create(pg_tss_t *key, pg_tss_dtor_t destructor)
> +{
> +#ifdef WIN32
> + //*key = FlsAlloc(destructor);
> + *key = FlsAlloc(NULL);
> + return pg_thrd_success;
Afaict FlsAlloc() can return errors:
> If the function fails, the return value is FLS_OUT_OF_INDEXES. To get
> extended error information, call GetLastError.
> +#else
> + return pg_thrd_maperror(pthread_key_create(key, destructor));
> +#endif
> +}
> +static inline int
> +pg_tss_set(pg_tss_t key, void *value)
> +{
> +#ifdef WIN32
> + return FlsSetValue(key, value) ? pg_thrd_success : pg_thrd_error;
> +#else
> + return pg_thrd_maperror(pthread_setspecific(key, value));
> +#endif
> +}
It's somewhat annoying that this can return errors.
> +static inline int
> +pg_mtx_init(pg_mtx_t *mutex, int type)
I am somewhat confused by the need for these three letter
abbreviations... Blaming C11, not you, to be clear.
>
> -# port needs to be in include path due to pthread-win32.h
> +# XXX why do we need this?
> libpq_inc = include_directories('.', '../../port')
> libpq_c_args = ['-DSO_MAJOR_VERSION=5']
Historically we need it because random places outside of libpq include
pthread-win32.h - but pthread-win32.h is in src/port, not in
src/include/port. Why on earth it was done this way, I do not know.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-06-10 15:19 Joe Conway <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 0 replies; 35+ messages in thread
From: Joe Conway @ 2023-06-10 15:19 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Thomas Munro <[email protected]>; +Cc: pgsql-hackers
On 6/10/23 01:26, Andres Freund wrote:
> On 2023-06-10 14:23:52 +1200, Thomas Munro wrote:
>> I'm not talking
>> about other stuff like C11 atomics, memory models, and the
>> thread_local storage class, which are all very good and interesting
>> topics for another day.
>
> Hm. I agree on C11 atomics and memory models, but I don't see a good reason to
> not add support for thread_local?
Perhaps obvious to most/all on this thread, but something that bit me
recently to keep in mind as we contemplate threads.
When a shared object library (e.g. extensions) is loaded, the init()
function, and any functions marked with the constructor attribute, are
executed. However they are not run again during thread_start().
So if there is thread local state that needs to be initialized, the
initialization needs to be manually redone in each thread. Unless there
is some mechanism similar to a constructor that I missed?
--
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-07-03 07:29 Peter Eisentraut <[email protected]>
parent: Andres Freund <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Peter Eisentraut @ 2023-07-03 07:29 UTC (permalink / raw)
To: Andres Freund <[email protected]>; Thomas Munro <[email protected]>; +Cc: pgsql-hackers
On 10.06.23 07:26, Andres Freund wrote:
>> -###############################################################
>> -# Threading
>> -###############################################################
>> -
>> -# XXX: About to rely on thread safety in the autoconf build, so not worth
>> -# implementing a fallback.
>> -cdata.set('ENABLE_THREAD_SAFETY', 1)
>
> I wonder if we should just unconditionally set that in c.h or such? It'd not
> be crazy for external projects to rely on that being set.
We definitely should keep the mention in ecpg_config.h.in, since that is
explicitly put there for client code to use. We keep HAVE_LONG_LONG_INT
etc. there for similar reasons.
Another comment on patch 0003: I think this removal in configure.ac is
too much:
- else
- LDAP_LIBS_FE="-lldap $EXTRA_LDAP_LIBS"
This could would still be reachable for $enable_thread_safety=yes and
$thread_safe_libldap=yes, which I suppose is the normal case?
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-07-03 08:43 Heikki Linnakangas <[email protected]>
parent: Peter Eisentraut <[email protected]>
0 siblings, 2 replies; 35+ messages in thread
From: Heikki Linnakangas @ 2023-07-03 08:43 UTC (permalink / raw)
To: Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; Thomas Munro <[email protected]>; +Cc: pgsql-hackers
On 03/07/2023 10:29, Peter Eisentraut wrote:
> On 10.06.23 07:26, Andres Freund wrote:
>>> -###############################################################
>>> -# Threading
>>> -###############################################################
>>> -
>>> -# XXX: About to rely on thread safety in the autoconf build, so not worth
>>> -# implementing a fallback.
>>> -cdata.set('ENABLE_THREAD_SAFETY', 1)
>>
>> I wonder if we should just unconditionally set that in c.h or such? It'd not
>> be crazy for external projects to rely on that being set.
>
> We definitely should keep the mention in ecpg_config.h.in, since that is
> explicitly put there for client code to use. We keep HAVE_LONG_LONG_INT
> etc. there for similar reasons.
+1. Patches 1-3 look good to me, with the things Andres & Peter already
pointed out.
The docs at https://www.postgresql.org/docs/current/libpq-threading.html
needs updating. It's technically still accurate, but it ought to at
least mention that libpq on v17 and above is always thread-safe. I
propose the attached. It moves the note on "you can only use one PGConn
from one thread at a time" to the top, before the description of the
PQisthreadsafe() function.
On 10/06/2023 05:23, Thomas Munro wrote:
>
> 2. I don't like the way we have to deal with POSIX vs Windows at
> every site where we use threads, and each place has a different style
> of wrappers. I considered a few different approaches to cleaning this
> up:
>
> * provide centralised and thorough pthread emulation for Windows; I
> don't like this, I don't even like all of pthreads and there are many
> details to get lost in
> * adopt C11 <threads.h>; unfortunately it is too early, so you'd need
> to write/borrow replacements for at least 3 of our 11 target systems
> * invent our own mini-abstraction for a carefully controlled subset of stuff
Google search on "c11 threads on Windows" found some emulation wrappers:
https://github.com/jtsiomb/c11threads and
https://github.com/tinycthread/tinycthread, for example. Would either of
those work for us?
Even if we use an existing emulation wrapper, I wouldn't mind having our
own pg_* abstration on top of it, to document which subset of the POSIX
or C11 functions we actually use.
--
Heikki Linnakangas
Neon (https://neon.tech)
Attachments:
[text/x-patch] update-libpq-thread-safety-docs.patch (3.1K, ../../[email protected]/2-update-libpq-thread-safety-docs.patch)
download | inline diff:
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index 2225e4e0ef..bef03cfa16 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -9129,14 +9129,28 @@ void PQinitSSL(int do_ssl);
</indexterm>
<para>
- <application>libpq</application> is reentrant and thread-safe by default.
- You might need to use special compiler command-line
- options when you compile your application code. Refer to your
- system's documentation for information about how to build
- thread-enabled applications, or look in
- <filename>src/Makefile.global</filename> for <literal>PTHREAD_CFLAGS</literal>
- and <literal>PTHREAD_LIBS</literal>. This function allows the querying of
- <application>libpq</application>'s thread-safe status:
+ As of version 17, <application>libpq</application> is always reentrant and thread-safe.
+ However, one restriction is that no two threads attempt to manipulate
+ the same <structname>PGconn</structname> object at the same time. In particular,
+ you cannot issue concurrent commands from different threads through
+ the same connection object. (If you need to run concurrent commands,
+ use multiple connections.)
+ </para>
+
+ <para>
+ <structname>PGresult</structname> objects are normally read-only after creation,
+ and so can be passed around freely between threads. However, if you use
+ any of the <structname>PGresult</structname>-modifying functions described in
+ <xref linkend="libpq-misc"/> or <xref linkend="libpq-events"/>, it's up
+ to you to avoid concurrent operations on the same <structname>PGresult</structname>,
+ too.
+ </para>
+
+ </para>
+ In earlier versions, <application>libpq</application> could be compiled
+ with or without thread support, depending on compiler options. This
+ function allows the querying of <application>libpq</application>'s
+ thread-safe status:
</para>
<variablelist>
@@ -9154,29 +9168,12 @@ int PQisthreadsafe();
<para>
Returns 1 if the <application>libpq</application> is thread-safe
- and 0 if it is not.
+ and 0 if it is not. Always returns 1 on version 17 and above.
</para>
</listitem>
</varlistentry>
</variablelist>
- <para>
- One thread restriction is that no two threads attempt to manipulate
- the same <structname>PGconn</structname> object at the same time. In particular,
- you cannot issue concurrent commands from different threads through
- the same connection object. (If you need to run concurrent commands,
- use multiple connections.)
- </para>
-
- <para>
- <structname>PGresult</structname> objects are normally read-only after creation,
- and so can be passed around freely between threads. However, if you use
- any of the <structname>PGresult</structname>-modifying functions described in
- <xref linkend="libpq-misc"/> or <xref linkend="libpq-events"/>, it's up
- to you to avoid concurrent operations on the same <structname>PGresult</structname>,
- too.
- </para>
-
<para>
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-07-09 22:45 Thomas Munro <[email protected]>
parent: Heikki Linnakangas <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Thomas Munro @ 2023-07-09 22:45 UTC (permalink / raw)
To: Heikki Linnakangas <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
Thanks all for the reviews. I pushed the first two small patches.
Here is a new version of the main --disable-thread-safety removal
part, with these changes:
* fixed LDAP_LIBS_FE snafu reported by Peter E
* defined ENABLE_THREAD_SAFETY 1 in c.h, for the benefit of extensions
* defined ENABLE_THREAD_SAFETY 1 ecpg_config.h, for the benefit of ECPG clients
* added Heikki's doc change as a separate patch (with minor xml snafu fixed)
I'll follow up with a new version of the later pg_threads.h proposal
and responses to feedback on that after getting this basic clean-up
work in. Any other thoughts on this part?
Attachments:
[text/x-patch] v2-0001-Remove-disable-thread-safety-and-related-code.patch (60.3K, ../../CA+hUKGK+UC4kZmT62noORCJ5gTc+ephVdss+cj7LzhsBZjA-NQ@mail.gmail.com/2-v2-0001-Remove-disable-thread-safety-and-related-code.patch)
download | inline diff:
From 18975f437e6b8e7c36da99238a90e5efa68434b9 Mon Sep 17 00:00:00 2001
From: Thomas Munro <[email protected]>
Date: Fri, 9 Jun 2023 14:07:26 +1200
Subject: [PATCH v2 1/2] Remove --disable-thread-safety and related code.
All supported computers have either POSIX or Windows threads, and we no
longer have any automated testing of --disable-thread-safety. We define
a vestigial ENABLE_THREAD_SAFETY macro to 1 in c.h and ecpg_config.h for
the benefit of external code that might be testing for that, but we no
longer test it anywhere in PostgreSQL code, and associated dead code
paths are removed.
The Meson and perl-based Windows build scripts never had an equivalent
build option.
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
---
configure | 54 ++------------
configure.ac | 26 ++-----
doc/src/sgml/installation.sgml | 13 ----
meson.build | 11 ---
src/Makefile.global.in | 1 -
src/bin/pgbench/pgbench.c | 22 +-----
src/include/c.h | 7 ++
src/include/pg_config.h.in | 4 --
src/interfaces/ecpg/ecpglib/connect.c | 40 -----------
src/interfaces/ecpg/ecpglib/descriptor.c | 10 ---
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 2 -
src/interfaces/ecpg/ecpglib/execute.c | 2 -
src/interfaces/ecpg/ecpglib/memory.c | 7 --
src/interfaces/ecpg/ecpglib/misc.c | 47 ------------
.../ecpg/include/ecpg-pthread-win32.h | 3 -
src/interfaces/ecpg/include/ecpg_config.h.in | 5 +-
src/interfaces/ecpg/include/ecpglib.h | 2 -
src/interfaces/ecpg/include/meson.build | 3 +-
.../ecpg/test/expected/thread-alloc.c | 43 +++++------
.../ecpg/test/expected/thread-descriptor.c | 22 +++---
.../ecpg/test/expected/thread-prep.c | 71 ++++++++-----------
.../ecpg/test/expected/thread-thread.c | 63 +++++++---------
.../test/expected/thread-thread_implicit.c | 63 +++++++---------
src/interfaces/ecpg/test/thread/alloc.pgc | 9 ---
.../ecpg/test/thread/descriptor.pgc | 8 +--
src/interfaces/ecpg/test/thread/prep.pgc | 9 ---
src/interfaces/ecpg/test/thread/thread.pgc | 9 ---
.../ecpg/test/thread/thread_implicit.pgc | 9 ---
src/interfaces/libpq/Makefile | 5 +-
src/interfaces/libpq/fe-connect.c | 4 --
src/interfaces/libpq/fe-exec.c | 4 --
src/interfaces/libpq/fe-print.c | 13 +---
src/interfaces/libpq/fe-secure-openssl.c | 17 +----
src/interfaces/libpq/fe-secure.c | 26 +------
src/interfaces/libpq/legacy-pqsignal.c | 4 +-
src/interfaces/libpq/libpq-int.h | 9 +--
src/makefiles/meson.build | 1 -
src/tools/msvc/Solution.pm | 3 +-
src/tools/msvc/ecpg_regression.proj | 2 +-
39 files changed, 144 insertions(+), 509 deletions(-)
diff --git a/configure b/configure
index 4a8f652129..2e518c8007 100755
--- a/configure
+++ b/configure
@@ -722,7 +722,6 @@ with_tcl
ICU_LIBS
ICU_CFLAGS
with_icu
-enable_thread_safety
INCLUDES
autodepend
PKG_CONFIG_LIBDIR
@@ -848,7 +847,6 @@ with_CC
with_llvm
enable_depend
enable_cassert
-enable_thread_safety
with_icu
with_tcl
with_tclconfig
@@ -1536,7 +1534,6 @@ Optional Features:
--enable-tap-tests enable TAP tests (requires Perl and IPC::Run)
--enable-depend turn on automatic dependency tracking
--enable-cassert enable assertion checks (for debugging)
- --disable-thread-safety disable thread-safety in client libraries
--disable-largefile omit support for large files
Optional Packages:
@@ -8338,43 +8335,6 @@ $as_echo "$as_me: WARNING: *** Library directory $dir does not exist." >&2;}
done
IFS=$ac_save_IFS
-#
-# Enable thread-safe client libraries
-#
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking allow thread-safe client libraries" >&5
-$as_echo_n "checking allow thread-safe client libraries... " >&6; }
-
-
-# Check whether --enable-thread-safety was given.
-if test "${enable_thread_safety+set}" = set; then :
- enableval=$enable_thread_safety;
- case $enableval in
- yes)
- :
- ;;
- no)
- :
- ;;
- *)
- as_fn_error $? "no argument expected for --enable-thread-safety option" "$LINENO" 5
- ;;
- esac
-
-else
- enable_thread_safety=yes
-
-fi
-
-
-if test "$enable_thread_safety" = yes; then
-
-$as_echo "#define ENABLE_THREAD_SAFETY 1" >>confdefs.h
-
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $enable_thread_safety" >&5
-$as_echo "$enable_thread_safety" >&6; }
-
-
#
# ICU
#
@@ -11064,7 +11024,7 @@ fi
done
-if test "$enable_thread_safety" = yes -a "$PORTNAME" != "win32"; then :
+if test "$PORTNAME" != "win32"; then :
# then
@@ -11723,7 +11683,7 @@ if test "x$ac_cv_header_pthread_h" = xyes; then :
else
as_fn_error $? "
-pthread.h not found; use --disable-thread-safety to disable thread safety" "$LINENO" 5
+pthread.h not found" "$LINENO" 5
fi
@@ -12407,8 +12367,7 @@ if test "$ac_res" != no; then :
fi
-if test "$enable_thread_safety" = yes; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing pthread_barrier_wait" >&5
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing pthread_barrier_wait" >&5
$as_echo_n "checking for library containing pthread_barrier_wait... " >&6; }
if ${ac_cv_search_pthread_barrier_wait+:} false; then :
$as_echo_n "(cached) " >&6
@@ -12464,7 +12423,6 @@ if test "$ac_res" != no; then :
fi
-fi
if test "$with_readline" = yes; then
@@ -13349,7 +13307,7 @@ else
thread_safe_libldap=no
fi
- if test "$enable_thread_safety" = yes -a "$thread_safe_libldap" = no; then
+ if test "$thread_safe_libldap" = no; then
# Use ldap_r for FE if available, else assume ldap is thread-safe.
# On some platforms ldap_r fails to link without PTHREAD_LIBS.
LIBS="$_LIBS"
@@ -16377,8 +16335,7 @@ fi
-if test "$enable_thread_safety" = yes; then
- ac_fn_c_check_func "$LINENO" "pthread_barrier_wait" "ac_cv_func_pthread_barrier_wait"
+ac_fn_c_check_func "$LINENO" "pthread_barrier_wait" "ac_cv_func_pthread_barrier_wait"
if test "x$ac_cv_func_pthread_barrier_wait" = xyes; then :
$as_echo "#define HAVE_PTHREAD_BARRIER_WAIT 1" >>confdefs.h
@@ -16392,7 +16349,6 @@ esac
fi
-fi
if test "$PORTNAME" = "win32" -o "$PORTNAME" = "cygwin"; then
# Cygwin and (apparently, based on test results) Mingw both
diff --git a/configure.ac b/configure.ac
index 0f802f3df4..3ebe1a796d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -837,18 +837,6 @@ for dir in $LIBRARY_DIRS $SRCH_LIB; do
done
IFS=$ac_save_IFS
-#
-# Enable thread-safe client libraries
-#
-AC_MSG_CHECKING([allow thread-safe client libraries])
-PGAC_ARG_BOOL(enable, thread-safety, yes, [disable thread-safety in client libraries])
-if test "$enable_thread_safety" = yes; then
- AC_DEFINE([ENABLE_THREAD_SAFETY], 1,
- [Define to 1 to build client libraries as thread-safe code. (--enable-thread-safety)])
-fi
-AC_MSG_RESULT([$enable_thread_safety])
-AC_SUBST(enable_thread_safety)
-
#
# ICU
#
@@ -1243,7 +1231,7 @@ dnl note: We have to use AS_IF here rather than plain if. The AC_CHECK_HEADER
dnl invocation below is the first one in the script, and autoconf generates
dnl additional code for that, which must not be inside the if-block. AS_IF
dnl knows how to do that.
-AS_IF([test "$enable_thread_safety" = yes -a "$PORTNAME" != "win32"],
+AS_IF([test "$PORTNAME" != "win32"],
[ # then
AX_PTHREAD # set thread flags
@@ -1258,7 +1246,7 @@ CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
LIBS="$LIBS $PTHREAD_LIBS"
AC_CHECK_HEADER(pthread.h, [], [AC_MSG_ERROR([
-pthread.h not found; use --disable-thread-safety to disable thread safety])])
+pthread.h not found])])
AC_CHECK_FUNCS([strerror_r])
@@ -1305,9 +1293,7 @@ AC_SEARCH_LIBS(shmget, cygipc)
# *BSD:
AC_SEARCH_LIBS(backtrace_symbols, execinfo)
-if test "$enable_thread_safety" = yes; then
- AC_SEARCH_LIBS(pthread_barrier_wait, pthread)
-fi
+AC_SEARCH_LIBS(pthread_barrier_wait, pthread)
if test "$with_readline" = yes; then
PGAC_CHECK_READLINE
@@ -1434,7 +1420,7 @@ if test "$with_ldap" = yes ; then
AC_CHECK_FUNC([ldap_verify_credentials],
[thread_safe_libldap=yes],
[thread_safe_libldap=no])
- if test "$enable_thread_safety" = yes -a "$thread_safe_libldap" = no; then
+ if test "$thread_safe_libldap" = no; then
# Use ldap_r for FE if available, else assume ldap is thread-safe.
# On some platforms ldap_r fails to link without PTHREAD_LIBS.
LIBS="$_LIBS"
@@ -1858,9 +1844,7 @@ AC_REPLACE_FUNCS(m4_normalize([
strnlen
]))
-if test "$enable_thread_safety" = yes; then
- AC_REPLACE_FUNCS(pthread_barrier_wait)
-fi
+AC_REPLACE_FUNCS(pthread_barrier_wait)
if test "$PORTNAME" = "win32" -o "$PORTNAME" = "cygwin"; then
# Cygwin and (apparently, based on test results) Mingw both
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index e9d797417a..ac8eee47c6 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -1306,19 +1306,6 @@ build-postgresql:
</listitem>
</varlistentry>
- <varlistentry id="configure-option-disable-thread-safety">
- <term><option>--disable-thread-safety</option></term>
- <listitem>
- <para>
- Disable the thread-safety of client libraries. This prevents
- concurrent threads in <application>libpq</application> and
- <application>ECPG</application> programs from safely controlling
- their private connection handles. Use this only on platforms
- with deficient threading support.
- </para>
- </listitem>
- </varlistentry>
-
</variablelist>
</sect3>
diff --git a/meson.build b/meson.build
index 0c44f19cb9..30473b951a 100644
--- a/meson.build
+++ b/meson.build
@@ -2508,17 +2508,6 @@ cdata.set_quoted('PG_VERSION_STR',
)
-
-###############################################################
-# Threading
-###############################################################
-
-# XXX: About to rely on thread safety in the autoconf build, so not worth
-# implementing a fallback.
-cdata.set('ENABLE_THREAD_SAFETY', 1)
-
-
-
###############################################################
# NLS / Gettext
###############################################################
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 974b1dfef9..df9f721a41 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -202,7 +202,6 @@ enable_debug = @enable_debug@
enable_dtrace = @enable_dtrace@
enable_coverage = @enable_coverage@
enable_tap_tests = @enable_tap_tests@
-enable_thread_safety = @enable_thread_safety@
python_includespec = @python_includespec@
python_libdir = @python_libdir@
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 1d1670d4c2..683fc5860f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -137,7 +137,7 @@ typedef struct socket_set
EnterSynchronizationBarrier((barrier), \
SYNCHRONIZATION_BARRIER_FLAGS_BLOCK_ONLY)
#define THREAD_BARRIER_DESTROY(barrier)
-#elif defined(ENABLE_THREAD_SAFETY)
+#else
/* Use POSIX threads */
#include "port/pg_pthread.h"
#define THREAD_T pthread_t
@@ -153,16 +153,6 @@ typedef struct socket_set
pthread_barrier_init((barrier), NULL, (n))
#define THREAD_BARRIER_WAIT(barrier) pthread_barrier_wait((barrier))
#define THREAD_BARRIER_DESTROY(barrier) pthread_barrier_destroy((barrier))
-#else
-/* No threads implementation, use none (-j 1) */
-#define THREAD_T void *
-#define THREAD_FUNC_RETURN_TYPE void *
-#define THREAD_FUNC_RETURN return NULL
-#define THREAD_FUNC_CC
-#define THREAD_BARRIER_T int
-#define THREAD_BARRIER_INIT(barrier, n) (*(barrier) = 0)
-#define THREAD_BARRIER_WAIT(barrier)
-#define THREAD_BARRIER_DESTROY(barrier)
#endif
@@ -6749,10 +6739,6 @@ main(int argc, char **argv)
{
exit(1);
}
-#ifndef ENABLE_THREAD_SAFETY
- if (nthreads != 1)
- pg_fatal("threads are not supported on this platform; use -j1");
-#endif /* !ENABLE_THREAD_SAFETY */
break;
case 'l':
benchmarking_option_set = true;
@@ -7236,7 +7222,6 @@ main(int argc, char **argv)
if (errno != 0)
pg_fatal("could not initialize barrier: %m");
-#ifdef ENABLE_THREAD_SAFETY
/* start all threads but thread 0 which is executed directly later */
for (i = 1; i < nthreads; i++)
{
@@ -7248,9 +7233,6 @@ main(int argc, char **argv)
if (errno != 0)
pg_fatal("could not create thread: %m");
}
-#else
- Assert(nthreads == 1);
-#endif /* ENABLE_THREAD_SAFETY */
/* compute when to stop */
threads[0].create_time = pg_time_now();
@@ -7268,10 +7250,8 @@ main(int argc, char **argv)
{
TState *thread = &threads[i];
-#ifdef ENABLE_THREAD_SAFETY
if (i > 0)
THREAD_JOIN(thread->thread);
-#endif /* ENABLE_THREAD_SAFETY */
for (int j = 0; j < thread->nstate; j++)
if (thread->state[j].state != CSTATE_FINISHED)
diff --git a/src/include/c.h b/src/include/c.h
index f69d739be5..080443fc68 100644
--- a/src/include/c.h
+++ b/src/include/c.h
@@ -1371,6 +1371,13 @@ typedef intptr_t sigjmp_buf[5];
#endif /* __MINGW64__ */
#endif /* WIN32 */
+/*
+ * Settings that were historically compile-time options, but are no longer.
+ * We retain these macros for the benefit of extension code and client code
+ * that might test them.
+ */
+#define ENABLE_THREAD_SAFETY 1
+
/* /port compatibility functions */
#include "port.h"
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index d03f6e8de8..ee209d6d70 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -51,10 +51,6 @@
/* Define to 1 if you want National Language Support. (--enable-nls) */
#undef ENABLE_NLS
-/* Define to 1 to build client libraries as thread-safe code.
- (--enable-thread-safety) */
-#undef ENABLE_THREAD_SAFETY
-
/* Define to 1 if you have the `append_history' function. */
#undef HAVE_APPEND_HISTORY
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index db0bae1fe0..8afb1f0a26 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -14,15 +14,12 @@
locale_t ecpg_clocale = (locale_t) 0;
#endif
-#ifdef ENABLE_THREAD_SAFETY
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
-#endif
static struct connection *actual_connection = NULL;
static struct connection *all_connections = NULL;
-#ifdef ENABLE_THREAD_SAFETY
static void
ecpg_actual_connection_init(void)
{
@@ -34,7 +31,6 @@ ecpg_pthreads_init(void)
{
pthread_once(&actual_connection_key_once, ecpg_actual_connection_init);
}
-#endif
static struct connection *
ecpg_get_connection_nr(const char *connection_name)
@@ -43,7 +39,6 @@ ecpg_get_connection_nr(const char *connection_name)
if ((connection_name == NULL) || (strcmp(connection_name, "CURRENT") == 0))
{
-#ifdef ENABLE_THREAD_SAFETY
ecpg_pthreads_init(); /* ensure actual_connection_key is valid */
ret = pthread_getspecific(actual_connection_key);
@@ -56,9 +51,6 @@ ecpg_get_connection_nr(const char *connection_name)
if (ret == NULL)
/* no TSD connection, going for global */
ret = actual_connection;
-#else
- ret = actual_connection;
-#endif
}
else
{
@@ -82,7 +74,6 @@ ecpg_get_connection(const char *connection_name)
if ((connection_name == NULL) || (strcmp(connection_name, "CURRENT") == 0))
{
-#ifdef ENABLE_THREAD_SAFETY
ecpg_pthreads_init(); /* ensure actual_connection_key is valid */
ret = pthread_getspecific(actual_connection_key);
@@ -95,21 +86,14 @@ ecpg_get_connection(const char *connection_name)
if (ret == NULL)
/* no TSD connection here either, using global */
ret = actual_connection;
-#else
- ret = actual_connection;
-#endif
}
else
{
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_lock(&connections_mutex);
-#endif
ret = ecpg_get_connection_nr(connection_name);
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&connections_mutex);
-#endif
}
return ret;
@@ -143,10 +127,8 @@ ecpg_finish(struct connection *act)
con->next = act->next;
}
-#ifdef ENABLE_THREAD_SAFETY
if (pthread_getspecific(actual_connection_key) == act)
pthread_setspecific(actual_connection_key, all_connections);
-#endif
if (actual_connection == act)
actual_connection = all_connections;
@@ -212,11 +194,7 @@ ECPGsetconn(int lineno, const char *connection_name)
if (!ecpg_init(con, connection_name, lineno))
return false;
-#ifdef ENABLE_THREAD_SAFETY
pthread_setspecific(actual_connection_key, con);
-#else
- actual_connection = con;
-#endif
return true;
}
@@ -326,9 +304,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
if (dbname == NULL && connection_name == NULL)
connection_name = "DEFAULT";
-#if ENABLE_THREAD_SAFETY
ecpg_pthreads_init();
-#endif
/* check if the identifier is unique */
if (ecpg_get_connection(connection_name))
@@ -505,9 +481,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
}
/* add connection to our list */
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_lock(&connections_mutex);
-#endif
/*
* ... but first, make certain we have created ecpg_clocale. Rely on
@@ -519,9 +493,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
if (!ecpg_clocale)
{
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&connections_mutex);
-#endif
ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
if (host)
@@ -558,9 +530,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
this->next = all_connections;
all_connections = this;
-#ifdef ENABLE_THREAD_SAFETY
pthread_setspecific(actual_connection_key, all_connections);
-#endif
actual_connection = all_connections;
ecpg_log("ECPGconnect: opening database %s on %s port %s %s%s %s%s\n",
@@ -678,9 +648,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
ecpg_log("ECPGconnect: %s", errmsg);
ecpg_finish(this);
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&connections_mutex);
-#endif
ecpg_raise(lineno, ECPG_CONNECT, ECPG_SQLSTATE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION, db);
if (realname)
@@ -692,9 +660,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
if (realname)
ecpg_free(realname);
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&connections_mutex);
-#endif
this->autocommit = autocommit;
@@ -716,9 +682,7 @@ ECPGdisconnect(int lineno, const char *connection_name)
return false;
}
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_lock(&connections_mutex);
-#endif
if (strcmp(connection_name, "ALL") == 0)
{
@@ -737,18 +701,14 @@ ECPGdisconnect(int lineno, const char *connection_name)
if (!ecpg_init(con, connection_name, lineno))
{
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&connections_mutex);
-#endif
return false;
}
else
ecpg_finish(con);
}
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&connections_mutex);
-#endif
return true;
}
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index 883a210a81..ad279e245c 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -19,7 +19,6 @@
static void descriptor_free(struct descriptor *desc);
/* We manage descriptors separately for each thread. */
-#ifdef ENABLE_THREAD_SAFETY
static pthread_key_t descriptor_key;
static pthread_once_t descriptor_once = PTHREAD_ONCE_INIT;
@@ -49,12 +48,6 @@ set_descriptors(struct descriptor *value)
{
pthread_setspecific(descriptor_key, value);
}
-#else
-static struct descriptor *all_descriptors = NULL;
-
-#define get_descriptors() (all_descriptors)
-#define set_descriptors(value) do { all_descriptors = (value); } while(0)
-#endif
/* old internal convenience function that might go away later */
static PGresult *
@@ -782,8 +775,6 @@ ECPGdeallocate_desc(int line, const char *name)
return false;
}
-#ifdef ENABLE_THREAD_SAFETY
-
/* Deallocate all descriptors in the list */
static void
descriptor_deallocate_all(struct descriptor *list)
@@ -796,7 +787,6 @@ descriptor_deallocate_all(struct descriptor *list)
list = next;
}
}
-#endif /* ENABLE_THREAD_SAFETY */
bool
ECPGallocate_desc(int line, const char *name)
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 8b8f081f27..01b4309a71 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -169,9 +169,7 @@ bool ecpg_get_data(const PGresult *, int, int, int, enum ECPGttype type,
enum ECPGttype, char *, char *, long, long, long,
enum ARRAY_TYPE, enum COMPAT_MODE, bool);
-#ifdef ENABLE_THREAD_SAFETY
void ecpg_pthreads_init(void);
-#endif
struct connection *ecpg_get_connection(const char *connection_name);
char *ecpg_alloc(long size, int lineno);
char *ecpg_auto_alloc(long size, int lineno);
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 93926fd4fb..04d0b40c53 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -1961,9 +1961,7 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
return false;
}
-#ifdef ENABLE_THREAD_SAFETY
ecpg_pthreads_init();
-#endif
con = ecpg_get_connection(connection_name);
diff --git a/src/interfaces/ecpg/ecpglib/memory.c b/src/interfaces/ecpg/ecpglib/memory.c
index bd81251054..a83637ac75 100644
--- a/src/interfaces/ecpg/ecpglib/memory.c
+++ b/src/interfaces/ecpg/ecpglib/memory.c
@@ -68,7 +68,6 @@ struct auto_mem
struct auto_mem *next;
};
-#ifdef ENABLE_THREAD_SAFETY
static pthread_key_t auto_mem_key;
static pthread_once_t auto_mem_once = PTHREAD_ONCE_INIT;
@@ -97,12 +96,6 @@ set_auto_allocs(struct auto_mem *am)
{
pthread_setspecific(auto_mem_key, am);
}
-#else
-static struct auto_mem *auto_allocs = NULL;
-
-#define get_auto_allocs() (auto_allocs)
-#define set_auto_allocs(am) do { auto_allocs = (am); } while(0)
-#endif
char *
ecpg_auto_alloc(long size, int lineno)
diff --git a/src/interfaces/ecpg/ecpglib/misc.c b/src/interfaces/ecpg/ecpglib/misc.c
index 7f75e18733..2b78caeaf5 100644
--- a/src/interfaces/ecpg/ecpglib/misc.c
+++ b/src/interfaces/ecpg/ecpglib/misc.c
@@ -55,42 +55,11 @@ static struct sqlca_t sqlca_init =
}
};
-#ifdef ENABLE_THREAD_SAFETY
static pthread_key_t sqlca_key;
static pthread_once_t sqlca_key_once = PTHREAD_ONCE_INIT;
-#else
-static struct sqlca_t sqlca =
-{
- {
- 'S', 'Q', 'L', 'C', 'A', ' ', ' ', ' '
- },
- sizeof(struct sqlca_t),
- 0,
- {
- 0,
- {
- 0
- }
- },
- {
- 'N', 'O', 'T', ' ', 'S', 'E', 'T', ' '
- },
- {
- 0, 0, 0, 0, 0, 0
- },
- {
- 0, 0, 0, 0, 0, 0, 0, 0
- },
- {
- '0', '0', '0', '0', '0'
- }
-};
-#endif
-#ifdef ENABLE_THREAD_SAFETY
static pthread_mutex_t debug_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_mutex_t debug_init_mutex = PTHREAD_MUTEX_INITIALIZER;
-#endif
static int simple_debug = 0;
static FILE *debugstream = NULL;
@@ -123,7 +92,6 @@ ecpg_init(const struct connection *con, const char *connection_name, const int l
return true;
}
-#ifdef ENABLE_THREAD_SAFETY
static void
ecpg_sqlca_key_destructor(void *arg)
{
@@ -135,12 +103,10 @@ ecpg_sqlca_key_init(void)
{
pthread_key_create(&sqlca_key, ecpg_sqlca_key_destructor);
}
-#endif
struct sqlca_t *
ECPGget_sqlca(void)
{
-#ifdef ENABLE_THREAD_SAFETY
struct sqlca_t *sqlca;
pthread_once(&sqlca_key_once, ecpg_sqlca_key_init);
@@ -155,9 +121,6 @@ ECPGget_sqlca(void)
pthread_setspecific(sqlca_key, sqlca);
}
return sqlca;
-#else
- return &sqlca;
-#endif
}
bool
@@ -240,9 +203,7 @@ ECPGtrans(int lineno, const char *connection_name, const char *transaction)
void
ECPGdebug(int n, FILE *dbgs)
{
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_lock(&debug_init_mutex);
-#endif
if (n > 100)
{
@@ -256,9 +217,7 @@ ECPGdebug(int n, FILE *dbgs)
ecpg_log("ECPGdebug: set to %d\n", simple_debug);
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&debug_init_mutex);
-#endif
}
void
@@ -290,9 +249,7 @@ ecpg_log(const char *format,...)
else
snprintf(fmt, bufsize, "[%d]: %s", (int) getpid(), intl_format);
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_lock(&debug_mutex);
-#endif
va_start(ap, format);
vfprintf(debugstream, fmt, ap);
@@ -307,9 +264,7 @@ ecpg_log(const char *format,...)
fflush(debugstream);
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&debug_mutex);
-#endif
free(fmt);
}
@@ -451,7 +406,6 @@ ECPGis_noind_null(enum ECPGttype type, const void *ptr)
}
#ifdef WIN32
-#ifdef ENABLE_THREAD_SAFETY
void
win32_pthread_mutex(volatile pthread_mutex_t *mutex)
@@ -482,7 +436,6 @@ win32_pthread_once(volatile pthread_once_t *once, void (*fn) (void))
pthread_mutex_unlock(&win32_pthread_once_lock);
}
}
-#endif /* ENABLE_THREAD_SAFETY */
#endif /* WIN32 */
#ifdef ENABLE_NLS
diff --git a/src/interfaces/ecpg/include/ecpg-pthread-win32.h b/src/interfaces/ecpg/include/ecpg-pthread-win32.h
index 33c897b633..8252a17809 100644
--- a/src/interfaces/ecpg/include/ecpg-pthread-win32.h
+++ b/src/interfaces/ecpg/include/ecpg-pthread-win32.h
@@ -5,8 +5,6 @@
#ifndef _ECPG_PTHREAD_WIN32_H
#define _ECPG_PTHREAD_WIN32_H
-#ifdef ENABLE_THREAD_SAFETY
-
#ifndef WIN32
#include <pthread.h>
@@ -53,6 +51,5 @@ void win32_pthread_once(volatile pthread_once_t *once, void (*fn) (void));
win32_pthread_once((once), (fn)); \
} while(0)
#endif /* WIN32 */
-#endif /* ENABLE_THREAD_SAFETY */
#endif /* _ECPG_PTHREAD_WIN32_H */
diff --git a/src/interfaces/ecpg/include/ecpg_config.h.in b/src/interfaces/ecpg/include/ecpg_config.h.in
index cbd24f11a0..5d0f448a86 100644
--- a/src/interfaces/ecpg/include/ecpg_config.h.in
+++ b/src/interfaces/ecpg/include/ecpg_config.h.in
@@ -1,6 +1,5 @@
-/* Define to 1 to build client libraries as thread-safe code.
- * (--enable-thread-safety) */
-#undef ENABLE_THREAD_SAFETY
+/* Define to 1 to build client libraries as thread-safe code. */
+#define ENABLE_THREAD_SAFETY 1
/* Define to 1 if the system has the type `int64'. */
#undef HAVE_INT64
diff --git a/src/interfaces/ecpg/include/ecpglib.h b/src/interfaces/ecpg/include/ecpglib.h
index 21a2134483..771761ffe4 100644
--- a/src/interfaces/ecpg/include/ecpglib.h
+++ b/src/interfaces/ecpg/include/ecpglib.h
@@ -98,9 +98,7 @@ void *ECPGget_var(int number);
/* dynamic result allocation */
void ECPGfree_auto_mem(void);
-#ifdef ENABLE_THREAD_SAFETY
void ecpg_pthreads_init(void);
-#endif
#ifdef __cplusplus
}
diff --git a/src/interfaces/ecpg/include/meson.build b/src/interfaces/ecpg/include/meson.build
index 2278f0d305..5dad643a0c 100644
--- a/src/interfaces/ecpg/include/meson.build
+++ b/src/interfaces/ecpg/include/meson.build
@@ -3,7 +3,6 @@
ecpg_inc = include_directories('.')
ecpg_conf_keys = [
- 'ENABLE_THREAD_SAFETY',
'HAVE_INT64',
'HAVE_LONG_INT_64',
'HAVE_LONG_LONG_INT_64',
@@ -12,6 +11,8 @@ ecpg_conf_keys = [
ecpg_conf_data = configuration_data()
+ecpg_conf_data.set('ENABLE_THREAD_SAFETY', 1)
+
foreach key : ecpg_conf_keys
if cdata.has(key)
ecpg_conf_data.set(key, cdata.get(key))
diff --git a/src/interfaces/ecpg/test/expected/thread-alloc.c b/src/interfaces/ecpg/test/expected/thread-alloc.c
index 37ef44ed94..3b31d27fd3 100644
--- a/src/interfaces/ecpg/test/expected/thread-alloc.c
+++ b/src/interfaces/ecpg/test/expected/thread-alloc.c
@@ -11,14 +11,6 @@
#include <stdlib.h>
#include "ecpg_config.h"
-#ifndef ENABLE_THREAD_SAFETY
-int
-main(void)
-{
- printf("No threading enabled.\n");
- return 0;
-}
-#else
#ifdef WIN32
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
@@ -101,7 +93,7 @@ struct sqlca_t *ECPGget_sqlca(void);
#endif
-#line 26 "alloc.pgc"
+#line 18 "alloc.pgc"
#line 1 "regression.h"
@@ -111,14 +103,14 @@ struct sqlca_t *ECPGget_sqlca(void);
-#line 27 "alloc.pgc"
+#line 19 "alloc.pgc"
/* exec sql whenever sqlerror sqlprint ; */
-#line 29 "alloc.pgc"
+#line 21 "alloc.pgc"
/* exec sql whenever not found sqlprint ; */
-#line 30 "alloc.pgc"
+#line 22 "alloc.pgc"
#ifdef WIN32
@@ -134,54 +126,54 @@ static void* fn(void* arg)
-#line 41 "alloc.pgc"
+#line 33 "alloc.pgc"
int value ;
-#line 42 "alloc.pgc"
+#line 34 "alloc.pgc"
char name [ 100 ] ;
-#line 43 "alloc.pgc"
+#line 35 "alloc.pgc"
char ** r = NULL ;
/* exec sql end declare section */
-#line 44 "alloc.pgc"
+#line 36 "alloc.pgc"
value = (intptr_t) arg;
sprintf(name, "Connection: %d", value);
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , name, 0);
-#line 49 "alloc.pgc"
+#line 41 "alloc.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 49 "alloc.pgc"
+#line 41 "alloc.pgc"
{ ECPGsetcommit(__LINE__, "on", NULL);
-#line 50 "alloc.pgc"
+#line 42 "alloc.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 50 "alloc.pgc"
+#line 42 "alloc.pgc"
for (i = 1; i <= REPEATS; ++i)
{
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "select relname from pg_class where relname = 'pg_class'", ECPGt_EOIT,
ECPGt_char,&(r),(long)0,(long)0,(1)*sizeof(char),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);
-#line 53 "alloc.pgc"
+#line 45 "alloc.pgc"
if (sqlca.sqlcode == ECPG_NOT_FOUND) sqlprint();
-#line 53 "alloc.pgc"
+#line 45 "alloc.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 53 "alloc.pgc"
+#line 45 "alloc.pgc"
free(r);
r = NULL;
}
{ ECPGdisconnect(__LINE__, name);
-#line 57 "alloc.pgc"
+#line 49 "alloc.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 57 "alloc.pgc"
+#line 49 "alloc.pgc"
return 0;
@@ -215,4 +207,3 @@ int main ()
return 0;
}
-#endif
diff --git a/src/interfaces/ecpg/test/expected/thread-descriptor.c b/src/interfaces/ecpg/test/expected/thread-descriptor.c
index f56cc25ab0..e34f4708d1 100644
--- a/src/interfaces/ecpg/test/expected/thread-descriptor.c
+++ b/src/interfaces/ecpg/test/expected/thread-descriptor.c
@@ -7,7 +7,6 @@
#define ECPGdebug(X,Y) ECPGdebug((X)+100,(Y))
#line 1 "descriptor.pgc"
-#ifdef ENABLE_THREAD_SAFETY
#ifdef WIN32
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
@@ -16,7 +15,6 @@
#else
#include <pthread.h>
#endif
-#endif
#include <stdio.h>
#define THREADS 16
@@ -91,16 +89,16 @@ struct sqlca_t *ECPGget_sqlca(void);
#endif
-#line 16 "descriptor.pgc"
+#line 14 "descriptor.pgc"
/* exec sql whenever sqlerror sqlprint ; */
-#line 17 "descriptor.pgc"
+#line 15 "descriptor.pgc"
/* exec sql whenever not found sqlprint ; */
-#line 18 "descriptor.pgc"
+#line 16 "descriptor.pgc"
-#if defined(ENABLE_THREAD_SAFETY) && defined(WIN32)
+#if defined(WIN32)
static unsigned __stdcall fn(void* arg)
#else
static void* fn(void* arg)
@@ -111,16 +109,16 @@ static void* fn(void* arg)
for (i = 1; i <= REPEATS; ++i)
{
ECPGallocate_desc(__LINE__, "mydesc");
-#line 30 "descriptor.pgc"
+#line 28 "descriptor.pgc"
if (sqlca.sqlcode < 0) sqlprint();
-#line 30 "descriptor.pgc"
+#line 28 "descriptor.pgc"
ECPGdeallocate_desc(__LINE__, "mydesc");
-#line 31 "descriptor.pgc"
+#line 29 "descriptor.pgc"
if (sqlca.sqlcode < 0) sqlprint();
-#line 31 "descriptor.pgc"
+#line 29 "descriptor.pgc"
}
@@ -129,7 +127,6 @@ if (sqlca.sqlcode < 0) sqlprint();
int main ()
{
-#ifdef ENABLE_THREAD_SAFETY
int i;
#ifdef WIN32
HANDLE threads[THREADS];
@@ -153,9 +150,6 @@ int main ()
for (i = 0; i < THREADS; ++i)
pthread_join(threads[i], NULL);
#endif
-#else
- fn(NULL);
-#endif
return 0;
}
diff --git a/src/interfaces/ecpg/test/expected/thread-prep.c b/src/interfaces/ecpg/test/expected/thread-prep.c
index 7cdf2505d3..052e27b634 100644
--- a/src/interfaces/ecpg/test/expected/thread-prep.c
+++ b/src/interfaces/ecpg/test/expected/thread-prep.c
@@ -11,14 +11,6 @@
#include <stdlib.h>
#include "ecpg_config.h"
-#ifndef ENABLE_THREAD_SAFETY
-int
-main(void)
-{
- printf("No threading enabled.\n");
- return 0;
-}
-#else
#ifdef WIN32
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
@@ -101,7 +93,7 @@ struct sqlca_t *ECPGget_sqlca(void);
#endif
-#line 26 "prep.pgc"
+#line 18 "prep.pgc"
#line 1 "regression.h"
@@ -111,14 +103,14 @@ struct sqlca_t *ECPGget_sqlca(void);
-#line 27 "prep.pgc"
+#line 19 "prep.pgc"
/* exec sql whenever sqlerror sqlprint ; */
-#line 29 "prep.pgc"
+#line 21 "prep.pgc"
/* exec sql whenever not found sqlprint ; */
-#line 30 "prep.pgc"
+#line 22 "prep.pgc"
#ifdef WIN32
@@ -134,64 +126,64 @@ static void* fn(void* arg)
-#line 41 "prep.pgc"
+#line 33 "prep.pgc"
int value ;
-#line 42 "prep.pgc"
+#line 34 "prep.pgc"
char name [ 100 ] ;
-#line 43 "prep.pgc"
+#line 35 "prep.pgc"
char query [ 256 ] = "INSERT INTO T VALUES ( ? )" ;
/* exec sql end declare section */
-#line 44 "prep.pgc"
+#line 36 "prep.pgc"
value = (intptr_t) arg;
sprintf(name, "Connection: %d", value);
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , name, 0);
-#line 49 "prep.pgc"
+#line 41 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 49 "prep.pgc"
+#line 41 "prep.pgc"
{ ECPGsetcommit(__LINE__, "on", NULL);
-#line 50 "prep.pgc"
+#line 42 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 50 "prep.pgc"
+#line 42 "prep.pgc"
for (i = 1; i <= REPEATS; ++i)
{
{ ECPGprepare(__LINE__, NULL, 0, "i", query);
-#line 53 "prep.pgc"
+#line 45 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 53 "prep.pgc"
+#line 45 "prep.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_execute, "i",
ECPGt_int,&(value),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EOIT, ECPGt_EORT);
-#line 54 "prep.pgc"
+#line 46 "prep.pgc"
if (sqlca.sqlcode == ECPG_NOT_FOUND) sqlprint();
-#line 54 "prep.pgc"
+#line 46 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 54 "prep.pgc"
+#line 46 "prep.pgc"
}
{ ECPGdeallocate(__LINE__, 0, NULL, "i");
-#line 56 "prep.pgc"
+#line 48 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 56 "prep.pgc"
+#line 48 "prep.pgc"
{ ECPGdisconnect(__LINE__, name);
-#line 57 "prep.pgc"
+#line 49 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 57 "prep.pgc"
+#line 49 "prep.pgc"
return 0;
@@ -207,34 +199,34 @@ int main ()
#endif
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0);
-#line 71 "prep.pgc"
+#line 63 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 71 "prep.pgc"
+#line 63 "prep.pgc"
{ ECPGsetcommit(__LINE__, "on", NULL);
-#line 72 "prep.pgc"
+#line 64 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 72 "prep.pgc"
+#line 64 "prep.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "drop table if exists T", ECPGt_EOIT, ECPGt_EORT);
-#line 73 "prep.pgc"
+#line 65 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 73 "prep.pgc"
+#line 65 "prep.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "create table T ( i int )", ECPGt_EOIT, ECPGt_EORT);
-#line 74 "prep.pgc"
+#line 66 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 74 "prep.pgc"
+#line 66 "prep.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");
-#line 75 "prep.pgc"
+#line 67 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 75 "prep.pgc"
+#line 67 "prep.pgc"
#ifdef WIN32
@@ -256,4 +248,3 @@ if (sqlca.sqlcode < 0) sqlprint();}
return 0;
}
-#endif
diff --git a/src/interfaces/ecpg/test/expected/thread-thread.c b/src/interfaces/ecpg/test/expected/thread-thread.c
index 0e75c47fab..95faa223c2 100644
--- a/src/interfaces/ecpg/test/expected/thread-thread.c
+++ b/src/interfaces/ecpg/test/expected/thread-thread.c
@@ -15,14 +15,6 @@
#include <stdlib.h>
#include "ecpg_config.h"
-#ifndef ENABLE_THREAD_SAFETY
-int
-main(void)
-{
- printf("No threading enabled.\n");
- return 0;
-}
-#else
#ifndef WIN32
#include <pthread.h>
#else
@@ -38,7 +30,7 @@ main(void)
-#line 24 "thread.pgc"
+#line 16 "thread.pgc"
void *test_thread(void *arg);
@@ -57,10 +49,10 @@ int main()
/* exec sql begin declare section */
-#line 40 "thread.pgc"
+#line 32 "thread.pgc"
int l_rows ;
/* exec sql end declare section */
-#line 41 "thread.pgc"
+#line 33 "thread.pgc"
/* Do not switch on debug output for regression tests. The threads get executed in
@@ -69,22 +61,22 @@ int main()
/* setup test_thread table */
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0); }
-#line 48 "thread.pgc"
+#line 40 "thread.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "drop table test_thread", ECPGt_EOIT, ECPGt_EORT);}
-#line 49 "thread.pgc"
+#line 41 "thread.pgc"
/* DROP might fail */
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 50 "thread.pgc"
+#line 42 "thread.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "create table test_thread ( tstamp timestamp not null default cast ( timeofday ( ) as timestamp ) , thread text not null , iteration integer not null , primary key ( thread , iteration ) )", ECPGt_EOIT, ECPGt_EORT);}
-#line 55 "thread.pgc"
+#line 47 "thread.pgc"
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 56 "thread.pgc"
+#line 48 "thread.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");}
-#line 57 "thread.pgc"
+#line 49 "thread.pgc"
/* create, and start, threads */
@@ -116,18 +108,18 @@ int main()
/* and check results */
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0); }
-#line 87 "thread.pgc"
+#line 79 "thread.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "select count ( * ) from test_thread", ECPGt_EOIT,
ECPGt_int,&(l_rows),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);}
-#line 88 "thread.pgc"
+#line 80 "thread.pgc"
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 89 "thread.pgc"
+#line 81 "thread.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");}
-#line 90 "thread.pgc"
+#line 82 "thread.pgc"
if( l_rows == (nthreads * iterations) )
printf("Success.\n");
@@ -145,13 +137,13 @@ void *test_thread(void *arg)
-#line 104 "thread.pgc"
+#line 96 "thread.pgc"
int l_i ;
-#line 105 "thread.pgc"
+#line 97 "thread.pgc"
char l_connection [ 128 ] ;
/* exec sql end declare section */
-#line 106 "thread.pgc"
+#line 98 "thread.pgc"
/* build up connection name, and connect to database */
@@ -161,13 +153,13 @@ void *test_thread(void *arg)
_snprintf(l_connection, sizeof(l_connection), "thread_%03ld", threadnum);
#endif
/* exec sql whenever sqlerror sqlprint ; */
-#line 114 "thread.pgc"
+#line 106 "thread.pgc"
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , l_connection, 0);
-#line 115 "thread.pgc"
+#line 107 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 115 "thread.pgc"
+#line 107 "thread.pgc"
if( sqlca.sqlcode != 0 )
{
@@ -175,10 +167,10 @@ if (sqlca.sqlcode < 0) sqlprint();}
return NULL;
}
{ ECPGtrans(__LINE__, l_connection, "begin");
-#line 121 "thread.pgc"
+#line 113 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 121 "thread.pgc"
+#line 113 "thread.pgc"
/* insert into test_thread table */
@@ -189,10 +181,10 @@ if (sqlca.sqlcode < 0) sqlprint();}
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_int,&(l_i),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EOIT, ECPGt_EORT);
-#line 126 "thread.pgc"
+#line 118 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 126 "thread.pgc"
+#line 118 "thread.pgc"
if( sqlca.sqlcode != 0 )
printf("%s: ERROR: insert failed!\n", l_connection);
@@ -200,17 +192,16 @@ if (sqlca.sqlcode < 0) sqlprint();}
/* all done */
{ ECPGtrans(__LINE__, l_connection, "commit");
-#line 132 "thread.pgc"
+#line 124 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 132 "thread.pgc"
+#line 124 "thread.pgc"
{ ECPGdisconnect(__LINE__, l_connection);
-#line 133 "thread.pgc"
+#line 125 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 133 "thread.pgc"
+#line 125 "thread.pgc"
return NULL;
}
-#endif /* ENABLE_THREAD_SAFETY */
diff --git a/src/interfaces/ecpg/test/expected/thread-thread_implicit.c b/src/interfaces/ecpg/test/expected/thread-thread_implicit.c
index 0df2794530..7ac0297a23 100644
--- a/src/interfaces/ecpg/test/expected/thread-thread_implicit.c
+++ b/src/interfaces/ecpg/test/expected/thread-thread_implicit.c
@@ -15,14 +15,6 @@
#include <stdlib.h>
#include "ecpg_config.h"
-#ifndef ENABLE_THREAD_SAFETY
-int
-main(void)
-{
- printf("No threading enabled.\n");
- return 0;
-}
-#else
#ifndef WIN32
#include <pthread.h>
#else
@@ -38,7 +30,7 @@ main(void)
-#line 24 "thread_implicit.pgc"
+#line 16 "thread_implicit.pgc"
void *test_thread(void *arg);
@@ -57,10 +49,10 @@ int main()
/* exec sql begin declare section */
-#line 40 "thread_implicit.pgc"
+#line 32 "thread_implicit.pgc"
int l_rows ;
/* exec sql end declare section */
-#line 41 "thread_implicit.pgc"
+#line 33 "thread_implicit.pgc"
/* Do not switch on debug output for regression tests. The threads get executed in
@@ -69,22 +61,22 @@ int main()
/* setup test_thread table */
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0); }
-#line 48 "thread_implicit.pgc"
+#line 40 "thread_implicit.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "drop table test_thread", ECPGt_EOIT, ECPGt_EORT);}
-#line 49 "thread_implicit.pgc"
+#line 41 "thread_implicit.pgc"
/* DROP might fail */
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 50 "thread_implicit.pgc"
+#line 42 "thread_implicit.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "create table test_thread ( tstamp timestamp not null default cast ( timeofday ( ) as timestamp ) , thread text not null , iteration integer not null , primary key ( thread , iteration ) )", ECPGt_EOIT, ECPGt_EORT);}
-#line 55 "thread_implicit.pgc"
+#line 47 "thread_implicit.pgc"
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 56 "thread_implicit.pgc"
+#line 48 "thread_implicit.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");}
-#line 57 "thread_implicit.pgc"
+#line 49 "thread_implicit.pgc"
/* create, and start, threads */
@@ -116,18 +108,18 @@ int main()
/* and check results */
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0); }
-#line 87 "thread_implicit.pgc"
+#line 79 "thread_implicit.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "select count ( * ) from test_thread", ECPGt_EOIT,
ECPGt_int,&(l_rows),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);}
-#line 88 "thread_implicit.pgc"
+#line 80 "thread_implicit.pgc"
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 89 "thread_implicit.pgc"
+#line 81 "thread_implicit.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");}
-#line 90 "thread_implicit.pgc"
+#line 82 "thread_implicit.pgc"
if( l_rows == (nthreads * iterations) )
printf("Success.\n");
@@ -145,13 +137,13 @@ void *test_thread(void *arg)
-#line 104 "thread_implicit.pgc"
+#line 96 "thread_implicit.pgc"
int l_i ;
-#line 105 "thread_implicit.pgc"
+#line 97 "thread_implicit.pgc"
char l_connection [ 128 ] ;
/* exec sql end declare section */
-#line 106 "thread_implicit.pgc"
+#line 98 "thread_implicit.pgc"
/* build up connection name, and connect to database */
@@ -161,13 +153,13 @@ void *test_thread(void *arg)
_snprintf(l_connection, sizeof(l_connection), "thread_%03ld", threadnum);
#endif
/* exec sql whenever sqlerror sqlprint ; */
-#line 114 "thread_implicit.pgc"
+#line 106 "thread_implicit.pgc"
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , l_connection, 0);
-#line 115 "thread_implicit.pgc"
+#line 107 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 115 "thread_implicit.pgc"
+#line 107 "thread_implicit.pgc"
if( sqlca.sqlcode != 0 )
{
@@ -175,10 +167,10 @@ if (sqlca.sqlcode < 0) sqlprint();}
return NULL;
}
{ ECPGtrans(__LINE__, NULL, "begin");
-#line 121 "thread_implicit.pgc"
+#line 113 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 121 "thread_implicit.pgc"
+#line 113 "thread_implicit.pgc"
/* insert into test_thread table */
@@ -189,10 +181,10 @@ if (sqlca.sqlcode < 0) sqlprint();}
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_int,&(l_i),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EOIT, ECPGt_EORT);
-#line 126 "thread_implicit.pgc"
+#line 118 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 126 "thread_implicit.pgc"
+#line 118 "thread_implicit.pgc"
if( sqlca.sqlcode != 0 )
printf("%s: ERROR: insert failed!\n", l_connection);
@@ -200,17 +192,16 @@ if (sqlca.sqlcode < 0) sqlprint();}
/* all done */
{ ECPGtrans(__LINE__, NULL, "commit");
-#line 132 "thread_implicit.pgc"
+#line 124 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 132 "thread_implicit.pgc"
+#line 124 "thread_implicit.pgc"
{ ECPGdisconnect(__LINE__, l_connection);
-#line 133 "thread_implicit.pgc"
+#line 125 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 133 "thread_implicit.pgc"
+#line 125 "thread_implicit.pgc"
return NULL;
}
-#endif /* ENABLE_THREAD_SAFETY */
diff --git a/src/interfaces/ecpg/test/thread/alloc.pgc b/src/interfaces/ecpg/test/thread/alloc.pgc
index c0021a737e..d3d35493bf 100644
--- a/src/interfaces/ecpg/test/thread/alloc.pgc
+++ b/src/interfaces/ecpg/test/thread/alloc.pgc
@@ -2,14 +2,6 @@
#include <stdlib.h>
#include "ecpg_config.h"
-#ifndef ENABLE_THREAD_SAFETY
-int
-main(void)
-{
- printf("No threading enabled.\n");
- return 0;
-}
-#else
#ifdef WIN32
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
@@ -87,4 +79,3 @@ int main ()
return 0;
}
-#endif
diff --git a/src/interfaces/ecpg/test/thread/descriptor.pgc b/src/interfaces/ecpg/test/thread/descriptor.pgc
index 76a7a5dff5..30bce7c87b 100644
--- a/src/interfaces/ecpg/test/thread/descriptor.pgc
+++ b/src/interfaces/ecpg/test/thread/descriptor.pgc
@@ -1,4 +1,3 @@
-#ifdef ENABLE_THREAD_SAFETY
#ifdef WIN32
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
@@ -7,7 +6,6 @@
#else
#include <pthread.h>
#endif
-#endif
#include <stdio.h>
#define THREADS 16
@@ -17,7 +15,7 @@ EXEC SQL include sqlca;
EXEC SQL whenever sqlerror sqlprint;
EXEC SQL whenever not found sqlprint;
-#if defined(ENABLE_THREAD_SAFETY) && defined(WIN32)
+#if defined(WIN32)
static unsigned __stdcall fn(void* arg)
#else
static void* fn(void* arg)
@@ -36,7 +34,6 @@ static void* fn(void* arg)
int main ()
{
-#ifdef ENABLE_THREAD_SAFETY
int i;
#ifdef WIN32
HANDLE threads[THREADS];
@@ -60,9 +57,6 @@ int main ()
for (i = 0; i < THREADS; ++i)
pthread_join(threads[i], NULL);
#endif
-#else
- fn(NULL);
-#endif
return 0;
}
diff --git a/src/interfaces/ecpg/test/thread/prep.pgc b/src/interfaces/ecpg/test/thread/prep.pgc
index d7ecfd4855..f61b31ce10 100644
--- a/src/interfaces/ecpg/test/thread/prep.pgc
+++ b/src/interfaces/ecpg/test/thread/prep.pgc
@@ -2,14 +2,6 @@
#include <stdlib.h>
#include "ecpg_config.h"
-#ifndef ENABLE_THREAD_SAFETY
-int
-main(void)
-{
- printf("No threading enabled.\n");
- return 0;
-}
-#else
#ifdef WIN32
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
@@ -93,4 +85,3 @@ int main ()
return 0;
}
-#endif
diff --git a/src/interfaces/ecpg/test/thread/thread.pgc b/src/interfaces/ecpg/test/thread/thread.pgc
index e7d8c00af6..b9b9ebb441 100644
--- a/src/interfaces/ecpg/test/thread/thread.pgc
+++ b/src/interfaces/ecpg/test/thread/thread.pgc
@@ -6,14 +6,6 @@
#include <stdlib.h>
#include "ecpg_config.h"
-#ifndef ENABLE_THREAD_SAFETY
-int
-main(void)
-{
- printf("No threading enabled.\n");
- return 0;
-}
-#else
#ifndef WIN32
#include <pthread.h>
#else
@@ -133,4 +125,3 @@ void *test_thread(void *arg)
EXEC SQL DISCONNECT :l_connection;
return NULL;
}
-#endif /* ENABLE_THREAD_SAFETY */
diff --git a/src/interfaces/ecpg/test/thread/thread_implicit.pgc b/src/interfaces/ecpg/test/thread/thread_implicit.pgc
index b4cae7e1ae..ff9b12a943 100644
--- a/src/interfaces/ecpg/test/thread/thread_implicit.pgc
+++ b/src/interfaces/ecpg/test/thread/thread_implicit.pgc
@@ -6,14 +6,6 @@
#include <stdlib.h>
#include "ecpg_config.h"
-#ifndef ENABLE_THREAD_SAFETY
-int
-main(void)
-{
- printf("No threading enabled.\n");
- return 0;
-}
-#else
#ifndef WIN32
#include <pthread.h>
#else
@@ -133,4 +125,3 @@ void *test_thread(void *arg)
EXEC SQL DISCONNECT :l_connection;
return NULL;
}
-#endif /* ENABLE_THREAD_SAFETY */
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 0919d8f32f..46653682b0 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -69,11 +69,8 @@ endif
ifeq ($(PORTNAME), win32)
OBJS += \
+ pthread-win32.o \
win32.o
-
-ifeq ($(enable_thread_safety), yes)
-OBJS += pthread-win32.o
-endif
endif
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index a8584d2c68..837c5321aa 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -52,13 +52,11 @@
#include <netinet/tcp.h>
#endif
-#ifdef ENABLE_THREAD_SAFETY
#ifdef WIN32
#include "pthread-win32.h"
#else
#include <pthread.h>
#endif
-#endif
#ifdef USE_LDAP
#ifdef WIN32
@@ -7784,7 +7782,6 @@ pqGetHomeDirectory(char *buf, int bufsize)
static void
default_threadlock(int acquire)
{
-#ifdef ENABLE_THREAD_SAFETY
#ifndef WIN32
static pthread_mutex_t singlethread_lock = PTHREAD_MUTEX_INITIALIZER;
#else
@@ -7813,7 +7810,6 @@ default_threadlock(int acquire)
if (pthread_mutex_unlock(&singlethread_lock))
Assert(false);
}
-#endif
}
pgthreadlock_t
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index 01f8efabbe..a868284ff8 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -3906,11 +3906,7 @@ PQisnonblocking(const PGconn *conn)
int
PQisthreadsafe(void)
{
-#ifdef ENABLE_THREAD_SAFETY
return true;
-#else
- return false;
-#endif
}
diff --git a/src/interfaces/libpq/fe-print.c b/src/interfaces/libpq/fe-print.c
index 40620b47e9..8af15032be 100644
--- a/src/interfaces/libpq/fe-print.c
+++ b/src/interfaces/libpq/fe-print.c
@@ -88,14 +88,11 @@ PQprint(FILE *fout, const PGresult *res, const PQprintOpt *po)
bool usePipe = false;
char *pagerenv;
-#if defined(ENABLE_THREAD_SAFETY) && !defined(WIN32)
+#if !defined(WIN32)
sigset_t osigset;
bool sigpipe_masked = false;
bool sigpipe_pending;
#endif
-#if !defined(ENABLE_THREAD_SAFETY) && !defined(WIN32)
- pqsigfunc oldsigpipehandler = NULL;
-#endif
#ifdef TIOCGWINSZ
struct winsize screen_size;
@@ -186,12 +183,8 @@ PQprint(FILE *fout, const PGresult *res, const PQprintOpt *po)
{
usePipe = true;
#ifndef WIN32
-#ifdef ENABLE_THREAD_SAFETY
if (pq_block_sigpipe(&osigset, &sigpipe_pending) == 0)
sigpipe_masked = true;
-#else
- oldsigpipehandler = pqsignal(SIGPIPE, SIG_IGN);
-#endif /* ENABLE_THREAD_SAFETY */
#endif /* WIN32 */
}
else
@@ -324,13 +317,9 @@ exit:
#else
pclose(fout);
-#ifdef ENABLE_THREAD_SAFETY
/* we can't easily verify if EPIPE occurred, so say it did */
if (sigpipe_masked)
pq_reset_sigpipe(&osigset, sigpipe_pending, true);
-#else
- pqsignal(SIGPIPE, oldsigpipehandler);
-#endif /* ENABLE_THREAD_SAFETY */
#endif /* WIN32 */
}
}
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index bea71660ab..f1192d28f2 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -44,13 +44,11 @@
#include <sys/stat.h>
-#ifdef ENABLE_THREAD_SAFETY
#ifdef WIN32
#include "pthread-win32.h"
#else
#include <pthread.h>
#endif
-#endif
/*
* These SSL-related #includes must come after all system-provided headers.
@@ -91,7 +89,6 @@ static bool pq_init_crypto_lib = true;
static bool ssl_lib_initialized = false;
-#ifdef ENABLE_THREAD_SAFETY
static long crypto_open_connections = 0;
#ifndef WIN32
@@ -100,7 +97,6 @@ static pthread_mutex_t ssl_config_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_mutex_t ssl_config_mutex = NULL;
static long win32_ssl_create_mutex = 0;
#endif
-#endif /* ENABLE_THREAD_SAFETY */
static PQsslKeyPassHook_OpenSSL_type PQsslKeyPassHook = NULL;
static int ssl_protocol_version_to_openssl(const char *protocol);
@@ -112,15 +108,12 @@ static int ssl_protocol_version_to_openssl(const char *protocol);
void
pgtls_init_library(bool do_ssl, int do_crypto)
{
-#ifdef ENABLE_THREAD_SAFETY
-
/*
* Disallow changing the flags while we have open connections, else we'd
* get completely confused.
*/
if (crypto_open_connections != 0)
return;
-#endif
pq_init_ssl_lib = do_ssl;
pq_init_crypto_lib = do_crypto;
@@ -718,7 +711,7 @@ pgtls_verify_peer_name_matches_certificate_guts(PGconn *conn,
return rc;
}
-#if defined(ENABLE_THREAD_SAFETY) && defined(HAVE_CRYPTO_LOCK)
+#if defined(HAVE_CRYPTO_LOCK)
/*
* Callback functions for OpenSSL internal locking. (OpenSSL 1.1.0
* does its own locking, and doesn't need these anymore. The
@@ -759,7 +752,7 @@ pq_lockingcallback(int mode, int n, const char *file, int line)
Assert(false);
}
}
-#endif /* ENABLE_THREAD_SAFETY && HAVE_CRYPTO_LOCK */
+#endif /* HAVE_CRYPTO_LOCK */
/*
* Initialize SSL library.
@@ -774,7 +767,6 @@ pq_lockingcallback(int mode, int n, const char *file, int line)
int
pgtls_init(PGconn *conn, bool do_ssl, bool do_crypto)
{
-#ifdef ENABLE_THREAD_SAFETY
#ifdef WIN32
/* Also see similar code in fe-connect.c, default_threadlock() */
if (ssl_config_mutex == NULL)
@@ -840,7 +832,6 @@ pgtls_init(PGconn *conn, bool do_ssl, bool do_crypto)
}
}
#endif /* HAVE_CRYPTO_LOCK */
-#endif /* ENABLE_THREAD_SAFETY */
if (!ssl_lib_initialized && do_ssl)
{
@@ -857,9 +848,7 @@ pgtls_init(PGconn *conn, bool do_ssl, bool do_crypto)
ssl_lib_initialized = true;
}
-#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_unlock(&ssl_config_mutex);
-#endif
return 0;
}
@@ -878,7 +867,7 @@ pgtls_init(PGconn *conn, bool do_ssl, bool do_crypto)
static void
destroy_ssl_system(void)
{
-#if defined(ENABLE_THREAD_SAFETY) && defined(HAVE_CRYPTO_LOCK)
+#if defined(HAVE_CRYPTO_LOCK)
/* Mutex is created in pgtls_init() */
if (pthread_mutex_lock(&ssl_config_mutex))
return;
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index 8069e38142..bd72a87bbb 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -35,13 +35,11 @@
#include <sys/stat.h>
-#ifdef ENABLE_THREAD_SAFETY
#ifdef WIN32
#include "pthread-win32.h"
#else
#include <pthread.h>
#endif
-#endif
#include "fe-auth.h"
#include "libpq-fe.h"
@@ -56,8 +54,6 @@
#define SIGPIPE_MASKED(conn) ((conn)->sigpipe_so || (conn)->sigpipe_flag)
-#ifdef ENABLE_THREAD_SAFETY
-
struct sigpipe_info
{
sigset_t oldsigmask;
@@ -90,24 +86,6 @@ struct sigpipe_info
pq_reset_sigpipe(&(spinfo).oldsigmask, (spinfo).sigpipe_pending, \
(spinfo).got_epipe); \
} while (0)
-#else /* !ENABLE_THREAD_SAFETY */
-
-#define DECLARE_SIGPIPE_INFO(spinfo) pqsigfunc spinfo = NULL
-
-#define DISABLE_SIGPIPE(conn, spinfo, failaction) \
- do { \
- if (!SIGPIPE_MASKED(conn)) \
- spinfo = pqsignal(SIGPIPE, SIG_IGN); \
- } while (0)
-
-#define REMEMBER_EPIPE(spinfo, cond)
-
-#define RESTORE_SIGPIPE(conn, spinfo) \
- do { \
- if (!SIGPIPE_MASKED(conn)) \
- pqsignal(SIGPIPE, spinfo); \
- } while (0)
-#endif /* ENABLE_THREAD_SAFETY */
#else /* WIN32 */
#define DECLARE_SIGPIPE_INFO(spinfo)
@@ -524,7 +502,7 @@ PQgssEncInUse(PGconn *conn)
#endif /* ENABLE_GSS */
-#if defined(ENABLE_THREAD_SAFETY) && !defined(WIN32)
+#if !defined(WIN32)
/*
* Block SIGPIPE for this thread. This prevents send()/write() from exiting
@@ -608,4 +586,4 @@ pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending, bool got_epipe)
SOCK_ERRNO_SET(save_errno);
}
-#endif /* ENABLE_THREAD_SAFETY && !WIN32 */
+#endif /* !WIN32 */
diff --git a/src/interfaces/libpq/legacy-pqsignal.c b/src/interfaces/libpq/legacy-pqsignal.c
index 790ab5a18c..97baa86a27 100644
--- a/src/interfaces/libpq/legacy-pqsignal.c
+++ b/src/interfaces/libpq/legacy-pqsignal.c
@@ -28,9 +28,7 @@
* with the semantics it had in 9.2; in particular, this has different
* behavior for SIGALRM than the version in src/port/pqsignal.c.
*
- * libpq itself uses this only for SIGPIPE (and even then, only in
- * non-ENABLE_THREAD_SAFETY builds), so the incompatibility isn't
- * troublesome for internal references.
+ * libpq itself does not use this.
*/
pqsigfunc
pqsignal(int signo, pqsigfunc func)
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 9d63472c35..c745facfec 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -31,14 +31,12 @@
#include <sys/time.h>
#endif
-#ifdef ENABLE_THREAD_SAFETY
#ifdef WIN32
#include "pthread-win32.h"
#else
#include <pthread.h>
#endif
#include <signal.h>
-#endif
/* include stuff common to fe and be */
#include "libpq/pqcomm.h"
@@ -681,15 +679,10 @@ extern int pqPacketSend(PGconn *conn, char pack_type,
const void *buf, size_t buf_len);
extern bool pqGetHomeDirectory(char *buf, int bufsize);
-#ifdef ENABLE_THREAD_SAFETY
extern pgthreadlock_t pg_g_threadlock;
#define pglock_thread() pg_g_threadlock(true)
#define pgunlock_thread() pg_g_threadlock(false)
-#else
-#define pglock_thread() ((void) 0)
-#define pgunlock_thread() ((void) 0)
-#endif
/* === in fe-exec.c === */
@@ -765,7 +758,7 @@ extern ssize_t pqsecure_write(PGconn *, const void *ptr, size_t len);
extern ssize_t pqsecure_raw_read(PGconn *, void *ptr, size_t len);
extern ssize_t pqsecure_raw_write(PGconn *, const void *ptr, size_t len);
-#if defined(ENABLE_THREAD_SAFETY) && !defined(WIN32)
+#if !defined(WIN32)
extern int pq_block_sigpipe(sigset_t *osigset, bool *sigpipe_pending);
extern void pq_reset_sigpipe(sigset_t *osigset, bool sigpipe_pending,
bool got_epipe);
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 13045cbd6e..be946f7b38 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -52,7 +52,6 @@ pgxs_kv = {
'abs_top_builddir': meson.build_root(),
'abs_top_srcdir': meson.source_root(),
- 'enable_thread_safety': 'yes',
'enable_rpath': get_option('rpath') ? 'yes' : 'no',
'enable_nls': libintl.found() ? 'yes' : 'no',
'enable_tap_tests': tap_tests_enabled ? 'yes' : 'no',
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index f09b3826ff..1cbc857e35 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -219,7 +219,6 @@ sub GenerateFiles
DLSUFFIX => '".dll"',
ENABLE_GSS => $self->{options}->{gss} ? 1 : undef,
ENABLE_NLS => $self->{options}->{nls} ? 1 : undef,
- ENABLE_THREAD_SAFETY => 1,
HAVE_APPEND_HISTORY => undef,
HAVE_ASN1_STRING_GET0_DATA => undef,
HAVE_ATOMICS => 1,
@@ -1209,7 +1208,7 @@ sub GetFakeConfigure
{
my $self = shift;
- my $cfg = '--enable-thread-safety';
+ my $cfg = '';
$cfg .= ' --enable-cassert' if ($self->{options}->{asserts});
$cfg .= ' --enable-nls' if ($self->{options}->{nls});
$cfg .= ' --enable-tap-tests' if ($self->{options}->{tap_tests});
diff --git a/src/tools/msvc/ecpg_regression.proj b/src/tools/msvc/ecpg_regression.proj
index ec2760b1f6..0ec60a275e 100644
--- a/src/tools/msvc/ecpg_regression.proj
+++ b/src/tools/msvc/ecpg_regression.proj
@@ -54,7 +54,7 @@
<!-- Run ECPG and the Visual C++ compiler on the files. Don't bother with dependency check between the steps -->
<Exec WorkingDirectory="%(Pgc.RelativeDir)" Command="$(OUTDIR)ecpg\ecpg -I ../../include --regression $(ECPGPARAM) -o %(Pgc.Filename).c %(Pgc.Filename).pgc" />
- <Exec WorkingDirectory="%(Pgc.RelativeDir)" Command="cl /nologo %(Pgc.FileName).c /TC /MD$(DEBUGLIB) /DENABLE_THREAD_SAFETY /DWIN32 /I. /I..\..\include /I..\..\..\libpq /I..\..\..\..\include /link /defaultlib:$(OUTDIR)libecpg\libecpg.lib /defaultlib:$(OUTDIR)libecpg_compat\libecpg_compat.lib /defaultlib:$(OUTDIR)libpgtypes\libpgtypes.lib" />
+ <Exec WorkingDirectory="%(Pgc.RelativeDir)" Command="cl /nologo %(Pgc.FileName).c /TC /MD$(DEBUGLIB) /DWIN32 /I. /I..\..\include /I..\..\..\libpq /I..\..\..\..\include /link /defaultlib:$(OUTDIR)libecpg\libecpg.lib /defaultlib:$(OUTDIR)libecpg_compat\libecpg_compat.lib /defaultlib:$(OUTDIR)libpgtypes\libpgtypes.lib" />
</Target>
<!-- Clean up all output files -->
--
2.41.0
[text/x-patch] v2-0002-Doc-Adjust-libpq-docs-about-thread-safety.patch (3.7K, ../../CA+hUKGK+UC4kZmT62noORCJ5gTc+ephVdss+cj7LzhsBZjA-NQ@mail.gmail.com/3-v2-0002-Doc-Adjust-libpq-docs-about-thread-safety.patch)
download | inline diff:
From 08b1ec7f141952cd416ada30369a51ec5c8a8338 Mon Sep 17 00:00:00 2001
From: Thomas Munro <[email protected]>
Date: Mon, 10 Jul 2023 09:02:57 +1200
Subject: [PATCH v2 2/2] Doc: Adjust libpq docs about thread safety.
Describe the situation now that --enable-thread-safety is gone.
Author: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
---
doc/src/sgml/libpq.sgml | 49 +++++++++++++++++++----------------------
1 file changed, 23 insertions(+), 26 deletions(-)
diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml
index b6f5aba1b1..a52baa27d5 100644
--- a/doc/src/sgml/libpq.sgml
+++ b/doc/src/sgml/libpq.sgml
@@ -9236,14 +9236,28 @@ void PQinitSSL(int do_ssl);
</indexterm>
<para>
- <application>libpq</application> is reentrant and thread-safe by default.
- You might need to use special compiler command-line
- options when you compile your application code. Refer to your
- system's documentation for information about how to build
- thread-enabled applications, or look in
- <filename>src/Makefile.global</filename> for <literal>PTHREAD_CFLAGS</literal>
- and <literal>PTHREAD_LIBS</literal>. This function allows the querying of
- <application>libpq</application>'s thread-safe status:
+ As of version 17, <application>libpq</application> is always reentrant and thread-safe.
+ However, one restriction is that no two threads attempt to manipulate
+ the same <structname>PGconn</structname> object at the same time. In particular,
+ you cannot issue concurrent commands from different threads through
+ the same connection object. (If you need to run concurrent commands,
+ use multiple connections.)
+ </para>
+
+ <para>
+ <structname>PGresult</structname> objects are normally read-only after creation,
+ and so can be passed around freely between threads. However, if you use
+ any of the <structname>PGresult</structname>-modifying functions described in
+ <xref linkend="libpq-misc"/> or <xref linkend="libpq-events"/>, it's up
+ to you to avoid concurrent operations on the same <structname>PGresult</structname>,
+ too.
+ </para>
+
+ <para>
+ In earlier versions, <application>libpq</application> could be compiled
+ with or without thread support, depending on compiler options. This
+ function allows the querying of <application>libpq</application>'s
+ thread-safe status:
</para>
<variablelist>
@@ -9261,29 +9275,12 @@ int PQisthreadsafe();
<para>
Returns 1 if the <application>libpq</application> is thread-safe
- and 0 if it is not.
+ and 0 if it is not. Always returns 1 on version 17 and above.
</para>
</listitem>
</varlistentry>
</variablelist>
- <para>
- One thread restriction is that no two threads attempt to manipulate
- the same <structname>PGconn</structname> object at the same time. In particular,
- you cannot issue concurrent commands from different threads through
- the same connection object. (If you need to run concurrent commands,
- use multiple connections.)
- </para>
-
- <para>
- <structname>PGresult</structname> objects are normally read-only after creation,
- and so can be passed around freely between threads. However, if you use
- any of the <structname>PGresult</structname>-modifying functions described in
- <xref linkend="libpq-misc"/> or <xref linkend="libpq-events"/>, it's up
- to you to avoid concurrent operations on the same <structname>PGresult</structname>,
- too.
- </para>
-
<para>
The deprecated functions <xref linkend="libpq-PQrequestCancel"/> and
<xref linkend="libpq-PQoidStatus"/> are not thread-safe and should not be
--
2.41.0
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-07-11 20:58 Thomas Munro <[email protected]>
parent: Thomas Munro <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Thomas Munro @ 2023-07-11 20:58 UTC (permalink / raw)
To: Heikki Linnakangas <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
On Mon, Jul 10, 2023 at 10:45 AM Thomas Munro <[email protected]> wrote:
> * defined ENABLE_THREAD_SAFETY 1 in c.h, for the benefit of extensions
I may lack imagination but I couldn't think of any use for that
vestigial macro in backend/extension code, and client code doesn't see
c.h and might not get the right answer anyway if it's dynamically
linked which is the usual case. I took it out for now. Open to
discussing further if someone can show what kinds of realistic
external code would be affected.
> * defined ENABLE_THREAD_SAFETY 1 ecpg_config.h, for the benefit of ECPG clients
I left this one in. I'm not sure if it could really be needed.
Perhaps at a stretch, perhaps ECPG code that is statically linked
might test that instead of calling PQisthreadsafe().
Pushed.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-07-11 22:49 Andres Freund <[email protected]>
parent: Thomas Munro <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Andres Freund @ 2023-07-11 22:49 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On 2023-07-12 08:58:29 +1200, Thomas Munro wrote:
> On Mon, Jul 10, 2023 at 10:45 AM Thomas Munro <[email protected]> wrote:
> > * defined ENABLE_THREAD_SAFETY 1 in c.h, for the benefit of extensions
>
> I may lack imagination but I couldn't think of any use for that
> vestigial macro in backend/extension code, and client code doesn't see
> c.h and might not get the right answer anyway if it's dynamically
> linked which is the usual case. I took it out for now. Open to
> discussing further if someone can show what kinds of realistic
> external code would be affected.
WFM.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-07-12 03:21 Thomas Munro <[email protected]>
parent: Andres Freund <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Thomas Munro @ 2023-07-12 03:21 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
Apparently "drongo" didn't like something about this commit, and an
ecpg test failed, but I can't immediately see why. Just "server
closed the connection unexpectedly". drongo is running the new Meson
buildfarm variant on Windows+MSVC, so, being still in development, I
wonder if some diagnostic clue/log/backtrace etc is not being uploaded
yet. FWIW Meson+Windows+MSVC passes on CI.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-07-12 03:34 Thomas Munro <[email protected]>
parent: Thomas Munro <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Thomas Munro @ 2023-07-12 03:34 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On Wed, Jul 12, 2023 at 3:21 PM Thomas Munro <[email protected]> wrote:
> Apparently "drongo" didn't like something about this commit, and an
> ecpg test failed, but I can't immediately see why. Just "server
> closed the connection unexpectedly". drongo is running the new Meson
> buildfarm variant on Windows+MSVC, so, being still in development, I
> wonder if some diagnostic clue/log/backtrace etc is not being uploaded
> yet. FWIW Meson+Windows+MSVC passes on CI.
Oh, that's probably unrelated to this commit. It's failed 6 times
like that in the past ~3 months.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2023-07-12 05:27 Thomas Munro <[email protected]>
parent: Thomas Munro <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Thomas Munro @ 2023-07-12 05:27 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Heikki Linnakangas <[email protected]>; Peter Eisentraut <[email protected]>; pgsql-hackers
On Wed, Jul 12, 2023 at 3:34 PM Thomas Munro <[email protected]> wrote:
> On Wed, Jul 12, 2023 at 3:21 PM Thomas Munro <[email protected]> wrote:
> > Apparently "drongo" didn't like something about this commit, and an
> > ecpg test failed, but I can't immediately see why. Just "server
> > closed the connection unexpectedly". drongo is running the new Meson
> > buildfarm variant on Windows+MSVC, so, being still in development, I
> > wonder if some diagnostic clue/log/backtrace etc is not being uploaded
> > yet. FWIW Meson+Windows+MSVC passes on CI.
>
> Oh, that's probably unrelated to this commit. It's failed 6 times
> like that in the past ~3 months.
Ah, right, that is a symptom of the old Windows TCP linger vs process
exit thing. Something on my list to try to investigate again, but not
today.
https://www.postgresql.org/message-id/flat/16678-253e48d34dc0c376%40postgresql.org
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: MERGE ... WHEN NOT MATCHED BY SOURCE
@ 2024-01-22 02:10 Peter Smith <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Peter Smith @ 2024-01-22 02:10 UTC (permalink / raw)
To: Dean Rasheed <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]>
Hi, this patch was marked in CF as "Needs Review" [1], but there has
been no activity on this thread for 6+ months.
Is anything else planned? Can you post something to elicit more
interest in the latest patch? Otherwise, if nothing happens then the
CF entry will be closed ("Returned with feedback") at the end of this
CF.
======
[1] https://commitfest.postgresql.org/46/4092/
Kind Regards,
Peter Smith.
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: MERGE ... WHEN NOT MATCHED BY SOURCE
@ 2024-01-26 14:59 vignesh C <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: vignesh C @ 2024-01-26 14:59 UTC (permalink / raw)
To: Dean Rasheed <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]>
On Sat, 1 Jul 2023 at 18:04, Dean Rasheed <[email protected]> wrote:
>
> On Tue, 21 Mar 2023 at 12:26, Alvaro Herrera <[email protected]> wrote:
> >
> > On 2023-Mar-21, Dean Rasheed wrote:
> >
> > > Looking at it with fresh eyes though, I realise that I could have just written
> > >
> > > action->qual = make_and_qual((Node *) ntest, action->qual);
> > >
> > > which is equivalent, but more concise.
> >
> > Nice.
> >
> > I have no further observations about this patch.
> >
>
> Looking at this one afresh, it seems that the change to make Vars
> outer-join aware broke it -- the Var in the qual to test whether the
> source row is null needs to be marked as nullable by the join added by
> transform_MERGE_to_join(). That's something that needs to be done in
> transform_MERGE_to_join(), so it makes more sense to add the new qual
> there rather than in transformMergeStmt().
>
> Also, now that MERGE has ruleutils support, it's clear that adding the
> qual in transformMergeStmt() isn't right anyway, since it would then
> appear in the deparsed output.
>
> So attached is an updated patch doing that, which seems neater all
> round, since adding the qual is closely related to the join-type
> choice, which is now a decision taken entirely in
> transform_MERGE_to_join(). This requires a new "mergeSourceRelation"
> field on the Query structure, but as before, it does away with the
> "mergeUseOuterJoin" field.
>
> I've also updated the ruleutils support. In the absence of any WHEN
> NOT MATCHED BY SOURCE actions, this will output not-matched actions
> simply as "WHEN NOT MATCHED" for backwards compatibility, and to be
> SQL-standard-compliant. If there are any WHEN NOT MATCHED BY SOURCE
> actions though, I think it's preferable to output explicit "BY SOURCE"
> and "BY TARGET" qualifiers for all not-matched actions, to make the
> meaning clearer.
CFBot shows that the patch does not apply anymore as in [1]:
=== Applying patches on top of PostgreSQL commit ID
f2bf8fb04886e3ea82e7f7f86696ac78e06b7e60 ===
=== applying patch ./support-merge-when-not-matched-by-source-v8.patch
...
patching file doc/src/sgml/ref/merge.sgml
Hunk #5 FAILED at 409.
Hunk #9 FAILED at 673.
2 out of 9 hunks FAILED -- saving rejects to file
doc/src/sgml/ref/merge.sgml.rej
..
patching file src/include/nodes/parsenodes.h
Hunk #1 succeeded at 175 (offset -8 lines).
Hunk #2 succeeded at 1657 (offset -6 lines).
Hunk #3 succeeded at 1674 (offset -6 lines).
Hunk #4 FAILED at 1696.
1 out of 4 hunks FAILED -- saving rejects to file
src/include/nodes/parsenodes.h.rej
Please post an updated version for the same.
[1] - http://cfbot.cputube.org/patch_46_4092.log
Regards,
Vignesh
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: MERGE ... WHEN NOT MATCHED BY SOURCE
@ 2024-01-26 15:48 Dean Rasheed <[email protected]>
parent: Peter Smith <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Dean Rasheed @ 2024-01-26 15:48 UTC (permalink / raw)
To: Peter Smith <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]>
On Mon, 22 Jan 2024 at 02:10, Peter Smith <[email protected]> wrote:
>
> Hi, this patch was marked in CF as "Needs Review" [1], but there has
> been no activity on this thread for 6+ months.
>
> Is anything else planned? Can you post something to elicit more
> interest in the latest patch? Otherwise, if nothing happens then the
> CF entry will be closed ("Returned with feedback") at the end of this
> CF.
>
I think it has had a decent amount of review and all the review
comments have been addressed. I'm not quite sure from Alvaro's last
comment whether he was implying that he thought it was ready for
commit.
Looking back through the thread, the general sentiment seems to be in
favour of adding this feature, and I still think it's worth doing, but
I haven't managed to find much time to progress it recently.
Regards,
Dean
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: MERGE ... WHEN NOT MATCHED BY SOURCE
@ 2024-01-26 15:48 Dean Rasheed <[email protected]>
parent: vignesh C <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Dean Rasheed @ 2024-01-26 15:48 UTC (permalink / raw)
To: vignesh C <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]>
n Fri, 26 Jan 2024 at 14:59, vignesh C <[email protected]> wrote:
>
> CFBot shows that the patch does not apply anymore as in [1]:
>
Rebased version attached.
Regards,
Dean
Attachments:
[text/x-patch] support-merge-when-not-matched-by-source-v9.patch (46.4K, ../../CAEZATCUR2sWck9C4833=NmFXV68PYeYVTZTu_vKUQ_bA=kVRJg@mail.gmail.com/2-support-merge-when-not-matched-by-source-v9.patch)
download | inline diff:
diff --git a/doc/src/sgml/mvcc.sgml b/doc/src/sgml/mvcc.sgml
new file mode 100644
index f8f83d4..6ef0c2b
--- a/doc/src/sgml/mvcc.sgml
+++ b/doc/src/sgml/mvcc.sgml
@@ -396,8 +396,8 @@
originally matched appears later in the list of actions.
On the other hand, if the row is concurrently updated or deleted so
that the join condition fails, then <command>MERGE</command> will
- evaluate the condition's <literal>NOT MATCHED</literal> actions next,
- and execute the first one that succeeds.
+ evaluate the condition's <literal>NOT MATCHED [BY TARGET]</literal>
+ actions next, and execute the first one that succeeds.
If <command>MERGE</command> attempts an <command>INSERT</command>
and a unique index is present and a duplicate row is concurrently
inserted, then a uniqueness violation error is raised;
diff --git a/doc/src/sgml/ref/merge.sgml b/doc/src/sgml/ref/merge.sgml
new file mode 100644
index 655f7dc..f421716
--- a/doc/src/sgml/ref/merge.sgml
+++ b/doc/src/sgml/ref/merge.sgml
@@ -33,7 +33,8 @@ USING <replaceable class="parameter">dat
<phrase>and <replaceable class="parameter">when_clause</replaceable> is:</phrase>
{ WHEN MATCHED [ AND <replaceable class="parameter">condition</replaceable> ] THEN { <replaceable class="parameter">merge_update</replaceable> | <replaceable class="parameter">merge_delete</replaceable> | DO NOTHING } |
- WHEN NOT MATCHED [ AND <replaceable class="parameter">condition</replaceable> ] THEN { <replaceable class="parameter">merge_insert</replaceable> | DO NOTHING } }
+ WHEN NOT MATCHED BY SOURCE [ AND <replaceable class="parameter">condition</replaceable> ] THEN { <replaceable class="parameter">merge_update</replaceable> | <replaceable class="parameter">merge_delete</replaceable> | DO NOTHING } |
+ WHEN NOT MATCHED [BY TARGET] [ AND <replaceable class="parameter">condition</replaceable> ] THEN { <replaceable class="parameter">merge_insert</replaceable> | DO NOTHING } }
<phrase>and <replaceable class="parameter">merge_insert</replaceable> is:</phrase>
@@ -70,7 +71,9 @@ DELETE
from <replaceable class="parameter">data_source</replaceable> to
<replaceable class="parameter">target_table_name</replaceable>
producing zero or more candidate change rows. For each candidate change
- row, the status of <literal>MATCHED</literal> or <literal>NOT MATCHED</literal>
+ row, the status of <literal>MATCHED</literal>,
+ <literal>NOT MATCHED BY SOURCE</literal>,
+ or <literal>NOT MATCHED [BY TARGET]</literal>
is set just once, after which <literal>WHEN</literal> clauses are evaluated
in the order specified. For each candidate change row, the first clause to
evaluate as true is executed. No more than one <literal>WHEN</literal>
@@ -226,16 +229,37 @@ DELETE
At least one <literal>WHEN</literal> clause is required.
</para>
<para>
+ The <literal>WHEN</literal> clause may specify <literal>WHEN MATCHED</literal>,
+ <literal>WHEN NOT MATCHED BY SOURCE</literal>, or
+ <literal>WHEN NOT MATCHED [BY TARGET]</literal>.
+ Note that the <acronym>SQL</acronym> standard only defines
+ <literal>WHEN MATCHED</literal> and <literal>WHEN NOT MATCHED</literal>
+ (which is defined to mean no matching target row).
+ <literal>WHEN NOT MATCHED BY SOURCE</literal> is an extension to the
+ <acronym>SQL</acronym> standard, as is the option to append
+ <literal>BY TARGET</literal> to <literal>WHEN NOT MATCHED</literal>, to
+ make its meaning more explicit.
+ </para>
+ <para>
If the <literal>WHEN</literal> clause specifies <literal>WHEN MATCHED</literal>
- and the candidate change row matches a row in the
+ and the candidate change row matches a row in the source to a row in the
<replaceable class="parameter">target_table_name</replaceable>,
the <literal>WHEN</literal> clause is executed if the
<replaceable class="parameter">condition</replaceable> is
absent or it evaluates to <literal>true</literal>.
</para>
<para>
- Conversely, if the <literal>WHEN</literal> clause specifies
- <literal>WHEN NOT MATCHED</literal>
+ If the <literal>WHEN</literal> clause specifies
+ <literal>WHEN NOT MATCHED BY SOURCE</literal> and the candidate change
+ row represents a row in the
+ <replaceable class="parameter">target_table_name</replaceable> that does
+ not match a source row, the <literal>WHEN</literal> clause is executed
+ if the <replaceable class="parameter">condition</replaceable> is
+ absent or it evaluates to <literal>true</literal>.
+ </para>
+ <para>
+ If the <literal>WHEN</literal> clause specifies
+ <literal>WHEN NOT MATCHED [BY TARGET]</literal>
and the candidate change row does not match a row in the
<replaceable class="parameter">target_table_name</replaceable>,
the <literal>WHEN</literal> clause is executed if the
@@ -257,7 +281,10 @@ DELETE
<para>
A condition on a <literal>WHEN MATCHED</literal> clause can refer to columns
in both the source and the target relations. A condition on a
- <literal>WHEN NOT MATCHED</literal> clause can only refer to columns from
+ <literal>WHEN NOT MATCHED BY SOURCE</literal> clause can only refer to
+ columns from the target relation, since by definition there is no matching
+ source row. A condition on a <literal>WHEN NOT MATCHED [BY TARGET]</literal>
+ clause can only refer to columns from
the source relation, since by definition there is no matching target row.
Only the system attributes from the target table are accessible.
</para>
@@ -382,8 +409,10 @@ DELETE
<literal>WHEN MATCHED</literal> clause, the expression can use values
from the original row in the target table, and values from the
<replaceable>data_source</replaceable> row.
- If used in a <literal>WHEN NOT MATCHED</literal> clause, the
- expression can use values from the <replaceable>data_source</replaceable>.
+ If used in a <literal>WHEN NOT MATCHED BY SOURCE</literal> clause, the
+ expression can only use values from the original row in the target table.
+ If used in a <literal>WHEN NOT MATCHED [BY TARGET]</literal> clause, the
+ expression can only use values from the <replaceable>data_source</replaceable>.
</para>
</listitem>
</varlistentry>
@@ -452,8 +481,9 @@ MERGE <replaceable class="parameter">tot
<orderedlist>
<listitem>
<para>
- Evaluate whether each row is <literal>MATCHED</literal> or
- <literal>NOT MATCHED</literal>.
+ Evaluate whether each row is <literal>MATCHED</literal>,
+ <literal>NOT MATCHED BY SOURCE</literal>, or
+ <literal>NOT MATCHED [BY TARGET]</literal>.
</para>
</listitem>
<listitem>
@@ -528,7 +558,8 @@ MERGE <replaceable class="parameter">tot
<para>
If a <literal>WHEN</literal> clause omits an <literal>AND</literal>
sub-clause, it becomes the final reachable clause of that
- kind (<literal>MATCHED</literal> or <literal>NOT MATCHED</literal>).
+ kind (<literal>MATCHED</literal>, <literal>NOT MATCHED BY SOURCE</literal>,
+ or <literal>NOT MATCHED [BY TARGET]</literal>).
If a later <literal>WHEN</literal> clause of that kind
is specified it would be provably unreachable and an error is raised.
If no final reachable clause is specified of either kind, it is
@@ -619,6 +650,23 @@ WHEN MATCHED THEN
temporary table recently loaded into the database.
</para>
+ <para>
+ Update <literal>wine</literal> based on a replacement wine list, inserting
+ rows for any new stock, updating modified stock entries, and deleting any
+ wines not present in the new list.
+<programlisting>
+MERGE INTO wines w
+USING new_wine_list s
+ON s.winename = w.winename
+WHEN NOT MATCHED BY TARGET THEN
+ INSERT VALUES(s.winename, s.stock)
+WHEN MATCHED AND w.stock != s.stock THEN
+ UPDATE SET stock = s.stock
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE;
+</programlisting>
+ </para>
+
</refsect1>
<refsect1>
@@ -627,7 +675,9 @@ WHEN MATCHED THEN
This command conforms to the <acronym>SQL</acronym> standard.
</para>
<para>
- The <literal>WITH</literal> clause and <literal>DO NOTHING</literal>
+ The <literal>WITH</literal> clause, <literal>BY SOURCE</literal> and
+ <literal>BY TARGET</literal> qualifiers to
+ <literal>WHEN NOT MATCHED</literal>, and <literal>DO NOTHING</literal>
action are extensions to the <acronym>SQL</acronym> standard.
</para>
</refsect1>
diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c
new file mode 100644
index b22040a..b5a18d2
--- a/src/backend/executor/execPartition.c
+++ b/src/backend/executor/execPartition.c
@@ -902,8 +902,16 @@ ExecInitPartitionInfo(ModifyTableState *
action_state = makeNode(MergeActionState);
action_state->mas_action = action;
- /* And put the action in the appropriate list */
- if (action->matched)
+ /*
+ * Put the action in the appropriate list.
+ *
+ * Note that the executor treats WHEN NOT MATCHED BY SOURCE
+ * actions in exactly the same way as WHEN MATCHED actions, since
+ * they both match the target (see ExecMerge). Thus both types go
+ * in the "matched" list. Only WHEN NOT MATCHED BY TARGET actions
+ * go in the "not matched" list.
+ */
+ if (action->matchKind != MERGE_WHEN_NOT_MATCHED_BY_TARGET)
list = &leaf_part_rri->ri_matchedMergeAction;
else
list = &leaf_part_rri->ri_notMatchedMergeAction;
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
new file mode 100644
index 9fc5abf..8cb8933
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -24,13 +24,15 @@
* values plus row-locating info for UPDATE and MERGE cases, or just the
* row-locating info for DELETE cases.
*
- * MERGE runs a join between the source relation and the target
- * table; if any WHEN NOT MATCHED clauses are present, then the
- * join is an outer join. In this case, any unmatched tuples will
- * have NULL row-locating info, and only INSERT can be run. But for
- * matched tuples, then row-locating info is used to determine the
- * tuple to UPDATE or DELETE. When all clauses are WHEN MATCHED,
- * then an inner join is used, so all tuples contain row-locating info.
+ * MERGE runs a join between the source relation and the target table.
+ * If any WHEN NOT MATCHED [BY TARGET] clauses are present, then the join
+ * is an outer join that might output tuples without a matching target
+ * tuple. In this case, any unmatched target tuples will have NULL
+ * row-locating info, and only INSERT can be run. But for matched
+ * target tuples, the row-locating info is used to determine the tuple
+ * to UPDATE or DELETE. When all clauses are WHEN MATCHED or WHEN NOT
+ * MATCHED BY SOURCE, all tuples produced by the join will include a
+ * matching target tuple, so all tuples contain row-locating info.
*
* If the query specifies RETURNING, then the ModifyTable returns a
* RETURNING tuple after completing each row insert, update, or delete.
@@ -2721,6 +2723,17 @@ ExecMerge(ModifyTableContext *context, R
bool matched;
/*-----
+ * Note that as far as the executor is concerned, WHEN NOT MATCHED BY
+ * SOURCE actions are treated exactly the same as WHEN MATCHED actions,
+ * since both match target tuples. They are distinguished from one
+ * another by a qual that tests if the source tuple is NULL, but the
+ * executor knows nothing about the contents of the merge action quals.
+ * Thus WHEN MATCHED and WHEN NOT MATCHED BY SOURCE actions are stored
+ * together in the same "matched" list. Thus, in the dicussion that
+ * follows "MATCHED" means "matched by target", and should be taken to
+ * include both WHEN MATCHED and WHEN NOT MATCHED BY SOURCE, while "NOT
+ * MATCHED" means NOT MATCHED BY TARGET.
+ *
* If we are dealing with a WHEN MATCHED case (tupleid is valid), we
* execute the first action for which the additional WHEN MATCHED AND
* quals pass. If an action without quals is found, that action is
@@ -3065,9 +3078,13 @@ lmerge_matched:
* is executed.
*
* Update tupleid to that of the new tuple, for
- * the refetch we do at the top.
+ * the refetch we do at the top, and install the
+ * updated plan slot in ecxt_innertuple in case
+ * the source tuple is now NULL (a change from
+ * MATCHED to NOT MATCHED BY SOURCE).
*/
ItemPointerCopy(&context->tmfd.ctid, tupleid);
+ econtext->ecxt_innertuple = epqslot;
goto lmerge_matched;
case TM_Deleted:
@@ -3272,8 +3289,14 @@ ExecInitMerge(ModifyTableState *mtstate,
* We create two lists - one for WHEN MATCHED actions and one for
* WHEN NOT MATCHED actions - and stick the MergeActionState into
* the appropriate list.
+ *
+ * Note that the executor treats WHEN NOT MATCHED BY SOURCE
+ * actions in exactly the same way as WHEN MATCHED actions, since
+ * they both match the target (see ExecMerge). Thus both types go
+ * in the "matched" list. Only WHEN NOT MATCHED BY TARGET actions
+ * go in the "not matched" list.
*/
- if (action_state->mas_action->matched)
+ if (action->matchKind != MERGE_WHEN_NOT_MATCHED_BY_TARGET)
list = &resultRelInfo->ri_matchedMergeAction;
else
list = &resultRelInfo->ri_notMatchedMergeAction;
diff --git a/src/backend/optimizer/prep/prepjointree.c b/src/backend/optimizer/prep/prepjointree.c
new file mode 100644
index aa83dd3..38020bd
--- a/src/backend/optimizer/prep/prepjointree.c
+++ b/src/backend/optimizer/prep/prepjointree.c
@@ -153,6 +153,8 @@ transform_MERGE_to_join(Query *parse)
{
RangeTblEntry *joinrte;
JoinExpr *joinexpr;
+ bool src_only_tuples;
+ bool tgt_only_tuples;
JoinType jointype;
int joinrti;
List *vars;
@@ -164,12 +166,30 @@ transform_MERGE_to_join(Query *parse)
vars = NIL;
/*
- * When any WHEN NOT MATCHED THEN INSERT clauses exist, we need to use an
- * outer join so that we process all unmatched tuples from the source
- * relation. If none exist, we can use an inner join.
+ * Work out what kind of join is required. If there any WHEN NOT MATCHED
+ * BY SOURCE/TARGET actions, an outer join is required so that we process
+ * all unmatched tuples from the source and/or target relations.
+ * Otherwise, we can use an inner join.
*/
- if (parse->mergeUseOuterJoin)
+ src_only_tuples = false;
+ tgt_only_tuples = false;
+ foreach_node(MergeAction, action, parse->mergeActionList)
+ {
+ if (action->commandType != CMD_NOTHING)
+ {
+ if (action->matchKind == MERGE_WHEN_NOT_MATCHED_BY_SOURCE)
+ tgt_only_tuples = true;
+ if (action->matchKind == MERGE_WHEN_NOT_MATCHED_BY_TARGET)
+ src_only_tuples = true;
+ }
+ }
+
+ if (src_only_tuples && tgt_only_tuples)
+ jointype = JOIN_FULL;
+ else if (src_only_tuples)
jointype = JOIN_RIGHT;
+ else if (tgt_only_tuples)
+ jointype = JOIN_LEFT;
else
jointype = JOIN_INNER;
@@ -215,6 +235,50 @@ transform_MERGE_to_join(Query *parse)
/* Make the new join be the sole entry in the query's jointree */
parse->jointree->fromlist = list_make1(joinexpr);
parse->jointree->quals = NULL;
+
+ /*
+ * If there any WHEN NOT MATCHED BY SOURCE actions that require unmatched
+ * tuples from the target relation to be processed, add additional WHEN
+ * conditions to every action to check whether tuples from the source
+ * match or not, as necessary.
+ *
+ * This distinguishes WHEN NOT MATCHED BY SOURCE actions (identified by a
+ * "source IS NOT DISTINCT FROM NULL" clause) from WHEN MATCHED actions
+ * (identified by a "source IS DISTINCT FROM NULL" clause).
+ *
+ * Additionally, a "source IS DISTINCT FROM NULL" clause is required for
+ * WHEN NOT MATCHED [BY TARGET] actions in case the executor attempts to
+ * invoke such an action for a concurrently deleted target row that ends
+ * up matching neither source nor target.
+ */
+ if (tgt_only_tuples)
+ {
+ foreach_node(MergeAction, action, parse->mergeActionList)
+ {
+ bool src_null;
+ Var *var;
+ NullTest *ntest;
+
+ /* Make a "source IS [NOT] DISTINCT FROM NULL" clause */
+ src_null = action->matchKind == MERGE_WHEN_NOT_MATCHED_BY_SOURCE;
+
+ var = makeWholeRowVar(rt_fetch(parse->mergeSourceRelation,
+ parse->rtable),
+ parse->mergeSourceRelation, 0, false);
+
+ /* source wholerow Var is nullable by the new join */
+ var->varnullingrels = bms_make_singleton(joinrti);
+
+ ntest = makeNode(NullTest);
+ ntest->arg = (Expr *) var;
+ ntest->nulltesttype = src_null ? IS_NULL : IS_NOT_NULL;
+ ntest->argisrow = false;
+ ntest->location = -1;
+
+ /* Combine it with the action's WHEN condition */
+ action->qual = make_and_qual((Node *) ntest, action->qual);
+ }
+ }
}
/*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
new file mode 100644
index 130f7fc..f6b4c80
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -275,6 +275,7 @@ static Node *makeRecursiveViewSelect(cha
struct SelectLimit *selectlimit;
SetQuantifier setquantifier;
struct GroupClause *groupclause;
+ MergeMatchKind mergematch;
MergeWhenClause *mergewhen;
struct KeyActions *keyactions;
struct KeyAction *keyaction;
@@ -515,6 +516,7 @@ static Node *makeRecursiveViewSelect(cha
%type <onconflict> opt_on_conflict
%type <mergewhen> merge_insert merge_update merge_delete
+%type <mergematch> merge_when_tgt_matched merge_when_tgt_not_matched
%type <node> merge_when_clause opt_merge_when_condition
%type <list> merge_when_list
@@ -759,11 +761,11 @@ static Node *makeRecursiveViewSelect(cha
SAVEPOINT SCALAR SCHEMA SCHEMAS SCROLL SEARCH SECOND_P SECURITY SELECT
SEQUENCE SEQUENCES
SERIALIZABLE SERVER SESSION SESSION_USER SET SETS SETOF SHARE SHOW
- SIMILAR SIMPLE SKIP SMALLINT SNAPSHOT SOME SQL_P STABLE STANDALONE_P
+ SIMILAR SIMPLE SKIP SMALLINT SNAPSHOT SOME SOURCE SQL_P STABLE STANDALONE_P
START STATEMENT STATISTICS STDIN STDOUT STORAGE STORED STRICT_P STRIP_P
SUBSCRIPTION SUBSTRING SUPPORT SYMMETRIC SYSID SYSTEM_P SYSTEM_USER
- TABLE TABLES TABLESAMPLE TABLESPACE TEMP TEMPLATE TEMPORARY TEXT_P THEN
+ TABLE TABLES TABLESAMPLE TABLESPACE TARGET TEMP TEMPLATE TEMPORARY TEXT_P THEN
TIES TIME TIMESTAMP TO TRAILING TRANSACTION TRANSFORM
TREAT TRIGGER TRIM TRUE_P
TRUNCATE TRUSTED TYPE_P TYPES_P
@@ -12386,50 +12388,66 @@ merge_when_list:
| merge_when_list merge_when_clause { $$ = lappend($1,$2); }
;
+/*
+ * A WHEN clause may be WHEN MATCHED, WHEN NOT MATCHED BY SOURCE, or WHEN NOT
+ * MATCHED [BY TARGET]. The first two cases match target tuples, and support
+ * UPDATE/DELETE/DO NOTHING actions. The third case does not match target
+ * tuples, and only supports INSERT/DO NOTHING actions.
+ */
merge_when_clause:
- WHEN MATCHED opt_merge_when_condition THEN merge_update
+ merge_when_tgt_matched opt_merge_when_condition THEN merge_update
{
- $5->matched = true;
- $5->condition = $3;
+ $4->matchKind = $1;
+ $4->condition = $2;
- $$ = (Node *) $5;
+ $$ = (Node *) $4;
}
- | WHEN MATCHED opt_merge_when_condition THEN merge_delete
+ | merge_when_tgt_matched opt_merge_when_condition THEN merge_delete
{
- $5->matched = true;
- $5->condition = $3;
+ $4->matchKind = $1;
+ $4->condition = $2;
- $$ = (Node *) $5;
+ $$ = (Node *) $4;
}
- | WHEN NOT MATCHED opt_merge_when_condition THEN merge_insert
+ | merge_when_tgt_not_matched opt_merge_when_condition THEN merge_insert
{
- $6->matched = false;
- $6->condition = $4;
+ $4->matchKind = $1;
+ $4->condition = $2;
- $$ = (Node *) $6;
+ $$ = (Node *) $4;
}
- | WHEN MATCHED opt_merge_when_condition THEN DO NOTHING
+ | merge_when_tgt_matched opt_merge_when_condition THEN DO NOTHING
{
MergeWhenClause *m = makeNode(MergeWhenClause);
- m->matched = true;
+ m->matchKind = $1;
m->commandType = CMD_NOTHING;
- m->condition = $3;
+ m->condition = $2;
$$ = (Node *) m;
}
- | WHEN NOT MATCHED opt_merge_when_condition THEN DO NOTHING
+ | merge_when_tgt_not_matched opt_merge_when_condition THEN DO NOTHING
{
MergeWhenClause *m = makeNode(MergeWhenClause);
- m->matched = false;
+ m->matchKind = $1;
m->commandType = CMD_NOTHING;
- m->condition = $4;
+ m->condition = $2;
$$ = (Node *) m;
}
;
+merge_when_tgt_matched:
+ WHEN MATCHED { $$ = MERGE_WHEN_MATCHED; }
+ | WHEN NOT MATCHED BY SOURCE { $$ = MERGE_WHEN_NOT_MATCHED_BY_SOURCE; }
+ ;
+
+merge_when_tgt_not_matched:
+ WHEN NOT MATCHED { $$ = MERGE_WHEN_NOT_MATCHED_BY_TARGET; }
+ | WHEN NOT MATCHED BY TARGET { $$ = MERGE_WHEN_NOT_MATCHED_BY_TARGET; }
+ ;
+
opt_merge_when_condition:
AND a_expr { $$ = $2; }
| { $$ = NULL; }
@@ -17388,6 +17406,7 @@ unreserved_keyword:
| SIMPLE
| SKIP
| SNAPSHOT
+ | SOURCE
| SQL_P
| STABLE
| STANDALONE_P
@@ -17406,6 +17425,7 @@ unreserved_keyword:
| SYSTEM_P
| TABLES
| TABLESPACE
+ | TARGET
| TEMP
| TEMPLATE
| TEMPORARY
@@ -18001,6 +18021,7 @@ bare_label_keyword:
| SMALLINT
| SNAPSHOT
| SOME
+ | SOURCE
| SQL_P
| STABLE
| STANDALONE_P
@@ -18024,6 +18045,7 @@ bare_label_keyword:
| TABLES
| TABLESAMPLE
| TABLESPACE
+ | TARGET
| TEMP
| TEMPLATE
| TEMPORARY
diff --git a/src/backend/parser/parse_merge.c b/src/backend/parser/parse_merge.c
new file mode 100644
index 5f6a683..57902f5
--- a/src/backend/parser/parse_merge.c
+++ b/src/backend/parser/parse_merge.c
@@ -43,9 +43,9 @@ static void setNamespaceVisibilityForRTE
* Make appropriate changes to the namespace visibility while transforming
* individual action's quals and targetlist expressions. In particular, for
* INSERT actions we must only see the source relation (since INSERT action is
- * invoked for NOT MATCHED tuples and hence there is no target tuple to deal
- * with). On the other hand, UPDATE and DELETE actions can see both source and
- * target relations.
+ * invoked for NOT MATCHED [BY TARGET] tuples and hence there is no target
+ * tuple to deal with). On the other hand, UPDATE and DELETE actions can see
+ * both source and target relations, unless invoked for NOT MATCHED BY SOURCE.
*
* Also, since the internal join node can hide the source and target
* relations, we must explicitly make the respective relation as visible so
@@ -61,7 +61,7 @@ setNamespaceForMergeWhen(ParseState *pst
targetRelRTE = rt_fetch(targetRTI, pstate->p_rtable);
sourceRelRTE = rt_fetch(sourceRTI, pstate->p_rtable);
- if (mergeWhenClause->matched)
+ if (mergeWhenClause->matchKind == MERGE_WHEN_MATCHED)
{
Assert(mergeWhenClause->commandType == CMD_UPDATE ||
mergeWhenClause->commandType == CMD_DELETE ||
@@ -73,11 +73,25 @@ setNamespaceForMergeWhen(ParseState *pst
setNamespaceVisibilityForRTE(pstate->p_namespace,
sourceRelRTE, true, true);
}
- else
+ else if (mergeWhenClause->matchKind == MERGE_WHEN_NOT_MATCHED_BY_SOURCE)
{
/*
- * NOT MATCHED actions can't see target relation, but they can see
- * source relation.
+ * NOT MATCHED BY SOURCE actions can see the target relation, but they
+ * can't see the source relation.
+ */
+ Assert(mergeWhenClause->commandType == CMD_UPDATE ||
+ mergeWhenClause->commandType == CMD_DELETE ||
+ mergeWhenClause->commandType == CMD_NOTHING);
+ setNamespaceVisibilityForRTE(pstate->p_namespace,
+ targetRelRTE, true, true);
+ setNamespaceVisibilityForRTE(pstate->p_namespace,
+ sourceRelRTE, false, false);
+ }
+ else /* MERGE_WHEN_NOT_MATCHED_BY_TARGET */
+ {
+ /*
+ * NOT MATCHED [BY TARGET] actions can't see target relation, but they
+ * can see source relation.
*/
Assert(mergeWhenClause->commandType == CMD_INSERT ||
mergeWhenClause->commandType == CMD_NOTHING);
@@ -98,8 +112,7 @@ transformMergeStmt(ParseState *pstate, M
Query *qry = makeNode(Query);
ListCell *l;
AclMode targetPerms = ACL_NO_RIGHTS;
- bool is_terminal[2];
- Index sourceRTI;
+ bool is_terminal[3];
List *mergeActionList;
Node *joinExpr;
ParseNamespaceItem *nsitem;
@@ -125,12 +138,12 @@ transformMergeStmt(ParseState *pstate, M
/*
* Check WHEN clauses for permissions and sanity
*/
- is_terminal[0] = false;
- is_terminal[1] = false;
+ is_terminal[MERGE_WHEN_MATCHED] = false;
+ is_terminal[MERGE_WHEN_NOT_MATCHED_BY_SOURCE] = false;
+ is_terminal[MERGE_WHEN_NOT_MATCHED_BY_TARGET] = false;
foreach(l, stmt->mergeWhenClauses)
{
MergeWhenClause *mergeWhenClause = (MergeWhenClause *) lfirst(l);
- int when_type = (mergeWhenClause->matched ? 0 : 1);
/*
* Collect action types so we can check target permissions
@@ -155,12 +168,12 @@ transformMergeStmt(ParseState *pstate, M
/*
* Check for unreachable WHEN clauses
*/
- if (is_terminal[when_type])
+ if (is_terminal[mergeWhenClause->matchKind])
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unreachable WHEN clause specified after unconditional WHEN clause")));
if (mergeWhenClause->condition == NULL)
- is_terminal[when_type] = true;
+ is_terminal[mergeWhenClause->matchKind] = true;
}
/*
@@ -193,8 +206,8 @@ transformMergeStmt(ParseState *pstate, M
/* Now transform the source relation to produce the source RTE. */
transformFromClause(pstate,
list_make1(stmt->sourceRelation));
- sourceRTI = list_length(pstate->p_rtable);
- nsitem = GetNSItemByRangeTablePosn(pstate, sourceRTI, 0);
+ qry->mergeSourceRelation = list_length(pstate->p_rtable);
+ nsitem = GetNSItemByRangeTablePosn(pstate, qry->mergeSourceRelation, 0);
/*
* Check that the target table doesn't conflict with the source table.
@@ -255,11 +268,7 @@ transformMergeStmt(ParseState *pstate, M
action = makeNode(MergeAction);
action->commandType = mergeWhenClause->commandType;
- action->matched = mergeWhenClause->matched;
-
- /* Use an outer join if any INSERT actions exist in the command. */
- if (action->commandType == CMD_INSERT)
- qry->mergeUseOuterJoin = true;
+ action->matchKind = mergeWhenClause->matchKind;
/*
* Set namespace for the specific action. This must be done before
@@ -267,7 +276,7 @@ transformMergeStmt(ParseState *pstate, M
*/
setNamespaceForMergeWhen(pstate, mergeWhenClause,
qry->resultRelation,
- sourceRTI);
+ qry->mergeSourceRelation);
/*
* Transform the WHEN condition.
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
new file mode 100644
index 76c97a5..c04174c
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -496,9 +496,10 @@ OffsetVarNodes(Node *node, int offset, i
/*
* If we are starting at a Query, and sublevels_up is zero, then we
* must also fix rangetable indexes in the Query itself --- namely
- * resultRelation, exclRelIndex and rowMarks entries. sublevels_up
- * cannot be zero when recursing into a subquery, so there's no need
- * to have the same logic inside OffsetVarNodes_walker.
+ * resultRelation, mergeSourceRelation, exclRelIndex and rowMarks
+ * entries. sublevels_up cannot be zero when recursing into a
+ * subquery, so there's no need to have the same logic inside
+ * OffsetVarNodes_walker.
*/
if (sublevels_up == 0)
{
@@ -507,6 +508,9 @@ OffsetVarNodes(Node *node, int offset, i
if (qry->resultRelation)
qry->resultRelation += offset;
+ if (qry->mergeSourceRelation)
+ qry->mergeSourceRelation += offset;
+
if (qry->onConflict && qry->onConflict->exclRelIndex)
qry->onConflict->exclRelIndex += offset;
@@ -687,9 +691,10 @@ ChangeVarNodes(Node *node, int rt_index,
/*
* If we are starting at a Query, and sublevels_up is zero, then we
* must also fix rangetable indexes in the Query itself --- namely
- * resultRelation and rowMarks entries. sublevels_up cannot be zero
- * when recursing into a subquery, so there's no need to have the same
- * logic inside ChangeVarNodes_walker.
+ * resultRelation, mergeSourceRelation, exclRelIndex and rowMarks
+ * entries. sublevels_up cannot be zero when recursing into a
+ * subquery, so there's no need to have the same logic inside
+ * ChangeVarNodes_walker.
*/
if (sublevels_up == 0)
{
@@ -698,6 +703,9 @@ ChangeVarNodes(Node *node, int rt_index,
if (qry->resultRelation == rt_index)
qry->resultRelation = new_index;
+ if (qry->mergeSourceRelation == rt_index)
+ qry->mergeSourceRelation = new_index;
+
/* this is unlikely to ever be used, but ... */
if (qry->onConflict && qry->onConflict->exclRelIndex == rt_index)
qry->onConflict->exclRelIndex = new_index;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
new file mode 100644
index b625f47..51a1a2a
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -7093,6 +7093,7 @@ get_merge_query_def(Query *query, depars
StringInfo buf = context->buf;
RangeTblEntry *rte;
ListCell *lc;
+ bool haveNotMatchedBySource;
/* Insert the WITH clause if given */
get_with_clause(query, context);
@@ -7120,6 +7121,25 @@ get_merge_query_def(Query *query, depars
-PRETTYINDENT_STD, PRETTYINDENT_STD, 2);
get_rule_expr(query->jointree->quals, context, false);
+ /*
+ * Test for any NOT MATCHED BY SOURCE actions. If there are none, then
+ * any NOT MATCHED BY TARGET actions are output as "WHEN NOT MATCHED", per
+ * SQL standard. Otherwise, we have a non-SQL-standard query, so output
+ * "BY SOURCE" / "BY TARGET" qualifiers for all NOT MATCHED actions, to be
+ * more explicit.
+ */
+ haveNotMatchedBySource = false;
+ foreach(lc, query->mergeActionList)
+ {
+ MergeAction *action = lfirst_node(MergeAction, lc);
+
+ if (action->matchKind == MERGE_WHEN_NOT_MATCHED_BY_SOURCE)
+ {
+ haveNotMatchedBySource = true;
+ break;
+ }
+ }
+
/* Print each merge action */
foreach(lc, query->mergeActionList)
{
@@ -7127,7 +7147,24 @@ get_merge_query_def(Query *query, depars
appendContextKeyword(context, " WHEN ",
-PRETTYINDENT_STD, PRETTYINDENT_STD, 2);
- appendStringInfo(buf, "%sMATCHED", action->matched ? "" : "NOT ");
+ switch (action->matchKind)
+ {
+ case MERGE_WHEN_MATCHED:
+ appendStringInfo(buf, "MATCHED");
+ break;
+ case MERGE_WHEN_NOT_MATCHED_BY_SOURCE:
+ appendStringInfo(buf, "NOT MATCHED BY SOURCE");
+ break;
+ case MERGE_WHEN_NOT_MATCHED_BY_TARGET:
+ if (haveNotMatchedBySource)
+ appendStringInfo(buf, "NOT MATCHED BY TARGET");
+ else
+ appendStringInfo(buf, "NOT MATCHED");
+ break;
+ default:
+ elog(ERROR, "unrecognized matchKind: %d",
+ (int) action->matchKind);
+ }
if (action->qual)
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
new file mode 100644
index ada711d..8349f11
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -4331,17 +4331,35 @@ psql_completion(const char *text, int st
TailMatches("USING", MatchAny, MatchAny, "ON", MatchAny, MatchAny, MatchAny, "WHEN"))
COMPLETE_WITH("MATCHED", "NOT MATCHED");
- /* Complete ... WHEN [NOT] MATCHED with THEN/AND */
+ /*
+ * Complete ... WHEN MATCHED and WHEN NOT MATCHED BY SOURCE|TARGET with
+ * THEN/AND
+ */
else if (TailMatches("WHEN", "MATCHED") ||
- TailMatches("WHEN", "NOT", "MATCHED"))
+ TailMatches("WHEN", "NOT", "MATCHED", "BY", "SOURCE|TARGET"))
COMPLETE_WITH("THEN", "AND");
- /* Complete ... WHEN MATCHED THEN with UPDATE SET/DELETE/DO NOTHING */
- else if (TailMatches("WHEN", "MATCHED", "THEN"))
+ /* Complete ... WHEN NOT MATCHED with BY/THEN/AND */
+ else if (TailMatches("WHEN", "NOT", "MATCHED"))
+ COMPLETE_WITH("BY", "THEN", "AND");
+
+ /* Complete ... WHEN NOT MATCHED BY with SOURCE/TARGET */
+ else if (TailMatches("WHEN", "NOT", "MATCHED", "BY"))
+ COMPLETE_WITH("SOURCE", "TARGET");
+
+ /*
+ * Complete ... WHEN MATCHED THEN and WHEN NOT MATCHED BY SOURCE THEN with
+ * UPDATE SET/DELETE/DO NOTHING
+ */
+ else if (TailMatches("WHEN", "MATCHED", "THEN") ||
+ TailMatches("WHEN", "NOT", "MATCHED", "BY", "SOURCE", "THEN"))
COMPLETE_WITH("UPDATE SET", "DELETE", "DO NOTHING");
- /* Complete ... WHEN NOT MATCHED THEN with INSERT/DO NOTHING */
- else if (TailMatches("WHEN", "NOT", "MATCHED", "THEN"))
+ /*
+ * Complete ... WHEN NOT MATCHED [BY TARGET] THEN with INSERT/DO NOTHING
+ */
+ else if (TailMatches("WHEN", "NOT", "MATCHED", "THEN") ||
+ TailMatches("WHEN", "NOT", "MATCHED", "BY", "TARGET", "THEN"))
COMPLETE_WITH("INSERT", "DO NOTHING");
/* NOTIFY --- can be inside EXPLAIN, RULE, etc */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
new file mode 100644
index 476d55d..d9ed9ed
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -175,8 +175,9 @@ typedef struct Query
* also USING clause for MERGE */
List *mergeActionList; /* list of actions for MERGE (only) */
- /* whether to use outer join */
- bool mergeUseOuterJoin pg_node_attr(query_jumble_ignore);
+
+ /* rtable index of source relation for MERGE */
+ int mergeSourceRelation pg_node_attr(query_jumble_ignore);
List *targetList; /* target list (of TargetEntry) */
@@ -1665,7 +1666,7 @@ typedef struct CommonTableExpr
typedef struct MergeWhenClause
{
NodeTag type;
- bool matched; /* true=MATCHED, false=NOT MATCHED */
+ MergeMatchKind matchKind; /* MATCHED/NOT MATCHED BY SOURCE/TARGET */
CmdType commandType; /* INSERT/UPDATE/DELETE/DO NOTHING */
OverridingKind override; /* OVERRIDING clause */
Node *condition; /* WHEN conditions (raw parser) */
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
new file mode 100644
index 4a15460..f7eebd8
--- a/src/include/nodes/primnodes.h
+++ b/src/include/nodes/primnodes.h
@@ -1732,10 +1732,18 @@ typedef struct BooleanTest
*
* Transformed representation of a WHEN clause in a MERGE statement
*/
+
+typedef enum MergeMatchKind
+{
+ MERGE_WHEN_MATCHED,
+ MERGE_WHEN_NOT_MATCHED_BY_SOURCE,
+ MERGE_WHEN_NOT_MATCHED_BY_TARGET
+} MergeMatchKind;
+
typedef struct MergeAction
{
NodeTag type;
- bool matched; /* true=MATCHED, false=NOT MATCHED */
+ MergeMatchKind matchKind; /* MATCHED/NOT MATCHED BY SOURCE/TARGET */
CmdType commandType; /* INSERT/UPDATE/DELETE/DO NOTHING */
/* OVERRIDING clause */
OverridingKind override pg_node_attr(query_jumble_ignore);
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
new file mode 100644
index 2331aca..b7d5515
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -403,6 +403,7 @@ PG_KEYWORD("skip", SKIP, UNRESERVED_KEYW
PG_KEYWORD("smallint", SMALLINT, COL_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("snapshot", SNAPSHOT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("some", SOME, RESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("source", SOURCE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("sql", SQL_P, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("stable", STABLE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("standalone", STANDALONE_P, UNRESERVED_KEYWORD, BARE_LABEL)
@@ -426,6 +427,7 @@ PG_KEYWORD("table", TABLE, RESERVED_KEYW
PG_KEYWORD("tables", TABLES, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("tablesample", TABLESAMPLE, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("tablespace", TABLESPACE, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("target", TARGET, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("temp", TEMP, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("template", TEMPLATE, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("temporary", TEMPORARY, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/test/regress/expected/merge.out b/src/test/regress/expected/merge.out
new file mode 100644
index f87905f..b13eda0
--- a/src/test/regress/expected/merge.out
+++ b/src/test/regress/expected/merge.out
@@ -71,6 +71,15 @@ WHEN MATCHED THEN
ERROR: syntax error at or near "INSERT"
LINE 5: INSERT DEFAULT VALUES;
^
+-- NOT MATCHED BY SOURCE/INSERT error
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE THEN
+ INSERT DEFAULT VALUES;
+ERROR: syntax error at or near "INSERT"
+LINE 5: INSERT DEFAULT VALUES;
+ ^
-- incorrectly specifying INTO target
MERGE INTO target t
USING source AS s
@@ -107,6 +116,15 @@ WHEN NOT MATCHED THEN
ERROR: syntax error at or near "UPDATE"
LINE 5: UPDATE SET balance = 0;
^
+-- NOT MATCHED BY TARGET/UPDATE
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY TARGET THEN
+ UPDATE SET balance = 0;
+ERROR: syntax error at or near "UPDATE"
+LINE 5: UPDATE SET balance = 0;
+ ^
-- UPDATE tablename
MERGE INTO target t
USING source AS s
@@ -271,6 +289,22 @@ SELECT * FROM target ORDER BY tid;
(4 rows)
ROLLBACK;
+-- DELETE/INSERT not matched by source/target
+BEGIN;
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE
+WHEN NOT MATCHED BY TARGET THEN
+ INSERT VALUES (s.sid, s.delta);
+SELECT * FROM target ORDER BY tid;
+ tid | balance
+-----+---------
+ 4 | 40
+(1 row)
+
+ROLLBACK;
-- index plans
INSERT INTO target SELECT generate_series(1000,2500), 0;
ALTER TABLE target ADD PRIMARY KEY (tid);
@@ -738,6 +772,19 @@ SELECT * FROM wq_target;
1 | -1
(1 row)
+-- conditions in NOT MATCHED BY SOURCE clause can only refer to target columns
+MERGE INTO wq_target t
+USING wq_source s ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE AND s.balance = 100 THEN
+ DELETE;
+ERROR: invalid reference to FROM-clause entry for table "s"
+LINE 3: WHEN NOT MATCHED BY SOURCE AND s.balance = 100 THEN
+ ^
+DETAIL: There is an entry for table "s", but it cannot be referenced from this part of the query.
+MERGE INTO wq_target t
+USING wq_source s ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE AND t.balance = 100 THEN
+ DELETE;
-- conditions in MATCHED clause can refer to both source and target
SELECT * FROM wq_source;
balance | sid
@@ -923,6 +970,45 @@ SELECT * FROM target ORDER BY tid;
(3 rows)
ROLLBACK;
+-- UPSERT with UPDATE/DELETE when not matched by source
+BEGIN;
+DELETE FROM SOURCE WHERE sid = 2;
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN MATCHED AND t.balance > s.delta THEN
+ UPDATE SET balance = t.balance - s.delta
+WHEN MATCHED THEN
+ UPDATE SET balance = 0
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.sid, s.delta)
+WHEN NOT MATCHED BY SOURCE AND tid = 1 THEN
+ UPDATE SET balance = 0
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE;
+NOTICE: BEFORE INSERT STATEMENT trigger
+NOTICE: BEFORE UPDATE STATEMENT trigger
+NOTICE: BEFORE DELETE STATEMENT trigger
+NOTICE: BEFORE UPDATE ROW trigger row: (3,30) -> (3,10)
+NOTICE: BEFORE INSERT ROW trigger row: (4,40)
+NOTICE: BEFORE DELETE ROW trigger row: (2,20)
+NOTICE: BEFORE UPDATE ROW trigger row: (1,10) -> (1,0)
+NOTICE: AFTER UPDATE ROW trigger row: (3,30) -> (3,10)
+NOTICE: AFTER INSERT ROW trigger row: (4,40)
+NOTICE: AFTER DELETE ROW trigger row: (2,20)
+NOTICE: AFTER UPDATE ROW trigger row: (1,10) -> (1,0)
+NOTICE: AFTER DELETE STATEMENT trigger
+NOTICE: AFTER UPDATE STATEMENT trigger
+NOTICE: AFTER INSERT STATEMENT trigger
+SELECT * FROM target ORDER BY tid;
+ tid | balance
+-----+---------
+ 1 | 0
+ 3 | 10
+ 4 | 40
+(3 rows)
+
+ROLLBACK;
-- Test behavior of triggers that turn UPDATE/DELETE into no-ops
create or replace function skip_merge_op() returns trigger
language plpgsql as
@@ -1451,6 +1537,50 @@ WHEN NOT MATCHED AND s.a < 20 THEN
-> Seq Scan on ex_mtarget t (actual rows=49 loops=1)
(12 rows)
+-- not matched by source
+SELECT explain_merge('
+MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a
+WHEN NOT MATCHED BY SOURCE and t.a < 10 THEN
+ DELETE');
+ explain_merge
+----------------------------------------------------------------------
+ Merge on ex_mtarget t (actual rows=0 loops=1)
+ Tuples: skipped=54
+ -> Merge Left Join (actual rows=54 loops=1)
+ Merge Cond: (t.a = s.a)
+ -> Sort (actual rows=54 loops=1)
+ Sort Key: t.a
+ Sort Method: quicksort Memory: xxx
+ -> Seq Scan on ex_mtarget t (actual rows=54 loops=1)
+ -> Sort (actual rows=100 loops=1)
+ Sort Key: s.a
+ Sort Method: quicksort Memory: xxx
+ -> Seq Scan on ex_msource s (actual rows=100 loops=1)
+(12 rows)
+
+-- not matched by source and target
+SELECT explain_merge('
+MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a
+WHEN NOT MATCHED BY SOURCE AND t.a < 10 THEN
+ DELETE
+WHEN NOT MATCHED BY TARGET AND s.a < 20 THEN
+ INSERT VALUES (a, b)');
+ explain_merge
+----------------------------------------------------------------------
+ Merge on ex_mtarget t (actual rows=0 loops=1)
+ Tuples: skipped=100
+ -> Merge Full Join (actual rows=100 loops=1)
+ Merge Cond: (t.a = s.a)
+ -> Sort (actual rows=54 loops=1)
+ Sort Key: t.a
+ Sort Method: quicksort Memory: xxx
+ -> Seq Scan on ex_mtarget t (actual rows=54 loops=1)
+ -> Sort (actual rows=100 loops=1)
+ Sort Key: s.a
+ Sort Method: quicksort Memory: xxx
+ -> Seq Scan on ex_msource s (actual rows=100 loops=1)
+(12 rows)
+
-- nothing
SELECT explain_merge('
MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a AND t.a < -1000
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
new file mode 100644
index abc944e..f1c0094
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3705,7 +3705,39 @@ BEGIN ATOMIC
THEN INSERT (filling[1], id)
VALUES (s.a, s.a);
END
+CREATE FUNCTION merge_sf_test2()
+ RETURNS void
+ LANGUAGE sql
+BEGIN ATOMIC
+ MERGE INTO sf_target t
+ USING rule_merge1 s
+ ON (s.a = t.id)
+WHEN NOT MATCHED
+ THEN INSERT (data, id)
+ VALUES (s.a, s.a)
+WHEN MATCHED
+ THEN UPDATE SET data = s.b
+WHEN NOT MATCHED BY SOURCE
+ THEN DELETE;
+END;
+\sf merge_sf_test2
+CREATE OR REPLACE FUNCTION public.merge_sf_test2()
+ RETURNS void
+ LANGUAGE sql
+BEGIN ATOMIC
+ MERGE INTO sf_target t
+ USING rule_merge1 s
+ ON (s.a = t.id)
+ WHEN NOT MATCHED BY TARGET
+ THEN INSERT (data, id)
+ VALUES (s.a, s.a)
+ WHEN MATCHED
+ THEN UPDATE SET data = s.b
+ WHEN NOT MATCHED BY SOURCE
+ THEN DELETE;
+END
DROP FUNCTION merge_sf_test;
+DROP FUNCTION merge_sf_test2;
DROP TABLE sf_target;
--
-- Test enabling/disabling
diff --git a/src/test/regress/sql/merge.sql b/src/test/regress/sql/merge.sql
new file mode 100644
index 66cb75a..7e59c9e
--- a/src/test/regress/sql/merge.sql
+++ b/src/test/regress/sql/merge.sql
@@ -51,6 +51,12 @@ USING source AS s
ON t.tid = s.sid
WHEN MATCHED THEN
INSERT DEFAULT VALUES;
+-- NOT MATCHED BY SOURCE/INSERT error
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE THEN
+ INSERT DEFAULT VALUES;
-- incorrectly specifying INTO target
MERGE INTO target t
USING source AS s
@@ -75,6 +81,12 @@ USING source AS s
ON t.tid = s.sid
WHEN NOT MATCHED THEN
UPDATE SET balance = 0;
+-- NOT MATCHED BY TARGET/UPDATE
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY TARGET THEN
+ UPDATE SET balance = 0;
-- UPDATE tablename
MERGE INTO target t
USING source AS s
@@ -213,6 +225,18 @@ WHEN NOT MATCHED THEN
SELECT * FROM target ORDER BY tid;
ROLLBACK;
+-- DELETE/INSERT not matched by source/target
+BEGIN;
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE
+WHEN NOT MATCHED BY TARGET THEN
+ INSERT VALUES (s.sid, s.delta);
+SELECT * FROM target ORDER BY tid;
+ROLLBACK;
+
-- index plans
INSERT INTO target SELECT generate_series(1000,2500), 0;
ALTER TABLE target ADD PRIMARY KEY (tid);
@@ -497,6 +521,17 @@ WHEN NOT MATCHED AND s.balance = 100 THE
INSERT (tid) VALUES (s.sid);
SELECT * FROM wq_target;
+-- conditions in NOT MATCHED BY SOURCE clause can only refer to target columns
+MERGE INTO wq_target t
+USING wq_source s ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE AND s.balance = 100 THEN
+ DELETE;
+
+MERGE INTO wq_target t
+USING wq_source s ON t.tid = s.sid
+WHEN NOT MATCHED BY SOURCE AND t.balance = 100 THEN
+ DELETE;
+
-- conditions in MATCHED clause can refer to both source and target
SELECT * FROM wq_source;
MERGE INTO wq_target t
@@ -624,6 +659,25 @@ WHEN NOT MATCHED THEN
SELECT * FROM target ORDER BY tid;
ROLLBACK;
+-- UPSERT with UPDATE/DELETE when not matched by source
+BEGIN;
+DELETE FROM SOURCE WHERE sid = 2;
+MERGE INTO target t
+USING source AS s
+ON t.tid = s.sid
+WHEN MATCHED AND t.balance > s.delta THEN
+ UPDATE SET balance = t.balance - s.delta
+WHEN MATCHED THEN
+ UPDATE SET balance = 0
+WHEN NOT MATCHED THEN
+ INSERT VALUES (s.sid, s.delta)
+WHEN NOT MATCHED BY SOURCE AND tid = 1 THEN
+ UPDATE SET balance = 0
+WHEN NOT MATCHED BY SOURCE THEN
+ DELETE;
+SELECT * FROM target ORDER BY tid;
+ROLLBACK;
+
-- Test behavior of triggers that turn UPDATE/DELETE into no-ops
create or replace function skip_merge_op() returns trigger
language plpgsql as
@@ -929,6 +983,20 @@ WHEN MATCHED AND t.a >= 30 AND t.a <= 40
WHEN NOT MATCHED AND s.a < 20 THEN
INSERT VALUES (a, b)');
+-- not matched by source
+SELECT explain_merge('
+MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a
+WHEN NOT MATCHED BY SOURCE and t.a < 10 THEN
+ DELETE');
+
+-- not matched by source and target
+SELECT explain_merge('
+MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a
+WHEN NOT MATCHED BY SOURCE AND t.a < 10 THEN
+ DELETE
+WHEN NOT MATCHED BY TARGET AND s.a < 20 THEN
+ INSERT VALUES (a, b)');
+
-- nothing
SELECT explain_merge('
MERGE INTO ex_mtarget t USING ex_msource s ON t.a = s.a AND t.a < -1000
diff --git a/src/test/regress/sql/rules.sql b/src/test/regress/sql/rules.sql
new file mode 100644
index 93aff4e..4ea85a2
--- a/src/test/regress/sql/rules.sql
+++ b/src/test/regress/sql/rules.sql
@@ -1323,7 +1323,26 @@ END;
\sf merge_sf_test
+CREATE FUNCTION merge_sf_test2()
+ RETURNS void
+ LANGUAGE sql
+BEGIN ATOMIC
+ MERGE INTO sf_target t
+ USING rule_merge1 s
+ ON (s.a = t.id)
+WHEN NOT MATCHED
+ THEN INSERT (data, id)
+ VALUES (s.a, s.a)
+WHEN MATCHED
+ THEN UPDATE SET data = s.b
+WHEN NOT MATCHED BY SOURCE
+ THEN DELETE;
+END;
+
+\sf merge_sf_test2
+
DROP FUNCTION merge_sf_test;
+DROP FUNCTION merge_sf_test2;
DROP TABLE sf_target;
--
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
new file mode 100644
index 7e866e3..7892693
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1570,6 +1570,7 @@ MergeAppendState
MergeJoin
MergeJoinClause
MergeJoinState
+MergeMatchKind
MergePath
MergeScanSelCache
MergeStmt
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: MERGE ... WHEN NOT MATCHED BY SOURCE
@ 2024-01-26 15:57 Alvaro Herrera <[email protected]>
parent: Dean Rasheed <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Alvaro Herrera @ 2024-01-26 15:57 UTC (permalink / raw)
To: Dean Rasheed <[email protected]>; +Cc: Peter Smith <[email protected]>; Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]>
On 2024-Jan-26, Dean Rasheed wrote:
> I think it has had a decent amount of review and all the review
> comments have been addressed. I'm not quite sure from Alvaro's last
> comment whether he was implying that he thought it was ready for
> commit.
Well, firstly this is clearly a feature we want to have, even though
it's non-standard, because people use it and other implementations have
it. (Eh, so maybe somebody should be talking to the SQL standard
committee about it). As for code quality, I didn't do a comprehensive
review, but I think it is quite reasonable. Therefore, my inclination
would be to get it committed soonish, and celebrate it widely so that
people can test it soon and complain if they see something they don't
like.
I have to say that I find the idea of booting patches as Returned with
Feedback just because of inactivity (as opposed to unresponsive authors)
rather wrong-headed, and I wish we didn't do it.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: MERGE ... WHEN NOT MATCHED BY SOURCE
@ 2024-01-29 10:07 Dean Rasheed <[email protected]>
parent: Alvaro Herrera <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Dean Rasheed @ 2024-01-29 10:07 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Peter Smith <[email protected]>; Vik Fearing <[email protected]>; PostgreSQL Hackers <[email protected]>
On Fri, 26 Jan 2024 at 15:57, Alvaro Herrera <[email protected]> wrote:
>
> Well, firstly this is clearly a feature we want to have, even though
> it's non-standard, because people use it and other implementations have
> it. (Eh, so maybe somebody should be talking to the SQL standard
> committee about it). As for code quality, I didn't do a comprehensive
> review, but I think it is quite reasonable. Therefore, my inclination
> would be to get it committed soonish, and celebrate it widely so that
> people can test it soon and complain if they see something they don't
> like.
>
Thanks. I have been going over this patch again, and for the most
part, I'm pretty happy with it.
One thing that's bothering me though is what happens if a row being
merged is concurrently updated. Specifically, if a concurrent update
causes a formerly matching row to no longer match the join condition,
and there are both NOT MATCHED BY SOURCE and NOT MATCHED BY TARGET
actions, so that it's doing in full join between the source and target
relations. In this case, when the EPQ mechanism rescans the subplan
node, there will be 2 possible output tuples (one with source null,
and one with target null), and EvalPlanQual() will just return the
first one, which is a more-or-less arbitrary choice, depending on the
type of join (hash/merge), and (for a mergejoin) the values of the
inner and outer join keys. Thus, it may execute a NOT MATCHED BY
SOURCE action, or a NOT MATCHED BY TARGET action, and it's difficult
to predict which.
Arguably it's not worth worrying too much about what happens in a
corner-case concurrent update like this, when MERGE is already
inconsistent under other concurrent update scenarios, but I don't like
having unpredictable results like this, which can depend on the plan
chosen.
I think the best (and probably simplest) solution is to always opt for
a NOT MATCHED BY TARGET action in this case, so then the result is
predictable, and we can document what is expected to happen.
Regards,
Dean
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2024-04-14 03:16 Thomas Munro <[email protected]>
parent: Heikki Linnakangas <[email protected]>
1 sibling, 1 reply; 35+ messages in thread
From: Thomas Munro @ 2024-04-14 03:16 UTC (permalink / raw)
To: Heikki Linnakangas <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
On Mon, Jul 3, 2023 at 8:43 PM Heikki Linnakangas <[email protected]> wrote:
> On 10/06/2023 05:23, Thomas Munro wrote:
> > 2. I don't like the way we have to deal with POSIX vs Windows at
> > every site where we use threads, and each place has a different style
> > of wrappers. I considered a few different approaches to cleaning this
> > up:
> >
> > * provide centralised and thorough pthread emulation for Windows; I
> > don't like this, I don't even like all of pthreads and there are many
> > details to get lost in
> > * adopt C11 <threads.h>; unfortunately it is too early, so you'd need
> > to write/borrow replacements for at least 3 of our 11 target systems
> > * invent our own mini-abstraction for a carefully controlled subset of stuff
>
> Google search on "c11 threads on Windows" found some emulation wrappers:
> https://github.com/jtsiomb/c11threads and
> https://github.com/tinycthread/tinycthread, for example. Would either of
> those work for us?
>
> Even if we use an existing emulation wrapper, I wouldn't mind having our
> own pg_* abstration on top of it, to document which subset of the POSIX
> or C11 functions we actually use.
Yeah. I am still interested in getting our thread API tidied up, and
I intend to do that for PG 18. The patch I posted on that front,
which can be summarised as a very lightweight subset of standard
<threads.h> except with pg_ prefixes everywhere mapping to Windows or
POSIX threads, still needs one tiny bit more work: figuring out how to
get the TLS destructors to run on Windows FLS or similar, or
implementing destructors myself (= little destructor callback list
that a thread exit hook would run, work I was hoping to avoid by using
something from the OS libs... I will try again soon). Andres also
opined upthread that we should think about offering a thread_local
storage class and getting away from TLS with keys.
One thing to note is that the ECPG code is using TLS with destructors
(in fact they are b0rked in master, git grep "FIXME: destructor" so
ECPG leaks memory on Windows, so the thing that I'm trying to fix in
pg_threads.h is actually fixing a long-standing bug), and although
thread_local has destructors in C++ it doesn't in C, so if we decided
to add the storage class but not bother with the tss_create feature,
then ECPG would need to do cleanup another way. I will look into that
option.
One new development since last time I wrote the above stuff is that
the Microsoft toolchain finally implemented the library components of
C11 <threads.h>:
https://devblogs.microsoft.com/cppblog/c11-threads-in-visual-studio-2022-version-17-8-preview-2/
It seems like it would be a long time before we could contemplate
using that stuff though, especially given our duty of keeping libpq
and ECPG requirements low and reasonable. However, it seems fairly
straightforward to say that we require C99 + some way to access a
C11-like thread local storage class. In practice that means a
pg_thread_local macro that points to MSVC __declspec(thread) (which
works since MSVC 2014?) or GCC/Clang __thread_local (which works since
GCC4.8 in 2013?) or whatever. Of course every serious toolchain
supports this stuff somehow or other, having been required to for C11.
I can't immediately see any build farm animals that would be affected.
Grison's compiler info is out of date, it's really running
8.something. The old OpenBSD GCC 4.2 animal was upgraded, and antique
AIX got the boot: that's not even a coincidence, those retirements
came about because those systems didn't support arbitrary alignment,
another C11 feature that we now effectively require. (We could have
worked around it it we had to but on but they just weren't reasonable
targets.)
So I'll go ahead and add the storage class to the next version, and
contemplate a couple of different options for the tss stuff, including
perhaps leaving it out if that seems doable.
https://stackoverflow.com/questions/29399494/what-is-the-current-state-of-support-for-thread-local-a...
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2024-08-21 12:51 Thomas Munro <[email protected]>
parent: Thomas Munro <[email protected]>
0 siblings, 1 reply; 35+ messages in thread
From: Thomas Munro @ 2024-08-21 12:51 UTC (permalink / raw)
To: Heikki Linnakangas <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
On Sun, Apr 14, 2024 at 3:16 PM Thomas Munro <[email protected]> wrote:
> So I'll go ahead and add the storage class to the next version, and
> contemplate a couple of different options for the tss stuff, including
> perhaps leaving it out if that seems doable.
Here is a new attempt at pg_threads.h. Still work in progress, but
passing tests, with storage class and working TSS, showing various
users.
I eventually understood first that my TSS destructor problems on
Windows came from mismatched calling conventions, and that you can't
really trampoline your way around that, at least not without doing
some pretty unreasonable things, and that is why nobody can emulate
either tss_create() or pthread_key_create() directly with Windows'
FlsAlloc(), so everybody who tries finishes up building their own
infrastructure to track destructors, or in ECPG's case just leaks all
the memory instead.
Here's the simplest implementation I could come up with so far. I
don't have Windows so I made it possible to use emulated TSS
destructors on my local machine with a special macro for testing, and
then argued with CI for a while until the other machines agreed.
Otherwise, it's all a fairly thin wrapper and hopefully not suprising.
In one place, an ECPG thread-local variable has no destructor, so we
can use it as the first example of the new pg_thread_local storage
class.
One thing this would need to be complete, at least the way I've
implemented it, is memory barriers, for non-TSO hardware, which would
require lifting the ban on atomics.h in frontend code, or at least
parts of it. Only 64 bit emulation is actually tied to the backend
now (because it calls spinlock stuff, that itself is backend-only, but
also it doesn't actually need to be either). Or maybe I can figure
out a different scheme that doesn't need that. Or something...
WIP patch attached.
Attachments:
[text/x-patch] v3-0001-Add-port-pg_threads.h-for-a-common-threading-API.patch (96.4K, ../../CA+hUKG+gQjyAe-tt-j6X1t-RSPWywJc2x9dD69uQDZpmT-ujjw@mail.gmail.com/2-v3-0001-Add-port-pg_threads.h-for-a-common-threading-API.patch)
download | inline diff:
From ab649c89ca924c7aae39b56a3706d9f934e4e1ef Mon Sep 17 00:00:00 2001
From: Thomas Munro <[email protected]>
Date: Sat, 10 Jun 2023 09:14:07 +1200
Subject: [PATCH v3] Add port/pg_threads.h for a common threading API.
Loosely based on a subset of C11's <threads.h>, but with pg_ prefixes,
and some small additions:
* static initializers for mutexes
* read/write locks
* barriers
Clean up several several places that had to cope with POSIX and Windows
threads, by adopting pg_threads.h:
* pgbench had a lot of macros as a local abstraction
* ecpg had a sort of pthreads emulation for Windows
* libpq had a another pthread emulation for Windows
One place in ecpg can be the first to use the new pg_thread_local
storage class, which looks like a plain variable, because it has no
destructor. Other places can use pg_tss_XXX(), which now runs
destructors, even on Windows (which previously leaked memory when every
thread exited).
XXX To complete this, we need to make port/atomics.h available in
frontend code, for the memory barrier macros.
Reviewed-by: Andres Freund <[email protected]> (earlier versions)
Reviewed-by: Heikki Linnakangas <[email protected]>
Discussion: https://postgr.es/m/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
---
configure | 17 +-
configure.ac | 3 +-
src/bin/pgbench/pgbench.c | 67 +--
src/include/port/pg_pthread.h | 41 --
src/include/port/pg_threads.h | 542 ++++++++++++++++++
src/interfaces/ecpg/ecpglib/connect.c | 53 +-
src/interfaces/ecpg/ecpglib/descriptor.c | 14 +-
src/interfaces/ecpg/ecpglib/ecpglib_extern.h | 2 +-
src/interfaces/ecpg/ecpglib/memory.c | 14 +-
src/interfaces/ecpg/ecpglib/misc.c | 87 +--
src/interfaces/ecpg/ecpglib/sqlda.c | 1 -
.../ecpg/include/ecpg-pthread-win32.h | 49 --
src/interfaces/ecpg/include/ecpg_config.h.in | 3 +
src/interfaces/ecpg/include/meson.build | 1 +
.../ecpg/test/expected/thread-alloc.c | 73 +--
.../ecpg/test/expected/thread-descriptor.c | 52 +-
.../ecpg/test/expected/thread-prep.c | 99 ++--
.../ecpg/test/expected/thread-thread.c | 92 ++-
.../test/expected/thread-thread_implicit.c | 92 ++-
src/interfaces/ecpg/test/thread/alloc.pgc | 39 +-
.../ecpg/test/thread/descriptor.pgc | 38 +-
src/interfaces/ecpg/test/thread/prep.pgc | 37 +-
src/interfaces/ecpg/test/thread/thread.pgc | 38 +-
.../ecpg/test/thread/thread_implicit.pgc | 38 +-
src/interfaces/libpq/Makefile | 1 -
src/interfaces/libpq/fe-connect.c | 13 +-
src/interfaces/libpq/fe-secure-openssl.c | 45 +-
src/interfaces/libpq/fe-secure.c | 4 +-
src/interfaces/libpq/libpq-int.h | 5 -
src/interfaces/libpq/meson.build | 5 +-
src/interfaces/libpq/pthread-win32.c | 66 ---
src/port/Makefile | 1 +
src/port/meson.build | 5 +-
src/port/pg_threads.c | 436 ++++++++++++++
src/port/pthread-win32.h | 31 -
src/port/pthread_barrier_wait.c | 77 ---
src/tools/pginclude/headerscheck | 1 -
src/tools/pgindent/typedefs.list | 9 +
38 files changed, 1255 insertions(+), 936 deletions(-)
delete mode 100644 src/include/port/pg_pthread.h
create mode 100644 src/include/port/pg_threads.h
delete mode 100644 src/interfaces/ecpg/include/ecpg-pthread-win32.h
delete mode 100644 src/interfaces/libpq/pthread-win32.c
create mode 100644 src/port/pg_threads.c
delete mode 100644 src/port/pthread-win32.h
delete mode 100644 src/port/pthread_barrier_wait.c
diff --git a/configure b/configure
index 2abbeb2794..8cd797a3cb 100755
--- a/configure
+++ b/configure
@@ -15232,7 +15232,7 @@ fi
LIBS_including_readline="$LIBS"
LIBS=`echo "$LIBS" | sed -e 's/-ledit//g' -e 's/-lreadline//g'`
-for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
+for ac_func in backtrace_symbols copyfile copy_file_range getifaddrs getpeerucred inet_pton kqueue mbstowcs_l memset_s posix_fallocate ppoll pthread_barrier_wait pthread_is_threaded_np setproctitle setproctitle_fast strchrnul strsignal syncfs sync_file_range uselocale wcstombs_l
do :
as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -15920,21 +15920,6 @@ fi
-ac_fn_c_check_func "$LINENO" "pthread_barrier_wait" "ac_cv_func_pthread_barrier_wait"
-if test "x$ac_cv_func_pthread_barrier_wait" = xyes; then :
- $as_echo "#define HAVE_PTHREAD_BARRIER_WAIT 1" >>confdefs.h
-
-else
- case " $LIBOBJS " in
- *" pthread_barrier_wait.$ac_objext "* ) ;;
- *) LIBOBJS="$LIBOBJS pthread_barrier_wait.$ac_objext"
- ;;
-esac
-
-fi
-
-
-
if test "$PORTNAME" = "win32" -o "$PORTNAME" = "cygwin"; then
# Cygwin and (apparently, based on test results) Mingw both
# have a broken strtof(), so substitute its implementation.
diff --git a/configure.ac b/configure.ac
index c46ed2c591..59518a6335 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1740,6 +1740,7 @@ AC_CHECK_FUNCS(m4_normalize([
memset_s
posix_fallocate
ppoll
+ pthread_barrier_wait
pthread_is_threaded_np
setproctitle
setproctitle_fast
@@ -1799,8 +1800,6 @@ AC_REPLACE_FUNCS(m4_normalize([
strsep
]))
-AC_REPLACE_FUNCS(pthread_barrier_wait)
-
if test "$PORTNAME" = "win32" -o "$PORTNAME" = "cygwin"; then
# Cygwin and (apparently, based on test results) Mingw both
# have a broken strtof(), so substitute its implementation.
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 61618f2e18..a47a34f651 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -66,6 +66,7 @@
#include "libpq-fe.h"
#include "pgbench.h"
#include "port/pg_bitutils.h"
+#include "port/pg_threads.h"
#include "portability/instr_time.h"
/* X/Open (XSI) requires <math.h> to provide M_PI, but core POSIX does not */
@@ -113,49 +114,6 @@ typedef struct socket_set
#endif /* POLL_USING_SELECT */
-/*
- * Multi-platform thread implementations
- */
-
-#ifdef WIN32
-/* Use Windows threads */
-#include <windows.h>
-#define GETERRNO() (_dosmaperr(GetLastError()), errno)
-#define THREAD_T HANDLE
-#define THREAD_FUNC_RETURN_TYPE unsigned
-#define THREAD_FUNC_RETURN return 0
-#define THREAD_FUNC_CC __stdcall
-#define THREAD_CREATE(handle, function, arg) \
- ((*(handle) = (HANDLE) _beginthreadex(NULL, 0, (function), (arg), 0, NULL)) == 0 ? errno : 0)
-#define THREAD_JOIN(handle) \
- (WaitForSingleObject(handle, INFINITE) != WAIT_OBJECT_0 ? \
- GETERRNO() : CloseHandle(handle) ? 0 : GETERRNO())
-#define THREAD_BARRIER_T SYNCHRONIZATION_BARRIER
-#define THREAD_BARRIER_INIT(barrier, n) \
- (InitializeSynchronizationBarrier((barrier), (n), 0) ? 0 : GETERRNO())
-#define THREAD_BARRIER_WAIT(barrier) \
- EnterSynchronizationBarrier((barrier), \
- SYNCHRONIZATION_BARRIER_FLAGS_BLOCK_ONLY)
-#define THREAD_BARRIER_DESTROY(barrier)
-#else
-/* Use POSIX threads */
-#include "port/pg_pthread.h"
-#define THREAD_T pthread_t
-#define THREAD_FUNC_RETURN_TYPE void *
-#define THREAD_FUNC_RETURN return NULL
-#define THREAD_FUNC_CC
-#define THREAD_CREATE(handle, function, arg) \
- pthread_create((handle), NULL, (function), (arg))
-#define THREAD_JOIN(handle) \
- pthread_join((handle), NULL)
-#define THREAD_BARRIER_T pthread_barrier_t
-#define THREAD_BARRIER_INIT(barrier, n) \
- pthread_barrier_init((barrier), NULL, (n))
-#define THREAD_BARRIER_WAIT(barrier) pthread_barrier_wait((barrier))
-#define THREAD_BARRIER_DESTROY(barrier) pthread_barrier_destroy((barrier))
-#endif
-
-
/********************************************************************
* some configurable parameters */
@@ -478,7 +436,7 @@ typedef enum TStatus
static pg_prng_state base_random_sequence;
/* Synchronization barrier for start and connection */
-static THREAD_BARRIER_T barrier;
+static pg_barrier_t barrier;
/*
* Connection state machine states.
@@ -646,7 +604,7 @@ typedef struct
typedef struct
{
int tid; /* thread id */
- THREAD_T thread; /* thread handle */
+ pg_thrd_t thread; /* thread handle */
CState *state; /* array of CState */
int nstate; /* length of state[] */
@@ -830,7 +788,7 @@ static void doLog(TState *thread, CState *st,
static void processXactStats(TState *thread, CState *st, pg_time_usec_t *now,
bool skipped, StatsData *agg);
static void addScript(const ParsedScript *script);
-static THREAD_FUNC_RETURN_TYPE THREAD_FUNC_CC threadRun(void *arg);
+static int threadRun(void *arg);
static void finishCon(CState *st);
static void setalarm(int seconds);
static socket_set *alloc_socket_set(int count);
@@ -7311,7 +7269,7 @@ main(int argc, char **argv)
if (duration > 0)
setalarm(duration);
- errno = THREAD_BARRIER_INIT(&barrier, nthreads);
+ errno = pg_barrier_init(&barrier, nthreads);
if (errno != 0)
pg_fatal("could not initialize barrier: %m");
@@ -7321,7 +7279,7 @@ main(int argc, char **argv)
TState *thread = &threads[i];
thread->create_time = pg_time_now();
- errno = THREAD_CREATE(&thread->thread, threadRun, thread);
+ errno = pg_thrd_create(&thread->thread, threadRun, thread);
if (errno != 0)
pg_fatal("could not create thread: %m");
@@ -7344,7 +7302,7 @@ main(int argc, char **argv)
TState *thread = &threads[i];
if (i > 0)
- THREAD_JOIN(thread->thread);
+ pg_thrd_join(thread->thread, NULL);
for (int j = 0; j < thread->nstate; j++)
if (thread->state[j].state != CSTATE_FINISHED)
@@ -7384,7 +7342,7 @@ main(int argc, char **argv)
printResults(&stats, pg_time_now() - bench_start, conn_total_duration,
bench_start - start_time, latency_late);
- THREAD_BARRIER_DESTROY(&barrier);
+ pg_barrier_destroy(&barrier);
if (exit_code != 0)
pg_log_error("Run was aborted; the above results are incomplete.");
@@ -7392,7 +7350,7 @@ main(int argc, char **argv)
return exit_code;
}
-static THREAD_FUNC_RETURN_TYPE THREAD_FUNC_CC
+static int
threadRun(void *arg)
{
TState *thread = (TState *) arg;
@@ -7429,7 +7387,7 @@ threadRun(void *arg)
state[i].state = CSTATE_CHOOSE_SCRIPT;
/* READY */
- THREAD_BARRIER_WAIT(&barrier);
+ pg_barrier_wait(&barrier);
thread_start = pg_time_now();
thread->started_time = thread_start;
@@ -7453,7 +7411,7 @@ threadRun(void *arg)
}
/* GO */
- THREAD_BARRIER_WAIT(&barrier);
+ pg_barrier_wait(&barrier);
start = pg_time_now();
thread->bench_start = start;
@@ -7689,7 +7647,8 @@ done:
thread->logfile = NULL;
}
free_socket_set(sockets);
- THREAD_FUNC_RETURN;
+
+ return 0;
}
static void
diff --git a/src/include/port/pg_pthread.h b/src/include/port/pg_pthread.h
deleted file mode 100644
index d102ce9d6f..0000000000
--- a/src/include/port/pg_pthread.h
+++ /dev/null
@@ -1,41 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * Declarations for missing POSIX thread components.
- *
- * Currently this supplies an implementation of pthread_barrier_t for the
- * benefit of macOS, which lacks it. These declarations are not in port.h,
- * because that'd require <pthread.h> to be included by every translation
- * unit.
- *
- *-------------------------------------------------------------------------
- */
-
-#ifndef PG_PTHREAD_H
-#define PG_PTHREAD_H
-
-#include <pthread.h>
-
-#ifndef HAVE_PTHREAD_BARRIER_WAIT
-
-#ifndef PTHREAD_BARRIER_SERIAL_THREAD
-#define PTHREAD_BARRIER_SERIAL_THREAD (-1)
-#endif
-
-typedef struct pg_pthread_barrier
-{
- bool sense; /* we only need a one bit phase */
- int count; /* number of threads expected */
- int arrived; /* number of threads that have arrived */
- pthread_mutex_t mutex;
- pthread_cond_t cond;
-} pthread_barrier_t;
-
-extern int pthread_barrier_init(pthread_barrier_t *barrier,
- const void *attr,
- int count);
-extern int pthread_barrier_wait(pthread_barrier_t *barrier);
-extern int pthread_barrier_destroy(pthread_barrier_t *barrier);
-
-#endif
-
-#endif
diff --git a/src/include/port/pg_threads.h b/src/include/port/pg_threads.h
new file mode 100644
index 0000000000..2425d5cb1f
--- /dev/null
+++ b/src/include/port/pg_threads.h
@@ -0,0 +1,542 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_threads.h
+ * Portable multi-threading API.
+ *
+ * A multi-threading API abstraction loosely based on a subset C11
+ * standard's <threads.h> header. The identifiers have a pg_ prefix.
+ *
+ * We have some extensions of our own, not present in C11:
+ *
+ * - pg_rwlock_t for read/write locks
+ * - pg_mtx_t static initializer PG_MTX_STATIC_INIT
+ * - pg_barrier_t
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/port/pg_threads.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef PG_THREADS_H
+#define PG_THREADS_H
+
+#ifdef WIN32
+/*
+ * We use the macro PG_THREADS_WIN32 rather than WIN32 directly, because we
+ * might want to use the C11 APIs in Visual Studio 2022+ at some point.
+ * While using Windows native APIs, need an in-house implementation of TSS
+ * destructors, which we also gate separately so that it can be
+ * tested/maintained on other OSes too.
+ */
+#define PG_THREADS_WIN32
+#define PG_THREADS_NEED_DESTRUCTOR_TABLE
+#endif
+
+/*
+ * To test our own destructor mechanism on POSIX systems, for the
+ * benefit of developers maintaining it, define this macro.
+ */
+/* #define PG_THREADS_NEED_DESTRUCTOR_TABLE */
+
+#if defined(PG_THREADS_WIN32)
+#include <windows.h>
+#else
+#include <pthread.h>
+#endif
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Thread-local storage class. This is a C11 language feature, not a
+ * library feature. We don't require C11, but we expect compilers to
+ * provide some way to request thread-local storage. (See also
+ * pg_tss_t, which is similar but uses explicit set/get functions and
+ * supports destructor function that are called at thread exit.)
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#if defined(_MSC_VER)
+/* MSVC */
+#define pg_thread_local __declspec(thread)
+#elif defined(__GNUC__) || defined(__INTEL_COMPILER) || defined(__SUNPRO_C)
+/* GCC, Clang, Intel C, XLC, Solaris Studio */
+#define pg_thread_local __thread
+#else
+#error "no known thread_local storage class for this compiler"
+#endif
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Return values.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+typedef enum pg_thrd_error_t
+{
+ pg_thrd_success = 0,
+ pg_thrd_nomem = 1,
+ pg_thrd_timedout = 2,
+ pg_thrd_busy = 3,
+ pg_thrd_error = 4,
+
+ /* Not from C11. Needed by our pg_barrier_wait(). */
+ pg_thrd_success_last = 5,
+} pg_thrd_error_t;
+
+static inline int
+pg_thrd_maperror(int error)
+{
+#ifdef PG_THREADS_WIN32
+ return error ? pg_thrd_success : pg_thrd_error;
+#else
+ return error == 0 ? pg_thrd_success : pg_thrd_error;
+#endif
+}
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Threads.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef PG_THREADS_WIN32
+typedef HANDLE pg_thrd_t;
+#else
+typedef pthread_t pg_thrd_t;
+#endif
+
+typedef int (*pg_thrd_start_t) (void *);
+
+extern int pg_thrd_create(pg_thrd_t *thread, pg_thrd_start_t function, void *argument);
+extern int pg_thrd_join(pg_thrd_t thread, int *result);
+extern void pg_thrd_exit(int result);
+
+static inline pg_thrd_t
+pg_thrd_current(void)
+{
+#ifdef PG_THREADS_WIN32
+ return GetCurrentThreadId();
+#else
+ return pthread_self();
+#endif
+}
+
+static inline int
+pg_thrd_equal(pg_thrd_t lhs, pg_thrd_t rhs)
+{
+#ifdef PG_THREADS_WIN32
+ return lhs == rhs;
+#else
+ return pthread_equal(lhs, rhs);
+#endif
+}
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Initialization functions.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef PG_THREADS_WIN32
+typedef INIT_ONCE pg_once_flag;
+#define PG_ONCE_FLAG_INIT INIT_ONCE_STATIC_INIT
+#else
+typedef pthread_once_t pg_once_flag;
+#define PG_ONCE_FLAG_INIT PTHREAD_ONCE_INIT
+#endif
+
+typedef void (*pg_call_once_function_t) (void);
+
+#ifdef PG_THREADS_WIN32
+extern BOOL CALLBACK pg_call_once_trampoline(pg_once_flag *flag,
+ void *parameter,
+ void **context);
+#endif
+
+static inline void
+pg_call_once(pg_once_flag *flag, pg_call_once_function_t function)
+{
+#ifdef PG_THREADS_WIN32
+ InitOnceExecuteOnce(flag, pg_call_once_trampoline, (void *) function, NULL);
+#else
+ pthread_once(flag, function);
+#endif
+}
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Thread-specific storage. This mechanism is an alternative to using
+ * the pg_thread_local storage class, which should be preferred where
+ * possible. The only advantage is that the TSS interface allows a
+ * destructor functions to be run for non-NULL values when each thread
+ * exits.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef PG_THREADS_WIN32
+typedef DWORD pg_tss_t;
+#else
+typedef pthread_key_t pg_tss_t;
+#endif
+
+typedef void (*pg_tss_dtor_t) (void *);
+
+/*
+ * How long before we give up trying to call all the registered
+ * destructors, if the destructors themselves are calling pg_tss_set()
+ * to befuddle us by storing new non-NULL values?
+ */
+#ifdef PG_THREADS_NEED_DESTRUCTOR_TABLE
+#define PG_TSS_DTOR_ITERATIONS 8
+#else
+#define PG_TSS_DTOR_ITERATIONS PTHREAD_DESTRUCTOR_ITERATIONS
+#endif
+
+extern int pg_tss_create(pg_tss_t *tss_id, pg_tss_dtor_t destructor);
+extern void pg_tss_dtor_delete(pg_tss_t tss_id);
+#ifdef PG_THREADS_NEED_DESTRUCTOR_TABLE
+extern void pg_tss_ensure_destructors_in_this_thread(void);
+#endif
+
+static inline void *
+pg_tss_get(pg_tss_t key)
+{
+#ifdef PG_THREADS_WIN32
+ return TlsGetValue(key);
+#else
+ return pthread_getspecific(key);
+#endif
+}
+
+static inline int
+pg_tss_set(pg_tss_t tss_id, void *value)
+{
+#ifdef PG_THREADS_NEED_DESTRUCTOR_TABLE
+ if (value)
+ pg_tss_ensure_destructors_in_this_thread();
+#endif
+
+#ifdef PG_THREADS_WIN32
+ return pg_thrd_maperror(TlsSetValue(tss_id, value));
+#else
+ return pg_thrd_maperror(pthread_setspecific(tss_id, value));
+#endif
+}
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Read/write locks. Not in C11.
+ *
+ * Unfortunately Windows makes you say whether you're unlocking a read lock or
+ * a write lock, so we have to expose that here too. POSIX already knows.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef PG_THREADS_WIN32
+typedef SRWLOCK pg_rwlock_t;
+#define PG_RWLOCK_STATIC_INIT SRWLOCK_INIT
+#else
+typedef pthread_rwlock_t pg_rwlock_t;
+#define PG_RWLOCK_STATIC_INIT PTHREAD_RWLOCK_INITIALIZER
+#endif
+
+static inline int
+pg_rwlock_init(pg_rwlock_t * lock)
+{
+#ifdef PG_THREADS_WIN32
+ InitializeSRWLock(lock);
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_rwlock_init(lock, NULL));
+#endif
+}
+
+static inline int
+pg_rwlock_rdlock(pg_rwlock_t * lock)
+{
+#ifdef PG_THREADS_WIN32
+ AcquireSRWLockShared(lock);
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_rwlock_rdlock(lock));
+#endif
+}
+
+static inline int
+pg_rwlock_wrlock(pg_rwlock_t * lock)
+{
+#ifdef PG_THREADS_WIN32
+ AcquireSRWLockExclusive(lock);
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_rwlock_wrlock(lock));
+#endif
+}
+
+static inline int
+pg_wrlock_unlock(pg_rwlock_t * lock)
+{
+#ifdef PG_THREADS_WIN32
+ ReleaseSRWLockExclusive(lock);
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_rwlock_unlock(lock));
+#endif
+}
+
+static inline int
+pg_rdlock_unlock(pg_rwlock_t * lock)
+{
+#ifdef PG_THREADS_WIN32
+ ReleaseSRWLockShared(lock);
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_rwlock_unlock(lock));
+#endif
+}
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Simple mutexes.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef PG_THREADS_WIN32
+/*
+ * CRITICAL_SECTION might be the most obvious Windows mechanism for
+ * pg_mtx_t, but SRWLock is reported to be at least as fast when used
+ * only in exclusive mode, and has the advantage of a static
+ * initializer (CRITICAL_SECTION must be initialized and destroyed
+ * explicitly because it allocates resources other than the space it
+ * occupies.) C11 doesn't define a static initializer (possibly
+ * because CRITICAL_SECTION doesn't?), but we want one anyway. So
+ * we'll just point pg_mtx_t to pg_rwlock_t.
+ */
+typedef pg_rwlock_t pg_mtx_t;
+#define PG_MTX_STATIC_INIT PG_RWLOCK_STATIC_INIT
+#else
+typedef pthread_mutex_t pg_mtx_t;
+#define PG_MTX_STATIC_INIT PTHREAD_MUTEX_INITIALIZER
+#endif
+
+typedef enum pg_mtx_type_t
+{
+ pg_mtx_plain = 0
+} pg_mtx_type_t;
+
+
+static inline int
+pg_mtx_init(pg_mtx_t *mutex, int type)
+{
+#ifdef PG_THREADS_WIN32
+ return pg_rwlock_init(mutex);
+#else
+ return pg_thrd_maperror(pthread_mutex_init(mutex, NULL));
+#endif
+}
+
+static inline int
+pg_mtx_lock(pg_mtx_t *mutex)
+{
+#ifdef PG_THREADS_WIN32
+ return pg_rwlock_wrlock(mutex);
+#else
+ return pg_thrd_maperror(pthread_mutex_lock(mutex));
+#endif
+}
+
+static inline int
+pg_mtx_unlock(pg_mtx_t *mutex)
+{
+#ifdef PG_THREADS_WIN32
+ return pg_wrlock_unlock(mutex);
+#else
+ return pg_thrd_maperror(pthread_mutex_unlock(mutex));
+#endif
+}
+
+static inline int
+pg_mtx_destroy(pg_mtx_t *mutex)
+{
+#ifdef PG_THREADS_WIN32
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_mutex_destroy(mutex));
+#endif
+}
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Condition variables.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef PG_THREADS_WIN32
+typedef CONDITION_VARIABLE pg_cnd_t;
+#else
+typedef pthread_cond_t pg_cnd_t;
+#endif
+
+static inline int
+pg_cnd_init(pg_cnd_t *condvar)
+{
+#ifdef PG_THREADS_WIN32
+ InitializeConditionVariable(condvar);
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_cond_init(condvar, NULL));
+#endif
+}
+
+static inline int
+pg_cnd_broadcast(pg_cnd_t *condvar)
+{
+#ifdef PG_THREADS_WIN32
+ WakeAllConditionVariable(condvar);
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_cond_broadcast(condvar));
+#endif
+}
+
+static inline int
+pg_cnd_wait(pg_cnd_t *condvar, pg_mtx_t *mutex)
+{
+#ifdef PG_THREADS_WIN32
+ SleepConditionVariableSRW(condvar, mutex, INFINITE, 0);
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_cond_wait(condvar, mutex));
+#endif
+}
+
+static inline int
+pg_cnd_destroy(pg_cnd_t *condvar)
+{
+#ifdef PG_THREADS_WIN32
+ return pg_thrd_success;
+#else
+ return pg_thrd_maperror(pthread_cond_destroy(condvar));
+#endif
+}
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Barriers. Not in C11. Apple currently lacks the POSIX version.
+ * We assume that the OS might know a better way to implement it that
+ * we do, so we only provide our own if we have to.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef PG_THREADS_WIN32
+typedef SYNCHRONIZATION_BARRIER pg_barrier_t;
+#elif defined(HAVE_PTHREAD_BARRIER)
+typedef pthread_barrier_t pg_barrier_t;
+#else
+typedef struct pg_barrier_t
+{
+ bool sense;
+ int expected;
+ int arrived;
+ pg_mtx_t mutex;
+ pg_cnd_t cond;
+} pg_barrier_t;
+#endif
+
+static inline int
+pg_barrier_init(pg_barrier_t *barrier, int count)
+{
+#ifdef PG_THREADS_WIN32
+ return pg_thrd_maperror(InitializeSynchronizationBarrier(barrier, count, 0));
+#elif defined(HAVE_PTHREAD_BARRIER)
+ return pg_thrd_maperror(pthread_barrier_init(barrier, NULL, count));
+#else
+ barrier->sense = false;
+ barrier->expected = count;
+ barrier->arrived = 0;
+ if (pg_cnd_init(&barrier->cond) != pg_thrd_success)
+ return pg_thrd_error;
+ if (pg_mtx_init(&barrier->mutex, pg_mtx_plain) != pg_thrd_success)
+ {
+ pg_cnd_destroy(&barrier->cond);
+ return pg_thrd_error;
+ }
+ return pg_thrd_success;
+#endif
+}
+
+static inline int
+pg_barrier_wait(pg_barrier_t *barrier)
+{
+#ifdef PG_THREADS_WIN32
+ if (EnterSynchronizationBarrier(barrier, SYNCHRONIZATION_BARRIER_FLAGS_BLOCK_ONLY))
+ return pg_thrd_success_last;
+ else
+ return pg_thrd_success;
+#elif defined(HAVE_PTHREAD_BARRIER)
+ int error = pthread_barrier_wait(barrier);
+
+ if (error == 0)
+ return pg_thrd_success;
+ else if (error == PTHREAD_BARRIER_SERIAL_THREAD)
+ return pg_thrd_success_last;
+ else
+ return pg_thrd_error;
+#else
+ bool initial_sense;
+
+ pg_mtx_lock(&barrier->mutex);
+ barrier->arrived++;
+ if (barrier->arrived == barrier->expected)
+ {
+ barrier->arrived = 0;
+ barrier->sense = !barrier->sense;
+ pg_mtx_unlock(&barrier->mutex);
+ pg_cnd_broadcast(&barrier->cond);
+ return pg_thrd_success_last;
+ }
+ initial_sense = barrier->sense;
+ do
+ {
+ pg_cnd_wait(&barrier->cond, &barrier->mutex);
+ } while (barrier->sense == initial_sense);
+ pg_mtx_unlock(&barrier->mutex);
+ return pg_thrd_success;
+#endif
+}
+
+static inline int
+pg_barrier_destroy(pg_barrier_t *barrier)
+{
+#ifdef PG_THREADS_WIN32
+ return pg_thrd_success;
+#elif defined(HAVE_PTHREAD_BARRIER)
+ return pg_thrd_maperror(pthread_barrier_destroy(barrier));
+#else
+ pg_mtx_destroy(&barrier->mutex);
+ pg_cnd_destroy(&barrier->cond);
+ return pg_thrd_success;
+#endif
+}
+
+#endif
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 8afb1f0a26..073e069a21 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -2,8 +2,8 @@
#define POSTGRES_ECPG_INTERNAL
#include "postgres_fe.h"
+#include "port/pg_threads.h"
-#include "ecpg-pthread-win32.h"
#include "ecpgerrno.h"
#include "ecpglib.h"
#include "ecpglib_extern.h"
@@ -14,24 +14,11 @@
locale_t ecpg_clocale = (locale_t) 0;
#endif
-static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
-static pthread_key_t actual_connection_key;
-static pthread_once_t actual_connection_key_once = PTHREAD_ONCE_INIT;
+static pg_mtx_t connections_mutex = PG_MTX_STATIC_INIT;
+static pg_thread_local struct connection *actual_connection_this_thread;
static struct connection *actual_connection = NULL;
static struct connection *all_connections = NULL;
-static void
-ecpg_actual_connection_init(void)
-{
- pthread_key_create(&actual_connection_key, NULL);
-}
-
-void
-ecpg_pthreads_init(void)
-{
- pthread_once(&actual_connection_key_once, ecpg_actual_connection_init);
-}
-
static struct connection *
ecpg_get_connection_nr(const char *connection_name)
{
@@ -39,9 +26,7 @@ ecpg_get_connection_nr(const char *connection_name)
if ((connection_name == NULL) || (strcmp(connection_name, "CURRENT") == 0))
{
- ecpg_pthreads_init(); /* ensure actual_connection_key is valid */
-
- ret = pthread_getspecific(actual_connection_key);
+ ret = actual_connection_this_thread;
/*
* if no connection in TSD for this thread, get the global default
@@ -74,9 +59,7 @@ ecpg_get_connection(const char *connection_name)
if ((connection_name == NULL) || (strcmp(connection_name, "CURRENT") == 0))
{
- ecpg_pthreads_init(); /* ensure actual_connection_key is valid */
-
- ret = pthread_getspecific(actual_connection_key);
+ ret = actual_connection_this_thread;
/*
* if no connection in TSD for this thread, get the global default
@@ -89,11 +72,11 @@ ecpg_get_connection(const char *connection_name)
}
else
{
- pthread_mutex_lock(&connections_mutex);
+ pg_mtx_lock(&connections_mutex);
ret = ecpg_get_connection_nr(connection_name);
- pthread_mutex_unlock(&connections_mutex);
+ pg_mtx_unlock(&connections_mutex);
}
return ret;
@@ -127,8 +110,8 @@ ecpg_finish(struct connection *act)
con->next = act->next;
}
- if (pthread_getspecific(actual_connection_key) == act)
- pthread_setspecific(actual_connection_key, all_connections);
+ if (actual_connection_this_thread == act)
+ actual_connection_this_thread = all_connections;
if (actual_connection == act)
actual_connection = all_connections;
@@ -194,7 +177,7 @@ ECPGsetconn(int lineno, const char *connection_name)
if (!ecpg_init(con, connection_name, lineno))
return false;
- pthread_setspecific(actual_connection_key, con);
+ actual_connection_this_thread = con;
return true;
}
@@ -481,7 +464,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
}
/* add connection to our list */
- pthread_mutex_lock(&connections_mutex);
+ pg_mtx_lock(&connections_mutex);
/*
* ... but first, make certain we have created ecpg_clocale. Rely on
@@ -493,7 +476,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
if (!ecpg_clocale)
{
- pthread_mutex_unlock(&connections_mutex);
+ pg_mtx_unlock(&connections_mutex);
ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
if (host)
@@ -530,7 +513,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
this->next = all_connections;
all_connections = this;
- pthread_setspecific(actual_connection_key, all_connections);
+ actual_connection_this_thread = all_connections;
actual_connection = all_connections;
ecpg_log("ECPGconnect: opening database %s on %s port %s %s%s %s%s\n",
@@ -648,7 +631,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
ecpg_log("ECPGconnect: %s", errmsg);
ecpg_finish(this);
- pthread_mutex_unlock(&connections_mutex);
+ pg_mtx_unlock(&connections_mutex);
ecpg_raise(lineno, ECPG_CONNECT, ECPG_SQLSTATE_SQLCLIENT_UNABLE_TO_ESTABLISH_SQLCONNECTION, db);
if (realname)
@@ -660,7 +643,7 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
if (realname)
ecpg_free(realname);
- pthread_mutex_unlock(&connections_mutex);
+ pg_mtx_unlock(&connections_mutex);
this->autocommit = autocommit;
@@ -682,7 +665,7 @@ ECPGdisconnect(int lineno, const char *connection_name)
return false;
}
- pthread_mutex_lock(&connections_mutex);
+ pg_mtx_lock(&connections_mutex);
if (strcmp(connection_name, "ALL") == 0)
{
@@ -701,14 +684,14 @@ ECPGdisconnect(int lineno, const char *connection_name)
if (!ecpg_init(con, connection_name, lineno))
{
- pthread_mutex_unlock(&connections_mutex);
+ pg_mtx_unlock(&connections_mutex);
return false;
}
else
ecpg_finish(con);
}
- pthread_mutex_unlock(&connections_mutex);
+ pg_mtx_unlock(&connections_mutex);
return true;
}
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index ad279e245c..b57e065672 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -7,11 +7,11 @@
#include "postgres_fe.h"
#include "catalog/pg_type_d.h"
-#include "ecpg-pthread-win32.h"
#include "ecpgerrno.h"
#include "ecpglib.h"
#include "ecpglib_extern.h"
#include "ecpgtype.h"
+#include "port/pg_threads.h"
#include "sql3types.h"
#include "sqlca.h"
#include "sqlda.h"
@@ -19,8 +19,8 @@
static void descriptor_free(struct descriptor *desc);
/* We manage descriptors separately for each thread. */
-static pthread_key_t descriptor_key;
-static pthread_once_t descriptor_once = PTHREAD_ONCE_INIT;
+static pg_tss_t descriptor_key;
+static pg_once_flag descriptor_once = PG_ONCE_FLAG_INIT;
static void descriptor_deallocate_all(struct descriptor *list);
@@ -33,20 +33,20 @@ descriptor_destructor(void *arg)
static void
descriptor_key_init(void)
{
- pthread_key_create(&descriptor_key, descriptor_destructor);
+ pg_tss_create(&descriptor_key, descriptor_destructor);
}
static struct descriptor *
get_descriptors(void)
{
- pthread_once(&descriptor_once, descriptor_key_init);
- return (struct descriptor *) pthread_getspecific(descriptor_key);
+ pg_call_once(&descriptor_once, descriptor_key_init);
+ return (struct descriptor *) pg_tss_get(descriptor_key);
}
static void
set_descriptors(struct descriptor *value)
{
- pthread_setspecific(descriptor_key, value);
+ pg_tss_set(descriptor_key, value);
}
/* old internal convenience function that might go away later */
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 01b4309a71..d8416b19e3 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -169,7 +169,7 @@ bool ecpg_get_data(const PGresult *, int, int, int, enum ECPGttype type,
enum ECPGttype, char *, char *, long, long, long,
enum ARRAY_TYPE, enum COMPAT_MODE, bool);
-void ecpg_pthreads_init(void);
+#define ecpg_pthreads_init()
struct connection *ecpg_get_connection(const char *connection_name);
char *ecpg_alloc(long size, int lineno);
char *ecpg_auto_alloc(long size, int lineno);
diff --git a/src/interfaces/ecpg/ecpglib/memory.c b/src/interfaces/ecpg/ecpglib/memory.c
index a83637ac75..985f05087d 100644
--- a/src/interfaces/ecpg/ecpglib/memory.c
+++ b/src/interfaces/ecpg/ecpglib/memory.c
@@ -3,11 +3,11 @@
#define POSTGRES_ECPG_INTERNAL
#include "postgres_fe.h"
-#include "ecpg-pthread-win32.h"
#include "ecpgerrno.h"
#include "ecpglib.h"
#include "ecpglib_extern.h"
#include "ecpgtype.h"
+#include "port/pg_threads.h"
void
ecpg_free(void *ptr)
@@ -68,8 +68,8 @@ struct auto_mem
struct auto_mem *next;
};
-static pthread_key_t auto_mem_key;
-static pthread_once_t auto_mem_once = PTHREAD_ONCE_INIT;
+static pg_tss_t auto_mem_key;
+static pg_once_flag auto_mem_once = PG_ONCE_FLAG_INIT;
static void
auto_mem_destructor(void *arg)
@@ -81,20 +81,20 @@ auto_mem_destructor(void *arg)
static void
auto_mem_key_init(void)
{
- pthread_key_create(&auto_mem_key, auto_mem_destructor);
+ pg_tss_create(&auto_mem_key, auto_mem_destructor);
}
static struct auto_mem *
get_auto_allocs(void)
{
- pthread_once(&auto_mem_once, auto_mem_key_init);
- return (struct auto_mem *) pthread_getspecific(auto_mem_key);
+ pg_call_once(&auto_mem_once, auto_mem_key_init);
+ return (struct auto_mem *) pg_tss_get(auto_mem_key);
}
static void
set_auto_allocs(struct auto_mem *am)
{
- pthread_setspecific(auto_mem_key, am);
+ pg_tss_set(auto_mem_key, am);
}
char *
diff --git a/src/interfaces/ecpg/ecpglib/misc.c b/src/interfaces/ecpg/ecpglib/misc.c
index 2ae989e3e5..6fa50e9513 100644
--- a/src/interfaces/ecpg/ecpglib/misc.c
+++ b/src/interfaces/ecpg/ecpglib/misc.c
@@ -6,7 +6,6 @@
#include <limits.h>
#include <unistd.h>
-#include "ecpg-pthread-win32.h"
#include "ecpgerrno.h"
#include "ecpglib.h"
#include "ecpglib_extern.h"
@@ -16,6 +15,7 @@
#include "pgtypes_interval.h"
#include "pgtypes_numeric.h"
#include "pgtypes_timestamp.h"
+#include "port/pg_threads.h"
#include "sqlca.h"
#ifndef LONG_LONG_MIN
@@ -55,11 +55,10 @@ static struct sqlca_t sqlca_init =
}
};
-static pthread_key_t sqlca_key;
-static pthread_once_t sqlca_key_once = PTHREAD_ONCE_INIT;
-
-static pthread_mutex_t debug_mutex = PTHREAD_MUTEX_INITIALIZER;
-static pthread_mutex_t debug_init_mutex = PTHREAD_MUTEX_INITIALIZER;
+static pg_tss_t sqlca_key;
+static pg_once_flag ecpg_once = PG_ONCE_FLAG_INIT;
+static pg_mtx_t debug_mutex = PG_MTX_STATIC_INIT;
+static pg_mtx_t debug_init_mutex = PG_MTX_STATIC_INIT;
static volatile int simple_debug = 0;
static FILE *debugstream = NULL;
@@ -99,9 +98,9 @@ ecpg_sqlca_key_destructor(void *arg)
}
static void
-ecpg_sqlca_key_init(void)
+ecpg_init_once(void)
{
- pthread_key_create(&sqlca_key, ecpg_sqlca_key_destructor);
+ pg_tss_create(&sqlca_key, ecpg_sqlca_key_destructor);
}
struct sqlca_t *
@@ -109,16 +108,16 @@ ECPGget_sqlca(void)
{
struct sqlca_t *sqlca;
- pthread_once(&sqlca_key_once, ecpg_sqlca_key_init);
+ pg_call_once(&ecpg_once, ecpg_init_once);
- sqlca = pthread_getspecific(sqlca_key);
+ sqlca = pg_tss_get(sqlca_key);
if (sqlca == NULL)
{
sqlca = malloc(sizeof(struct sqlca_t));
if (sqlca == NULL)
return NULL;
ecpg_init_sqlca(sqlca);
- pthread_setspecific(sqlca_key, sqlca);
+ pg_tss_set(sqlca_key, sqlca);
}
return sqlca;
}
@@ -204,10 +203,10 @@ void
ECPGdebug(int n, FILE *dbgs)
{
/* Interlock against concurrent executions of ECPGdebug() */
- pthread_mutex_lock(&debug_init_mutex);
+ pg_mtx_lock(&debug_init_mutex);
/* Prevent ecpg_log() from printing while we change settings */
- pthread_mutex_lock(&debug_mutex);
+ pg_mtx_lock(&debug_mutex);
if (n > 100)
{
@@ -220,12 +219,12 @@ ECPGdebug(int n, FILE *dbgs)
debugstream = dbgs;
/* We must release debug_mutex before invoking ecpg_log() ... */
- pthread_mutex_unlock(&debug_mutex);
+ pg_mtx_unlock(&debug_mutex);
/* ... but keep holding debug_init_mutex to avoid racy printout */
ecpg_log("ECPGdebug: set to %d\n", simple_debug);
- pthread_mutex_unlock(&debug_init_mutex);
+ pg_mtx_unlock(&debug_init_mutex);
}
void
@@ -262,7 +261,7 @@ ecpg_log(const char *format,...)
else
snprintf(fmt, bufsize, "[%d]: %s", (int) getpid(), intl_format);
- pthread_mutex_lock(&debug_mutex);
+ pg_mtx_lock(&debug_mutex);
/* Now that we hold the mutex, recheck simple_debug */
if (simple_debug)
@@ -281,7 +280,7 @@ ecpg_log(const char *format,...)
fflush(debugstream);
}
- pthread_mutex_unlock(&debug_mutex);
+ pg_mtx_unlock(&debug_mutex);
free(fmt);
}
@@ -422,60 +421,6 @@ ECPGis_noind_null(enum ECPGttype type, const void *ptr)
return false;
}
-#ifdef WIN32
-
-int
-pthread_mutex_init(pthread_mutex_t *mp, void *attr)
-{
- mp->initstate = 0;
- return 0;
-}
-
-int
-pthread_mutex_lock(pthread_mutex_t *mp)
-{
- /* Initialize the csection if not already done */
- if (mp->initstate != 1)
- {
- LONG istate;
-
- while ((istate = InterlockedExchange(&mp->initstate, 2)) == 2)
- Sleep(0); /* wait, another thread is doing this */
- if (istate != 1)
- InitializeCriticalSection(&mp->csection);
- InterlockedExchange(&mp->initstate, 1);
- }
- EnterCriticalSection(&mp->csection);
- return 0;
-}
-
-int
-pthread_mutex_unlock(pthread_mutex_t *mp)
-{
- if (mp->initstate != 1)
- return EINVAL;
- LeaveCriticalSection(&mp->csection);
- return 0;
-}
-
-static pthread_mutex_t win32_pthread_once_lock = PTHREAD_MUTEX_INITIALIZER;
-
-void
-win32_pthread_once(volatile pthread_once_t *once, void (*fn) (void))
-{
- if (!*once)
- {
- pthread_mutex_lock(&win32_pthread_once_lock);
- if (!*once)
- {
- fn();
- *once = true;
- }
- pthread_mutex_unlock(&win32_pthread_once_lock);
- }
-}
-#endif /* WIN32 */
-
#ifdef ENABLE_NLS
char *
diff --git a/src/interfaces/ecpg/ecpglib/sqlda.c b/src/interfaces/ecpg/ecpglib/sqlda.c
index 081e32666f..7231cc4a6f 100644
--- a/src/interfaces/ecpg/ecpglib/sqlda.c
+++ b/src/interfaces/ecpg/ecpglib/sqlda.c
@@ -11,7 +11,6 @@
#include "catalog/pg_type_d.h"
#include "decimal.h"
-#include "ecpg-pthread-win32.h"
#include "ecpgerrno.h"
#include "ecpglib.h"
#include "ecpglib_extern.h"
diff --git a/src/interfaces/ecpg/include/ecpg-pthread-win32.h b/src/interfaces/ecpg/include/ecpg-pthread-win32.h
deleted file mode 100644
index 7b6ba46b34..0000000000
--- a/src/interfaces/ecpg/include/ecpg-pthread-win32.h
+++ /dev/null
@@ -1,49 +0,0 @@
-/* src/interfaces/ecpg/include/ecpg-pthread-win32.h */
-/*
- * pthread mapping macros for win32 native thread implementation
- */
-#ifndef _ECPG_PTHREAD_WIN32_H
-#define _ECPG_PTHREAD_WIN32_H
-
-#ifndef WIN32
-
-#include <pthread.h>
-#else
-
-typedef struct pthread_mutex_t
-{
- /* initstate = 0: not initialized; 1: init done; 2: init in progress */
- LONG initstate;
- CRITICAL_SECTION csection;
-} pthread_mutex_t;
-
-typedef DWORD pthread_key_t;
-typedef bool pthread_once_t;
-
-#define PTHREAD_MUTEX_INITIALIZER { 0 }
-#define PTHREAD_ONCE_INIT false
-
-int pthread_mutex_init(pthread_mutex_t *, void *attr);
-int pthread_mutex_lock(pthread_mutex_t *);
-int pthread_mutex_unlock(pthread_mutex_t *);
-
-void win32_pthread_once(volatile pthread_once_t *once, void (*fn) (void));
-
-#define pthread_getspecific(key) \
- TlsGetValue((key))
-
-#define pthread_setspecific(key, value) \
- TlsSetValue((key), (value))
-
-/* FIXME: destructor is never called in Win32. */
-#define pthread_key_create(key, destructor) \
- do { *(key) = TlsAlloc(); ((void)(destructor)); } while(0)
-
-#define pthread_once(once, fn) \
- do { \
- if (!*(once)) \
- win32_pthread_once((once), (fn)); \
- } while(0)
-#endif /* WIN32 */
-
-#endif /* _ECPG_PTHREAD_WIN32_H */
diff --git a/src/interfaces/ecpg/include/ecpg_config.h.in b/src/interfaces/ecpg/include/ecpg_config.h.in
index 824617b917..04629f937e 100644
--- a/src/interfaces/ecpg/include/ecpg_config.h.in
+++ b/src/interfaces/ecpg/include/ecpg_config.h.in
@@ -10,5 +10,8 @@
/* Define to 1 if `long long int' works and is 64 bits. */
#undef HAVE_LONG_LONG_INT_64
+/* Define to 1 if you have the `pthread_barrier_wait' function. */
+#undef HAVE_PTHREAD_BARRIER_WAIT
+
/* Define to 1 to use <stdbool.h> to define type bool. */
#undef PG_USE_STDBOOL
diff --git a/src/interfaces/ecpg/include/meson.build b/src/interfaces/ecpg/include/meson.build
index 31610fef58..a4023f3684 100644
--- a/src/interfaces/ecpg/include/meson.build
+++ b/src/interfaces/ecpg/include/meson.build
@@ -6,6 +6,7 @@ ecpg_conf_keys = [
'HAVE_INT64',
'HAVE_LONG_INT_64',
'HAVE_LONG_LONG_INT_64',
+ 'HAVE_PTHREAD_BARRIER_WAIT',
'PG_USE_STDBOOL',
]
diff --git a/src/interfaces/ecpg/test/expected/thread-alloc.c b/src/interfaces/ecpg/test/expected/thread-alloc.c
index 3b31d27fd3..97ed928ef1 100644
--- a/src/interfaces/ecpg/test/expected/thread-alloc.c
+++ b/src/interfaces/ecpg/test/expected/thread-alloc.c
@@ -10,16 +10,7 @@
#include <stdint.h>
#include <stdlib.h>
#include "ecpg_config.h"
-
-#ifdef WIN32
-#define WIN32_LEAN_AND_MEAN
-#include <windows.h>
-#include <process.h>
-#include <locale.h>
-#else
-#include <pthread.h>
-#endif
-#include <stdio.h>
+#include "port/pg_threads.h"
#define THREADS 16
#define REPEATS 50
@@ -93,7 +84,7 @@ struct sqlca_t *ECPGget_sqlca(void);
#endif
-#line 18 "alloc.pgc"
+#line 9 "alloc.pgc"
#line 1 "regression.h"
@@ -103,21 +94,17 @@ struct sqlca_t *ECPGget_sqlca(void);
-#line 19 "alloc.pgc"
+#line 10 "alloc.pgc"
/* exec sql whenever sqlerror sqlprint ; */
-#line 21 "alloc.pgc"
+#line 12 "alloc.pgc"
/* exec sql whenever not found sqlprint ; */
-#line 22 "alloc.pgc"
+#line 13 "alloc.pgc"
-#ifdef WIN32
-static unsigned __stdcall fn(void* arg)
-#else
-static void* fn(void* arg)
-#endif
+static int fn(void* arg)
{
int i;
@@ -126,54 +113,54 @@ static void* fn(void* arg)
-#line 33 "alloc.pgc"
+#line 20 "alloc.pgc"
int value ;
-#line 34 "alloc.pgc"
+#line 21 "alloc.pgc"
char name [ 100 ] ;
-#line 35 "alloc.pgc"
+#line 22 "alloc.pgc"
char ** r = NULL ;
/* exec sql end declare section */
-#line 36 "alloc.pgc"
+#line 23 "alloc.pgc"
value = (intptr_t) arg;
sprintf(name, "Connection: %d", value);
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , name, 0);
-#line 41 "alloc.pgc"
+#line 28 "alloc.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 41 "alloc.pgc"
+#line 28 "alloc.pgc"
{ ECPGsetcommit(__LINE__, "on", NULL);
-#line 42 "alloc.pgc"
+#line 29 "alloc.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 42 "alloc.pgc"
+#line 29 "alloc.pgc"
for (i = 1; i <= REPEATS; ++i)
{
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "select relname from pg_class where relname = 'pg_class'", ECPGt_EOIT,
ECPGt_char,&(r),(long)0,(long)0,(1)*sizeof(char),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);
-#line 45 "alloc.pgc"
+#line 32 "alloc.pgc"
if (sqlca.sqlcode == ECPG_NOT_FOUND) sqlprint();
-#line 45 "alloc.pgc"
+#line 32 "alloc.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 45 "alloc.pgc"
+#line 32 "alloc.pgc"
free(r);
r = NULL;
}
{ ECPGdisconnect(__LINE__, name);
-#line 49 "alloc.pgc"
+#line 36 "alloc.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 49 "alloc.pgc"
+#line 36 "alloc.pgc"
return 0;
@@ -182,28 +169,12 @@ if (sqlca.sqlcode < 0) sqlprint();}
int main ()
{
intptr_t i;
-#ifdef WIN32
- HANDLE threads[THREADS];
-#else
- pthread_t threads[THREADS];
-#endif
-
-#ifdef WIN32
- for (i = 0; i < THREADS; ++i)
- {
- unsigned id;
- threads[i] = (HANDLE)_beginthreadex(NULL, 0, fn, (void*)i, 0, &id);
- }
+ pg_thrd_t threads[THREADS];
- WaitForMultipleObjects(THREADS, threads, TRUE, INFINITE);
for (i = 0; i < THREADS; ++i)
- CloseHandle(threads[i]);
-#else
+ pg_thrd_create(&threads[i], fn, (void *) i);
for (i = 0; i < THREADS; ++i)
- pthread_create(&threads[i], NULL, fn, (void *) i);
- for (i = 0; i < THREADS; ++i)
- pthread_join(threads[i], NULL);
-#endif
+ pg_thrd_join(threads[i], NULL);
return 0;
}
diff --git a/src/interfaces/ecpg/test/expected/thread-descriptor.c b/src/interfaces/ecpg/test/expected/thread-descriptor.c
index e34f4708d1..50cdbf1b2f 100644
--- a/src/interfaces/ecpg/test/expected/thread-descriptor.c
+++ b/src/interfaces/ecpg/test/expected/thread-descriptor.c
@@ -7,15 +7,7 @@
#define ECPGdebug(X,Y) ECPGdebug((X)+100,(Y))
#line 1 "descriptor.pgc"
-#ifdef WIN32
-#define WIN32_LEAN_AND_MEAN
-#include <windows.h>
-#include <process.h>
-#include <locale.h>
-#else
-#include <pthread.h>
-#endif
-#include <stdio.h>
+#include "port/pg_threads.h"
#define THREADS 16
#define REPEATS 50000
@@ -89,36 +81,32 @@ struct sqlca_t *ECPGget_sqlca(void);
#endif
-#line 14 "descriptor.pgc"
+#line 6 "descriptor.pgc"
/* exec sql whenever sqlerror sqlprint ; */
-#line 15 "descriptor.pgc"
+#line 7 "descriptor.pgc"
/* exec sql whenever not found sqlprint ; */
-#line 16 "descriptor.pgc"
+#line 8 "descriptor.pgc"
-#if defined(WIN32)
-static unsigned __stdcall fn(void* arg)
-#else
-static void* fn(void* arg)
-#endif
+static int fn(void* arg)
{
int i;
for (i = 1; i <= REPEATS; ++i)
{
ECPGallocate_desc(__LINE__, "mydesc");
-#line 28 "descriptor.pgc"
+#line 16 "descriptor.pgc"
if (sqlca.sqlcode < 0) sqlprint();
-#line 28 "descriptor.pgc"
+#line 16 "descriptor.pgc"
ECPGdeallocate_desc(__LINE__, "mydesc");
-#line 29 "descriptor.pgc"
+#line 17 "descriptor.pgc"
if (sqlca.sqlcode < 0) sqlprint();
-#line 29 "descriptor.pgc"
+#line 17 "descriptor.pgc"
}
@@ -128,28 +116,12 @@ if (sqlca.sqlcode < 0) sqlprint();
int main ()
{
int i;
-#ifdef WIN32
- HANDLE threads[THREADS];
-#else
- pthread_t threads[THREADS];
-#endif
-
-#ifdef WIN32
- for (i = 0; i < THREADS; ++i)
- {
- unsigned id;
- threads[i] = (HANDLE)_beginthreadex(NULL, 0, fn, NULL, 0, &id);
- }
+ pg_thrd_t threads[THREADS];
- WaitForMultipleObjects(THREADS, threads, TRUE, INFINITE);
- for (i = 0; i < THREADS; ++i)
- CloseHandle(threads[i]);
-#else
for (i = 0; i < THREADS; ++i)
- pthread_create(&threads[i], NULL, fn, NULL);
+ pg_thrd_create(&threads[i], fn, NULL);
for (i = 0; i < THREADS; ++i)
- pthread_join(threads[i], NULL);
-#endif
+ pg_thrd_join(threads[i], NULL);
return 0;
}
diff --git a/src/interfaces/ecpg/test/expected/thread-prep.c b/src/interfaces/ecpg/test/expected/thread-prep.c
index 052e27b634..c5b60d18cb 100644
--- a/src/interfaces/ecpg/test/expected/thread-prep.c
+++ b/src/interfaces/ecpg/test/expected/thread-prep.c
@@ -10,15 +10,8 @@
#include <stdint.h>
#include <stdlib.h>
#include "ecpg_config.h"
+#include "port/pg_threads.h"
-#ifdef WIN32
-#define WIN32_LEAN_AND_MEAN
-#include <windows.h>
-#include <process.h>
-#include <locale.h>
-#else
-#include <pthread.h>
-#endif
#include <stdio.h>
#define THREADS 16
@@ -93,7 +86,7 @@ struct sqlca_t *ECPGget_sqlca(void);
#endif
-#line 18 "prep.pgc"
+#line 11 "prep.pgc"
#line 1 "regression.h"
@@ -103,21 +96,17 @@ struct sqlca_t *ECPGget_sqlca(void);
-#line 19 "prep.pgc"
+#line 12 "prep.pgc"
/* exec sql whenever sqlerror sqlprint ; */
-#line 21 "prep.pgc"
+#line 14 "prep.pgc"
/* exec sql whenever not found sqlprint ; */
-#line 22 "prep.pgc"
+#line 15 "prep.pgc"
-#ifdef WIN32
-static unsigned __stdcall fn(void* arg)
-#else
-static void* fn(void* arg)
-#endif
+static int fn(void* arg)
{
int i;
@@ -126,64 +115,64 @@ static void* fn(void* arg)
-#line 33 "prep.pgc"
+#line 22 "prep.pgc"
int value ;
-#line 34 "prep.pgc"
+#line 23 "prep.pgc"
char name [ 100 ] ;
-#line 35 "prep.pgc"
+#line 24 "prep.pgc"
char query [ 256 ] = "INSERT INTO T VALUES ( ? )" ;
/* exec sql end declare section */
-#line 36 "prep.pgc"
+#line 25 "prep.pgc"
value = (intptr_t) arg;
sprintf(name, "Connection: %d", value);
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , name, 0);
-#line 41 "prep.pgc"
+#line 30 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 41 "prep.pgc"
+#line 30 "prep.pgc"
{ ECPGsetcommit(__LINE__, "on", NULL);
-#line 42 "prep.pgc"
+#line 31 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 42 "prep.pgc"
+#line 31 "prep.pgc"
for (i = 1; i <= REPEATS; ++i)
{
{ ECPGprepare(__LINE__, NULL, 0, "i", query);
-#line 45 "prep.pgc"
+#line 34 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 45 "prep.pgc"
+#line 34 "prep.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_execute, "i",
ECPGt_int,&(value),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EOIT, ECPGt_EORT);
-#line 46 "prep.pgc"
+#line 35 "prep.pgc"
if (sqlca.sqlcode == ECPG_NOT_FOUND) sqlprint();
-#line 46 "prep.pgc"
+#line 35 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 46 "prep.pgc"
+#line 35 "prep.pgc"
}
{ ECPGdeallocate(__LINE__, 0, NULL, "i");
-#line 48 "prep.pgc"
+#line 37 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 48 "prep.pgc"
+#line 37 "prep.pgc"
{ ECPGdisconnect(__LINE__, name);
-#line 49 "prep.pgc"
+#line 38 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 49 "prep.pgc"
+#line 38 "prep.pgc"
return 0;
@@ -192,59 +181,43 @@ if (sqlca.sqlcode < 0) sqlprint();}
int main ()
{
intptr_t i;
-#ifdef WIN32
- HANDLE threads[THREADS];
-#else
- pthread_t threads[THREADS];
-#endif
+ pg_thrd_t threads[THREADS];
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0);
-#line 63 "prep.pgc"
+#line 48 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 63 "prep.pgc"
+#line 48 "prep.pgc"
{ ECPGsetcommit(__LINE__, "on", NULL);
-#line 64 "prep.pgc"
+#line 49 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 64 "prep.pgc"
+#line 49 "prep.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "drop table if exists T", ECPGt_EOIT, ECPGt_EORT);
-#line 65 "prep.pgc"
+#line 50 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 65 "prep.pgc"
+#line 50 "prep.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "create table T ( i int )", ECPGt_EOIT, ECPGt_EORT);
-#line 66 "prep.pgc"
+#line 51 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 66 "prep.pgc"
+#line 51 "prep.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");
-#line 67 "prep.pgc"
+#line 52 "prep.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 67 "prep.pgc"
+#line 52 "prep.pgc"
-#ifdef WIN32
for (i = 0; i < THREADS; ++i)
- {
- unsigned id;
- threads[i] = (HANDLE)_beginthreadex(NULL, 0, fn, (void*)i, 0, &id);
- }
-
- WaitForMultipleObjects(THREADS, threads, TRUE, INFINITE);
+ pg_thrd_create(&threads[i], fn, (void *) i);
for (i = 0; i < THREADS; ++i)
- CloseHandle(threads[i]);
-#else
- for (i = 0; i < THREADS; ++i)
- pthread_create(&threads[i], NULL, fn, (void *) i);
- for (i = 0; i < THREADS; ++i)
- pthread_join(threads[i], NULL);
-#endif
+ pg_thrd_join(threads[i], NULL);
return 0;
}
diff --git a/src/interfaces/ecpg/test/expected/thread-thread.c b/src/interfaces/ecpg/test/expected/thread-thread.c
index 95faa223c2..03c7169995 100644
--- a/src/interfaces/ecpg/test/expected/thread-thread.c
+++ b/src/interfaces/ecpg/test/expected/thread-thread.c
@@ -14,13 +14,7 @@
#include <stdint.h>
#include <stdlib.h>
#include "ecpg_config.h"
-
-#ifndef WIN32
-#include <pthread.h>
-#else
-#include <windows.h>
-#include <locale.h>
-#endif
+#include "port/pg_threads.h"
#line 1 "regression.h"
@@ -30,29 +24,25 @@
-#line 16 "thread.pgc"
+#line 10 "thread.pgc"
-void *test_thread(void *arg);
+int test_thread(void *arg);
int nthreads = 10;
int iterations = 20;
int main()
{
-#ifndef WIN32
- pthread_t *threads;
-#else
- HANDLE *threads;
-#endif
+ pg_thrd_t *threads;
intptr_t n;
/* exec sql begin declare section */
-#line 32 "thread.pgc"
+#line 22 "thread.pgc"
int l_rows ;
/* exec sql end declare section */
-#line 33 "thread.pgc"
+#line 23 "thread.pgc"
/* Do not switch on debug output for regression tests. The threads get executed in
@@ -61,22 +51,22 @@ int main()
/* setup test_thread table */
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0); }
-#line 40 "thread.pgc"
+#line 30 "thread.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "drop table test_thread", ECPGt_EOIT, ECPGt_EORT);}
-#line 41 "thread.pgc"
+#line 31 "thread.pgc"
/* DROP might fail */
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 42 "thread.pgc"
+#line 32 "thread.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "create table test_thread ( tstamp timestamp not null default cast ( timeofday ( ) as timestamp ) , thread text not null , iteration integer not null , primary key ( thread , iteration ) )", ECPGt_EOIT, ECPGt_EORT);}
-#line 47 "thread.pgc"
+#line 37 "thread.pgc"
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 48 "thread.pgc"
+#line 38 "thread.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");}
-#line 49 "thread.pgc"
+#line 39 "thread.pgc"
/* create, and start, threads */
@@ -87,39 +77,27 @@ int main()
return 1;
}
for( n = 0; n < nthreads; n++ )
- {
-#ifndef WIN32
- pthread_create(&threads[n], NULL, test_thread, (void *) (n + 1));
-#else
- threads[n] = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE) (void (*) (void)) test_thread, (void *) (n + 1), 0, NULL);
-#endif
- }
+ pg_thrd_create(&threads[n], test_thread, (void *) (n + 1));
/* wait for thread completion */
-#ifndef WIN32
for( n = 0; n < nthreads; n++ )
- {
- pthread_join(threads[n], NULL);
- }
-#else
- WaitForMultipleObjects(nthreads, threads, TRUE, INFINITE);
-#endif
+ pg_thrd_join(threads[n], NULL);
free(threads);
/* and check results */
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0); }
-#line 79 "thread.pgc"
+#line 57 "thread.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "select count ( * ) from test_thread", ECPGt_EOIT,
ECPGt_int,&(l_rows),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);}
-#line 80 "thread.pgc"
+#line 58 "thread.pgc"
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 81 "thread.pgc"
+#line 59 "thread.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");}
-#line 82 "thread.pgc"
+#line 60 "thread.pgc"
if( l_rows == (nthreads * iterations) )
printf("Success.\n");
@@ -129,7 +107,7 @@ int main()
return 0;
}
-void *test_thread(void *arg)
+int test_thread(void *arg)
{
long threadnum = (intptr_t) arg;
@@ -137,13 +115,13 @@ void *test_thread(void *arg)
-#line 96 "thread.pgc"
+#line 74 "thread.pgc"
int l_i ;
-#line 97 "thread.pgc"
+#line 75 "thread.pgc"
char l_connection [ 128 ] ;
/* exec sql end declare section */
-#line 98 "thread.pgc"
+#line 76 "thread.pgc"
/* build up connection name, and connect to database */
@@ -153,24 +131,24 @@ void *test_thread(void *arg)
_snprintf(l_connection, sizeof(l_connection), "thread_%03ld", threadnum);
#endif
/* exec sql whenever sqlerror sqlprint ; */
-#line 106 "thread.pgc"
+#line 84 "thread.pgc"
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , l_connection, 0);
-#line 107 "thread.pgc"
+#line 85 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 107 "thread.pgc"
+#line 85 "thread.pgc"
if( sqlca.sqlcode != 0 )
{
printf("%s: ERROR: cannot connect to database!\n", l_connection);
- return NULL;
+ return 0;
}
{ ECPGtrans(__LINE__, l_connection, "begin");
-#line 113 "thread.pgc"
+#line 91 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 113 "thread.pgc"
+#line 91 "thread.pgc"
/* insert into test_thread table */
@@ -181,10 +159,10 @@ if (sqlca.sqlcode < 0) sqlprint();}
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_int,&(l_i),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EOIT, ECPGt_EORT);
-#line 118 "thread.pgc"
+#line 96 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 118 "thread.pgc"
+#line 96 "thread.pgc"
if( sqlca.sqlcode != 0 )
printf("%s: ERROR: insert failed!\n", l_connection);
@@ -192,16 +170,16 @@ if (sqlca.sqlcode < 0) sqlprint();}
/* all done */
{ ECPGtrans(__LINE__, l_connection, "commit");
-#line 124 "thread.pgc"
+#line 102 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 124 "thread.pgc"
+#line 102 "thread.pgc"
{ ECPGdisconnect(__LINE__, l_connection);
-#line 125 "thread.pgc"
+#line 103 "thread.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 125 "thread.pgc"
+#line 103 "thread.pgc"
- return NULL;
+ return 0;
}
diff --git a/src/interfaces/ecpg/test/expected/thread-thread_implicit.c b/src/interfaces/ecpg/test/expected/thread-thread_implicit.c
index 7ac0297a23..c859012f50 100644
--- a/src/interfaces/ecpg/test/expected/thread-thread_implicit.c
+++ b/src/interfaces/ecpg/test/expected/thread-thread_implicit.c
@@ -14,13 +14,7 @@
#include <stdint.h>
#include <stdlib.h>
#include "ecpg_config.h"
-
-#ifndef WIN32
-#include <pthread.h>
-#else
-#include <windows.h>
-#include <locale.h>
-#endif
+#include "port/pg_threads.h"
#line 1 "regression.h"
@@ -30,29 +24,25 @@
-#line 16 "thread_implicit.pgc"
+#line 10 "thread_implicit.pgc"
-void *test_thread(void *arg);
+int test_thread(void *arg);
int nthreads = 10;
int iterations = 20;
int main()
{
-#ifndef WIN32
- pthread_t *threads;
-#else
- HANDLE *threads;
-#endif
+ pg_thrd_t *threads;
intptr_t n;
/* exec sql begin declare section */
-#line 32 "thread_implicit.pgc"
+#line 22 "thread_implicit.pgc"
int l_rows ;
/* exec sql end declare section */
-#line 33 "thread_implicit.pgc"
+#line 23 "thread_implicit.pgc"
/* Do not switch on debug output for regression tests. The threads get executed in
@@ -61,22 +51,22 @@ int main()
/* setup test_thread table */
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0); }
-#line 40 "thread_implicit.pgc"
+#line 30 "thread_implicit.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "drop table test_thread", ECPGt_EOIT, ECPGt_EORT);}
-#line 41 "thread_implicit.pgc"
+#line 31 "thread_implicit.pgc"
/* DROP might fail */
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 42 "thread_implicit.pgc"
+#line 32 "thread_implicit.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "create table test_thread ( tstamp timestamp not null default cast ( timeofday ( ) as timestamp ) , thread text not null , iteration integer not null , primary key ( thread , iteration ) )", ECPGt_EOIT, ECPGt_EORT);}
-#line 47 "thread_implicit.pgc"
+#line 37 "thread_implicit.pgc"
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 48 "thread_implicit.pgc"
+#line 38 "thread_implicit.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");}
-#line 49 "thread_implicit.pgc"
+#line 39 "thread_implicit.pgc"
/* create, and start, threads */
@@ -87,39 +77,27 @@ int main()
return 1;
}
for( n = 0; n < nthreads; n++ )
- {
-#ifndef WIN32
- pthread_create(&threads[n], NULL, test_thread, (void *) (n + 1));
-#else
- threads[n] = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE) (void (*) (void)) test_thread, (void *) (n+1), 0, NULL);
-#endif
- }
+ pg_thrd_create(&threads[n], test_thread, (void *) (n + 1));
/* wait for thread completion */
-#ifndef WIN32
for( n = 0; n < nthreads; n++ )
- {
- pthread_join(threads[n], NULL);
- }
-#else
- WaitForMultipleObjects(nthreads, threads, TRUE, INFINITE);
-#endif
+ pg_thrd_join(threads[n], NULL);
free(threads);
/* and check results */
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , NULL, 0); }
-#line 79 "thread_implicit.pgc"
+#line 57 "thread_implicit.pgc"
{ ECPGdo(__LINE__, 0, 1, NULL, 0, ECPGst_normal, "select count ( * ) from test_thread", ECPGt_EOIT,
ECPGt_int,&(l_rows),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EORT);}
-#line 80 "thread_implicit.pgc"
+#line 58 "thread_implicit.pgc"
{ ECPGtrans(__LINE__, NULL, "commit");}
-#line 81 "thread_implicit.pgc"
+#line 59 "thread_implicit.pgc"
{ ECPGdisconnect(__LINE__, "CURRENT");}
-#line 82 "thread_implicit.pgc"
+#line 60 "thread_implicit.pgc"
if( l_rows == (nthreads * iterations) )
printf("Success.\n");
@@ -129,7 +107,7 @@ int main()
return 0;
}
-void *test_thread(void *arg)
+int test_thread(void *arg)
{
long threadnum = (intptr_t) arg;
@@ -137,13 +115,13 @@ void *test_thread(void *arg)
-#line 96 "thread_implicit.pgc"
+#line 74 "thread_implicit.pgc"
int l_i ;
-#line 97 "thread_implicit.pgc"
+#line 75 "thread_implicit.pgc"
char l_connection [ 128 ] ;
/* exec sql end declare section */
-#line 98 "thread_implicit.pgc"
+#line 76 "thread_implicit.pgc"
/* build up connection name, and connect to database */
@@ -153,24 +131,24 @@ void *test_thread(void *arg)
_snprintf(l_connection, sizeof(l_connection), "thread_%03ld", threadnum);
#endif
/* exec sql whenever sqlerror sqlprint ; */
-#line 106 "thread_implicit.pgc"
+#line 84 "thread_implicit.pgc"
{ ECPGconnect(__LINE__, 0, "ecpg1_regression" , NULL, NULL , l_connection, 0);
-#line 107 "thread_implicit.pgc"
+#line 85 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 107 "thread_implicit.pgc"
+#line 85 "thread_implicit.pgc"
if( sqlca.sqlcode != 0 )
{
printf("%s: ERROR: cannot connect to database!\n", l_connection);
- return NULL;
+ return 0;
}
{ ECPGtrans(__LINE__, NULL, "begin");
-#line 113 "thread_implicit.pgc"
+#line 91 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 113 "thread_implicit.pgc"
+#line 91 "thread_implicit.pgc"
/* insert into test_thread table */
@@ -181,10 +159,10 @@ if (sqlca.sqlcode < 0) sqlprint();}
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L,
ECPGt_int,&(l_i),(long)1,(long)1,sizeof(int),
ECPGt_NO_INDICATOR, NULL , 0L, 0L, 0L, ECPGt_EOIT, ECPGt_EORT);
-#line 118 "thread_implicit.pgc"
+#line 96 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 118 "thread_implicit.pgc"
+#line 96 "thread_implicit.pgc"
if( sqlca.sqlcode != 0 )
printf("%s: ERROR: insert failed!\n", l_connection);
@@ -192,16 +170,16 @@ if (sqlca.sqlcode < 0) sqlprint();}
/* all done */
{ ECPGtrans(__LINE__, NULL, "commit");
-#line 124 "thread_implicit.pgc"
+#line 102 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 124 "thread_implicit.pgc"
+#line 102 "thread_implicit.pgc"
{ ECPGdisconnect(__LINE__, l_connection);
-#line 125 "thread_implicit.pgc"
+#line 103 "thread_implicit.pgc"
if (sqlca.sqlcode < 0) sqlprint();}
-#line 125 "thread_implicit.pgc"
+#line 103 "thread_implicit.pgc"
- return NULL;
+ return 0;
}
diff --git a/src/interfaces/ecpg/test/thread/alloc.pgc b/src/interfaces/ecpg/test/thread/alloc.pgc
index d3d35493bf..248a29447c 100644
--- a/src/interfaces/ecpg/test/thread/alloc.pgc
+++ b/src/interfaces/ecpg/test/thread/alloc.pgc
@@ -1,16 +1,7 @@
#include <stdint.h>
#include <stdlib.h>
#include "ecpg_config.h"
-
-#ifdef WIN32
-#define WIN32_LEAN_AND_MEAN
-#include <windows.h>
-#include <process.h>
-#include <locale.h>
-#else
-#include <pthread.h>
-#endif
-#include <stdio.h>
+#include "port/pg_threads.h"
#define THREADS 16
#define REPEATS 50
@@ -21,11 +12,7 @@ exec sql include ../regression;
exec sql whenever sqlerror sqlprint;
exec sql whenever not found sqlprint;
-#ifdef WIN32
-static unsigned __stdcall fn(void* arg)
-#else
-static void* fn(void* arg)
-#endif
+static int fn(void* arg)
{
int i;
@@ -54,28 +41,12 @@ static void* fn(void* arg)
int main ()
{
intptr_t i;
-#ifdef WIN32
- HANDLE threads[THREADS];
-#else
- pthread_t threads[THREADS];
-#endif
-
-#ifdef WIN32
- for (i = 0; i < THREADS; ++i)
- {
- unsigned id;
- threads[i] = (HANDLE)_beginthreadex(NULL, 0, fn, (void*)i, 0, &id);
- }
+ pg_thrd_t threads[THREADS];
- WaitForMultipleObjects(THREADS, threads, TRUE, INFINITE);
- for (i = 0; i < THREADS; ++i)
- CloseHandle(threads[i]);
-#else
for (i = 0; i < THREADS; ++i)
- pthread_create(&threads[i], NULL, fn, (void *) i);
+ pg_thrd_create(&threads[i], fn, (void *) i);
for (i = 0; i < THREADS; ++i)
- pthread_join(threads[i], NULL);
-#endif
+ pg_thrd_join(threads[i], NULL);
return 0;
}
diff --git a/src/interfaces/ecpg/test/thread/descriptor.pgc b/src/interfaces/ecpg/test/thread/descriptor.pgc
index 30bce7c87b..faa9a0a656 100644
--- a/src/interfaces/ecpg/test/thread/descriptor.pgc
+++ b/src/interfaces/ecpg/test/thread/descriptor.pgc
@@ -1,12 +1,4 @@
-#ifdef WIN32
-#define WIN32_LEAN_AND_MEAN
-#include <windows.h>
-#include <process.h>
-#include <locale.h>
-#else
-#include <pthread.h>
-#endif
-#include <stdio.h>
+#include "port/pg_threads.h"
#define THREADS 16
#define REPEATS 50000
@@ -15,11 +7,7 @@ EXEC SQL include sqlca;
EXEC SQL whenever sqlerror sqlprint;
EXEC SQL whenever not found sqlprint;
-#if defined(WIN32)
-static unsigned __stdcall fn(void* arg)
-#else
-static void* fn(void* arg)
-#endif
+static int fn(void* arg)
{
int i;
@@ -35,28 +23,12 @@ static void* fn(void* arg)
int main ()
{
int i;
-#ifdef WIN32
- HANDLE threads[THREADS];
-#else
- pthread_t threads[THREADS];
-#endif
+ pg_thrd_t threads[THREADS];
-#ifdef WIN32
for (i = 0; i < THREADS; ++i)
- {
- unsigned id;
- threads[i] = (HANDLE)_beginthreadex(NULL, 0, fn, NULL, 0, &id);
- }
-
- WaitForMultipleObjects(THREADS, threads, TRUE, INFINITE);
- for (i = 0; i < THREADS; ++i)
- CloseHandle(threads[i]);
-#else
- for (i = 0; i < THREADS; ++i)
- pthread_create(&threads[i], NULL, fn, NULL);
+ pg_thrd_create(&threads[i], fn, NULL);
for (i = 0; i < THREADS; ++i)
- pthread_join(threads[i], NULL);
-#endif
+ pg_thrd_join(threads[i], NULL);
return 0;
}
diff --git a/src/interfaces/ecpg/test/thread/prep.pgc b/src/interfaces/ecpg/test/thread/prep.pgc
index f61b31ce10..f5a875cc28 100644
--- a/src/interfaces/ecpg/test/thread/prep.pgc
+++ b/src/interfaces/ecpg/test/thread/prep.pgc
@@ -1,15 +1,8 @@
#include <stdint.h>
#include <stdlib.h>
#include "ecpg_config.h"
+#include "port/pg_threads.h"
-#ifdef WIN32
-#define WIN32_LEAN_AND_MEAN
-#include <windows.h>
-#include <process.h>
-#include <locale.h>
-#else
-#include <pthread.h>
-#endif
#include <stdio.h>
#define THREADS 16
@@ -21,11 +14,7 @@ exec sql include ../regression;
exec sql whenever sqlerror sqlprint;
exec sql whenever not found sqlprint;
-#ifdef WIN32
-static unsigned __stdcall fn(void* arg)
-#else
-static void* fn(void* arg)
-#endif
+static int fn(void* arg)
{
int i;
@@ -54,11 +43,7 @@ static void* fn(void* arg)
int main ()
{
intptr_t i;
-#ifdef WIN32
- HANDLE threads[THREADS];
-#else
- pthread_t threads[THREADS];
-#endif
+ pg_thrd_t threads[THREADS];
EXEC SQL CONNECT TO REGRESSDB1;
EXEC SQL SET AUTOCOMMIT TO ON;
@@ -66,22 +51,10 @@ int main ()
EXEC SQL CREATE TABLE T ( i int );
EXEC SQL DISCONNECT;
-#ifdef WIN32
for (i = 0; i < THREADS; ++i)
- {
- unsigned id;
- threads[i] = (HANDLE)_beginthreadex(NULL, 0, fn, (void*)i, 0, &id);
- }
-
- WaitForMultipleObjects(THREADS, threads, TRUE, INFINITE);
- for (i = 0; i < THREADS; ++i)
- CloseHandle(threads[i]);
-#else
- for (i = 0; i < THREADS; ++i)
- pthread_create(&threads[i], NULL, fn, (void *) i);
+ pg_thrd_create(&threads[i], fn, (void *) i);
for (i = 0; i < THREADS; ++i)
- pthread_join(threads[i], NULL);
-#endif
+ pg_thrd_join(threads[i], NULL);
return 0;
}
diff --git a/src/interfaces/ecpg/test/thread/thread.pgc b/src/interfaces/ecpg/test/thread/thread.pgc
index b9b9ebb441..bc967b9da8 100644
--- a/src/interfaces/ecpg/test/thread/thread.pgc
+++ b/src/interfaces/ecpg/test/thread/thread.pgc
@@ -5,28 +5,18 @@
#include <stdint.h>
#include <stdlib.h>
#include "ecpg_config.h"
-
-#ifndef WIN32
-#include <pthread.h>
-#else
-#include <windows.h>
-#include <locale.h>
-#endif
+#include "port/pg_threads.h"
exec sql include ../regression;
-void *test_thread(void *arg);
+int test_thread(void *arg);
int nthreads = 10;
int iterations = 20;
int main()
{
-#ifndef WIN32
- pthread_t *threads;
-#else
- HANDLE *threads;
-#endif
+ pg_thrd_t *threads;
intptr_t n;
EXEC SQL BEGIN DECLARE SECTION;
int l_rows;
@@ -56,23 +46,11 @@ int main()
return 1;
}
for( n = 0; n < nthreads; n++ )
- {
-#ifndef WIN32
- pthread_create(&threads[n], NULL, test_thread, (void *) (n + 1));
-#else
- threads[n] = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE) (void (*) (void)) test_thread, (void *) (n + 1), 0, NULL);
-#endif
- }
+ pg_thrd_create(&threads[n], test_thread, (void *) (n + 1));
/* wait for thread completion */
-#ifndef WIN32
for( n = 0; n < nthreads; n++ )
- {
- pthread_join(threads[n], NULL);
- }
-#else
- WaitForMultipleObjects(nthreads, threads, TRUE, INFINITE);
-#endif
+ pg_thrd_join(threads[n], NULL);
free(threads);
/* and check results */
@@ -88,7 +66,7 @@ int main()
return 0;
}
-void *test_thread(void *arg)
+int test_thread(void *arg)
{
long threadnum = (intptr_t) arg;
@@ -108,7 +86,7 @@ void *test_thread(void *arg)
if( sqlca.sqlcode != 0 )
{
printf("%s: ERROR: cannot connect to database!\n", l_connection);
- return NULL;
+ return 0;
}
EXEC SQL AT :l_connection BEGIN;
@@ -123,5 +101,5 @@ void *test_thread(void *arg)
/* all done */
EXEC SQL AT :l_connection COMMIT;
EXEC SQL DISCONNECT :l_connection;
- return NULL;
+ return 0;
}
diff --git a/src/interfaces/ecpg/test/thread/thread_implicit.pgc b/src/interfaces/ecpg/test/thread/thread_implicit.pgc
index ff9b12a943..7a793dc7f8 100644
--- a/src/interfaces/ecpg/test/thread/thread_implicit.pgc
+++ b/src/interfaces/ecpg/test/thread/thread_implicit.pgc
@@ -5,28 +5,18 @@
#include <stdint.h>
#include <stdlib.h>
#include "ecpg_config.h"
-
-#ifndef WIN32
-#include <pthread.h>
-#else
-#include <windows.h>
-#include <locale.h>
-#endif
+#include "port/pg_threads.h"
exec sql include ../regression;
-void *test_thread(void *arg);
+int test_thread(void *arg);
int nthreads = 10;
int iterations = 20;
int main()
{
-#ifndef WIN32
- pthread_t *threads;
-#else
- HANDLE *threads;
-#endif
+ pg_thrd_t *threads;
intptr_t n;
EXEC SQL BEGIN DECLARE SECTION;
int l_rows;
@@ -56,23 +46,11 @@ int main()
return 1;
}
for( n = 0; n < nthreads; n++ )
- {
-#ifndef WIN32
- pthread_create(&threads[n], NULL, test_thread, (void *) (n + 1));
-#else
- threads[n] = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE) (void (*) (void)) test_thread, (void *) (n+1), 0, NULL);
-#endif
- }
+ pg_thrd_create(&threads[n], test_thread, (void *) (n + 1));
/* wait for thread completion */
-#ifndef WIN32
for( n = 0; n < nthreads; n++ )
- {
- pthread_join(threads[n], NULL);
- }
-#else
- WaitForMultipleObjects(nthreads, threads, TRUE, INFINITE);
-#endif
+ pg_thrd_join(threads[n], NULL);
free(threads);
/* and check results */
@@ -88,7 +66,7 @@ int main()
return 0;
}
-void *test_thread(void *arg)
+int test_thread(void *arg)
{
long threadnum = (intptr_t) arg;
@@ -108,7 +86,7 @@ void *test_thread(void *arg)
if( sqlca.sqlcode != 0 )
{
printf("%s: ERROR: cannot connect to database!\n", l_connection);
- return NULL;
+ return 0;
}
EXEC SQL BEGIN;
@@ -123,5 +101,5 @@ void *test_thread(void *arg)
/* all done */
EXEC SQL COMMIT;
EXEC SQL DISCONNECT :l_connection;
- return NULL;
+ return 0;
}
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 27f8499d8a..b8440a081c 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -69,7 +69,6 @@ endif
ifeq ($(PORTNAME), win32)
OBJS += \
- pthread-win32.o \
win32.o
endif
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index ab308a0580..2397865a6f 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -32,6 +32,7 @@
#include "mb/pg_wchar.h"
#include "pg_config_paths.h"
#include "port/pg_bswap.h"
+#include "port/pg_threads.h"
#ifdef WIN32
#include "win32.h"
@@ -52,12 +53,6 @@
#include <netinet/tcp.h>
#endif
-#ifdef WIN32
-#include "pthread-win32.h"
-#else
-#include <pthread.h>
-#endif
-
#ifdef USE_LDAP
#ifdef WIN32
#include <winldap.h>
@@ -7776,16 +7771,16 @@ error:
static void
default_threadlock(int acquire)
{
- static pthread_mutex_t singlethread_lock = PTHREAD_MUTEX_INITIALIZER;
+ static pg_mtx_t singlethread_lock = PG_MTX_STATIC_INIT;
if (acquire)
{
- if (pthread_mutex_lock(&singlethread_lock))
+ if (pg_mtx_lock(&singlethread_lock) != pg_thrd_success)
Assert(false);
}
else
{
- if (pthread_mutex_unlock(&singlethread_lock))
+ if (pg_mtx_unlock(&singlethread_lock) != pg_thrd_success)
Assert(false);
}
}
diff --git a/src/interfaces/libpq/fe-secure-openssl.c b/src/interfaces/libpq/fe-secure-openssl.c
index b6fffd7b9b..e03a444b07 100644
--- a/src/interfaces/libpq/fe-secure-openssl.c
+++ b/src/interfaces/libpq/fe-secure-openssl.c
@@ -30,6 +30,7 @@
#include "fe-auth.h"
#include "fe-secure-common.h"
#include "libpq-int.h"
+#include "port/pg_threads.h"
#ifdef WIN32
#include "win32.h"
@@ -44,12 +45,6 @@
#include <sys/stat.h>
-#ifdef WIN32
-#include "pthread-win32.h"
-#else
-#include <pthread.h>
-#endif
-
/*
* These SSL-related #includes must come after all system-provided headers.
* This ensures that OpenSSL can take care of conflicts with Windows'
@@ -91,7 +86,7 @@ static bool ssl_lib_initialized = false;
static long crypto_open_connections = 0;
-static pthread_mutex_t ssl_config_mutex = PTHREAD_MUTEX_INITIALIZER;
+static pg_mtx_t ssl_config_mutex = PG_MTX_STATIC_INIT;
static PQsslKeyPassHook_OpenSSL_type PQsslKeyPassHook = NULL;
static int ssl_protocol_version_to_openssl(const char *protocol);
@@ -725,14 +720,14 @@ static unsigned long
pq_threadidcallback(void)
{
/*
- * This is not standards-compliant. pthread_self() returns pthread_t, and
- * shouldn't be cast to unsigned long, but CRYPTO_set_id_callback requires
- * it, so we have to do it.
+ * This is not standards-compliant. pg_thrd_current() returns pg_thrd_t,
+ * and shouldn't be cast to unsigned long, but CRYPTO_set_id_callback
+ * requires it, so we have to do it.
*/
- return (unsigned long) pthread_self();
+ return (unsigned long) pg_thrd_current();
}
-static pthread_mutex_t *pq_lockarray;
+static pg_mtx_t *pq_lockarray;
static void
pq_lockingcallback(int mode, int n, const char *file, int line)
@@ -744,12 +739,12 @@ pq_lockingcallback(int mode, int n, const char *file, int line)
*/
if (mode & CRYPTO_LOCK)
{
- if (pthread_mutex_lock(&pq_lockarray[n]))
+ if (pg_mtx_lock(&pq_lockarray[n]) != pg_thrd_success)
Assert(false);
}
else
{
- if (pthread_mutex_unlock(&pq_lockarray[n]))
+ if (pg_mtx_unlock(&pq_lockarray[n]) != pg_thrd_success)
Assert(false);
}
}
@@ -768,7 +763,7 @@ pq_lockingcallback(int mode, int n, const char *file, int line)
int
pgtls_init(PGconn *conn, bool do_ssl, bool do_crypto)
{
- if (pthread_mutex_lock(&ssl_config_mutex))
+ if (pg_mtx_lock(&ssl_config_mutex) != pg_thrd_success)
return -1;
#ifdef HAVE_CRYPTO_LOCK
@@ -782,19 +777,19 @@ pgtls_init(PGconn *conn, bool do_ssl, bool do_crypto)
{
int i;
- pq_lockarray = malloc(sizeof(pthread_mutex_t) * CRYPTO_num_locks());
+ pq_lockarray = malloc(sizeof(pg_mtx_t) * CRYPTO_num_locks());
if (!pq_lockarray)
{
- pthread_mutex_unlock(&ssl_config_mutex);
+ pg_mtx_unlock(&ssl_config_mutex);
return -1;
}
for (i = 0; i < CRYPTO_num_locks(); i++)
{
- if (pthread_mutex_init(&pq_lockarray[i], NULL))
+ if (pg_mtx_init(&pq_lockarray[i], NULL) != pg_thrd_success)
{
free(pq_lockarray);
pq_lockarray = NULL;
- pthread_mutex_unlock(&ssl_config_mutex);
+ pg_mtx_unlock(&ssl_config_mutex);
return -1;
}
}
@@ -835,7 +830,7 @@ pgtls_init(PGconn *conn, bool do_ssl, bool do_crypto)
ssl_lib_initialized = true;
}
- pthread_mutex_unlock(&ssl_config_mutex);
+ pg_mtx_unlock(&ssl_config_mutex);
return 0;
}
@@ -855,7 +850,7 @@ static void
destroy_ssl_system(void)
{
#if defined(HAVE_CRYPTO_LOCK)
- if (pthread_mutex_lock(&ssl_config_mutex))
+ if (pg_mtx_lock(&ssl_config_mutex) != pg_thrd_success)
return;
if (pq_init_crypto_lib && crypto_open_connections > 0)
@@ -881,7 +876,7 @@ destroy_ssl_system(void)
*/
}
- pthread_mutex_unlock(&ssl_config_mutex);
+ pg_mtx_unlock(&ssl_config_mutex);
#endif
}
@@ -1973,7 +1968,7 @@ my_BIO_s_socket(void)
{
BIO_METHOD *res;
- if (pthread_mutex_lock(&ssl_config_mutex))
+ if (pg_mtx_lock(&ssl_config_mutex) != pg_thrd_success)
return NULL;
res = my_bio_methods;
@@ -2018,7 +2013,7 @@ my_BIO_s_socket(void)
}
my_bio_methods = res;
- pthread_mutex_unlock(&ssl_config_mutex);
+ pg_mtx_unlock(&ssl_config_mutex);
return res;
err:
@@ -2029,7 +2024,7 @@ err:
if (res)
free(res);
#endif
- pthread_mutex_unlock(&ssl_config_mutex);
+ pg_mtx_unlock(&ssl_config_mutex);
return NULL;
}
diff --git a/src/interfaces/libpq/fe-secure.c b/src/interfaces/libpq/fe-secure.c
index f628082337..ea4b1e6288 100644
--- a/src/interfaces/libpq/fe-secure.c
+++ b/src/interfaces/libpq/fe-secure.c
@@ -35,9 +35,7 @@
#include <sys/stat.h>
-#ifdef WIN32
-#include "pthread-win32.h"
-#else
+#ifndef WIN32
#include <pthread.h>
#endif
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 8ed1b28fcc..ce78898b48 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -31,11 +31,6 @@
#include <sys/time.h>
#endif
-#ifdef WIN32
-#include "pthread-win32.h"
-#else
-#include <pthread.h>
-#endif
#include <signal.h>
/* include stuff common to fe and be */
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index 7623aeadab..89e39340d9 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -22,7 +22,7 @@ libpq_sources = files(
libpq_so_sources = [] # for shared lib, in addition to the above
if host_system == 'windows'
- libpq_sources += files('pthread-win32.c', 'win32.c')
+ libpq_sources += files('win32.c')
libpq_so_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
'--NAME', 'libpq',
'--FILEDESC', 'PostgreSQL Access Library',])
@@ -44,8 +44,7 @@ export_file = custom_target('libpq.exports',
kwargs: gen_export_kwargs,
)
-# port needs to be in include path due to pthread-win32.h
-libpq_inc = include_directories('.', '../../port')
+libpq_inc = include_directories('.')
libpq_c_args = ['-DSO_MAJOR_VERSION=5']
# Not using both_libraries() here as
diff --git a/src/interfaces/libpq/pthread-win32.c b/src/interfaces/libpq/pthread-win32.c
deleted file mode 100644
index b40872898d..0000000000
--- a/src/interfaces/libpq/pthread-win32.c
+++ /dev/null
@@ -1,66 +0,0 @@
-/*-------------------------------------------------------------------------
-*
-* pthread-win32.c
-* partial pthread implementation for win32
-*
-* Copyright (c) 2004-2024, PostgreSQL Global Development Group
-* IDENTIFICATION
-* src/interfaces/libpq/pthread-win32.c
-*
-*-------------------------------------------------------------------------
-*/
-
-#include "postgres_fe.h"
-
-#include "pthread-win32.h"
-
-DWORD
-pthread_self(void)
-{
- return GetCurrentThreadId();
-}
-
-void
-pthread_setspecific(pthread_key_t key, void *val)
-{
-}
-
-void *
-pthread_getspecific(pthread_key_t key)
-{
- return NULL;
-}
-
-int
-pthread_mutex_init(pthread_mutex_t *mp, void *attr)
-{
- mp->initstate = 0;
- return 0;
-}
-
-int
-pthread_mutex_lock(pthread_mutex_t *mp)
-{
- /* Initialize the csection if not already done */
- if (mp->initstate != 1)
- {
- LONG istate;
-
- while ((istate = InterlockedExchange(&mp->initstate, 2)) == 2)
- Sleep(0); /* wait, another thread is doing this */
- if (istate != 1)
- InitializeCriticalSection(&mp->csection);
- InterlockedExchange(&mp->initstate, 1);
- }
- EnterCriticalSection(&mp->csection);
- return 0;
-}
-
-int
-pthread_mutex_unlock(pthread_mutex_t *mp)
-{
- if (mp->initstate != 1)
- return EINVAL;
- LeaveCriticalSection(&mp->csection);
- return 0;
-}
diff --git a/src/port/Makefile b/src/port/Makefile
index db7c02117b..617d7e16d7 100644
--- a/src/port/Makefile
+++ b/src/port/Makefile
@@ -46,6 +46,7 @@ OBJS = \
path.o \
pg_bitutils.o \
pg_strong_random.o \
+ pg_threads.o \
pgcheckdir.o \
pgmkdirp.o \
pgsleep.o \
diff --git a/src/port/meson.build b/src/port/meson.build
index ff54b7b53e..3df9b1c9b5 100644
--- a/src/port/meson.build
+++ b/src/port/meson.build
@@ -8,6 +8,7 @@ pgport_sources = [
'path.c',
'pg_bitutils.c',
'pg_strong_random.c',
+ 'pg_threads.c',
'pgcheckdir.c',
'pgmkdirp.c',
'pgsleep.c',
@@ -73,10 +74,6 @@ replace_funcs_neg = [
['strsep'],
]
-if host_system != 'windows'
- replace_funcs_neg += [['pthread_barrier_wait']]
-endif
-
# Replacement functionality to be built if corresponding configure symbol
# is true
replace_funcs_pos = [
diff --git a/src/port/pg_threads.c b/src/port/pg_threads.c
new file mode 100644
index 0000000000..a40910715b
--- /dev/null
+++ b/src/port/pg_threads.c
@@ -0,0 +1,436 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_threads.c
+ * Out-of-line parts of portable multi-threading API.
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/port/pg_threads.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "c.h"
+#include "port/pg_threads.h"
+
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+
+
+/* XXX TODO: make atomics avialable in frontend so we can use these! */
+#define pg_read_barrier()
+#define pg_write_barrier()
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Threads.
+ *
+ * There are small differences between the function types in C11,
+ * POSIX (return type) and Windows (return type signedness, calling
+ * convention). The int return value will survive casting to/from
+ * void * and DWORD respectively, but we still need a small trampoline
+ * function to deal with the different function pointer type.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+typedef struct pg_thrd_thunk
+{
+ pg_thrd_start_t function;
+ void *argument;
+} pg_thrd_thunk;
+
+/*
+ * A trampoline function, to handle calling convention and parameter
+ * variations in the native APIs.
+ */
+#ifdef PG_THREADS_WIN32
+static DWORD __stdcall
+pg_thrd_body(void *vthunk)
+#else
+static void *
+pg_thrd_body(void *vthunk)
+#endif
+{
+ pg_thrd_thunk *thunk = (pg_thrd_thunk *) vthunk;
+ void *argument = thunk->argument;
+ pg_thrd_start_t function = thunk->function;
+ int result;
+
+ free(vthunk);
+
+ result = function(argument);
+
+#ifdef PG_THREADS_WIN32
+ return (DWORD) result;
+#else
+ return (void *) (intptr_t) result;
+#endif
+}
+
+int
+pg_thrd_create(pg_thrd_t *thread, pg_thrd_start_t function, void *argument)
+{
+ pg_thrd_thunk *thunk;
+
+ thunk = malloc(sizeof(*thunk));
+ if (thunk == NULL)
+ return pg_thrd_nomem;
+ thunk->function = function;
+ thunk->argument = argument;
+
+#ifdef WIN32
+ *thread = CreateThread(NULL, 0, pg_thrd_body, thunk, 0, 0);
+ if (*thread != NULL)
+ return pg_thrd_success;
+#else
+ if (pthread_create(thread, NULL, pg_thrd_body, thunk) == 0)
+ return pg_thrd_success;
+#endif
+
+ free(thunk);
+ return pg_thrd_error;
+}
+
+int
+pg_thrd_join(pg_thrd_t thread, int *result)
+{
+#ifdef WIN32
+ DWORD dword_result;
+
+ if (WaitForSingleObject(thread, INFINITE) == WAIT_OBJECT_0)
+ {
+ if (result)
+ {
+ if (!GetExitCodeThread(thread, &dword_result))
+ return pg_thrd_error;
+ *result = (int) dword_result;
+ }
+ CloseHandle(thread);
+ return pg_thrd_success;
+ }
+#else
+ void *void_star_result;
+
+ if (pthread_join(thread, &void_star_result) == 0)
+ {
+ if (result)
+ *result = (int) (intptr_t) void_star_result;
+ return pg_thrd_success;
+ }
+#endif
+ return pg_thrd_error;
+}
+
+void
+pg_thrd_exit(int result)
+{
+#ifdef WIN32
+ ExitThread((DWORD) result);
+#else
+ pthread_exit((void *) (intptr_t) result);
+#endif
+}
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Initialization functions.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef WIN32
+BOOL CALLBACK
+pg_call_once_trampoline(pg_once_flag *flag, void *parameter, void **context)
+{
+ pg_call_once_function_t function = (pg_call_once_function_t) parameter;
+
+ function();
+ return TRUE;
+}
+#endif
+
+
+/*-------------------------------------------------------------------------
+ *
+ * Thread-specific storage.
+ *
+ * This extra support code is only needed when we can't use the native
+ * support for thread-local storage destructors. Normally that is
+ * Windows, due to incompatible calling conventions, but this code
+ * path can be activated on POSIX systems too for testing.
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifdef PG_THREADS_NEED_DESTRUCTOR_TABLE
+
+struct dtor_table_entry
+{
+ pg_tss_t tss_id;
+ pg_tss_dtor_t function;
+};
+
+static pg_rwlock_t dtor_table_lock = PG_RWLOCK_STATIC_INIT;
+static size_t dtor_table_count = 0;
+static size_t dtor_table_capacity = 0;
+static struct dtor_table_entry *dtor_table;
+
+/* One native TLS key with a native destructor, which drives all others. */
+static pg_tss_t pg_tss_destructor_hook;
+static bool pg_tss_run_destructors_installed;
+
+/*
+ * Helper function for recording the destructor for a given tss_id.
+ * Returns true on success, or false if the table is full.
+ */
+static bool
+pg_tss_dtor_set(pg_tss_t tss_id, pg_tss_dtor_t destructor)
+{
+ bool have_space = true;
+
+ pg_rwlock_wrlock(&dtor_table_lock);
+
+ /* Make sure we have space, or fail. */
+ if (dtor_table_count == dtor_table_capacity)
+ {
+ struct dtor_table_entry *new_dtor_table;
+ size_t new_dtor_table_capacity;
+
+ new_dtor_table_capacity = Max(1, dtor_table_capacity * 2);
+ new_dtor_table = malloc(sizeof(dtor_table[0]) * new_dtor_table_capacity);
+ if (new_dtor_table == NULL)
+ {
+ /* Out of memory. */
+ have_space = false;
+ }
+ else
+ {
+ if (dtor_table_count > 0)
+ {
+ memcpy(new_dtor_table,
+ dtor_table,
+ sizeof(dtor_table[0]) * dtor_table_count);
+ free(dtor_table);
+ }
+ dtor_table = new_dtor_table;
+ dtor_table_capacity = new_dtor_table_capacity;
+ }
+ }
+
+#ifdef USE_ASSERTION_CHECKING
+ /* We don't expect to see this ID already in the table. */
+ for (size_t i = 0; i < dtor_table_count; i++)
+ Assert(dtor_table[i].tss_id != tss_id);
+#endif
+
+ /* Store it. */
+ if (have_space)
+ {
+ Assert(dtor_table_count < dtor_table_capacity);
+ dtor_table[dtor_table_count].tss_id = tss_id;
+ dtor_table[dtor_table_count].function = destructor;
+
+ dtor_table_count++;
+ }
+
+ pg_wrlock_unlock(&dtor_table_lock);
+
+ return have_space;
+}
+
+/*
+ * The destructor installed for the single special FLS value that will
+ * be called by Windows (or POSIX if we are using the special test
+ * mode). This must have CALLBACK calling convention on Windows,
+ * which is the reason we can't just use its FlsAlloc() destructors
+ * directly for pg_tss_create().
+ */
+static void
+#ifdef WIN32
+ CALLBACK
+#endif
+pg_tss_run_destructors(void *data)
+{
+ pg_rwlock_rdlock(&dtor_table_lock);
+
+ for (int i = 0; i < PG_TSS_DTOR_ITERATIONS; ++i)
+ {
+ bool seen_non_null_value = false;
+
+ for (size_t slot = 0; slot < dtor_table_count; ++slot)
+ {
+ pg_tss_t tss_id = dtor_table[slot].tss_id;
+ void *value = pg_tss_get(tss_id);
+
+ if (value)
+ {
+ pg_tss_dtor_t function = dtor_table[slot].function;
+
+ Assert(function);
+
+ /* Clear value. */
+ pg_tss_set(tss_id, NULL);
+
+ /*
+ * We'll need to go around again to make sure that a
+ * destructor called in this iteration didn't set something.
+ */
+ seen_non_null_value = true;
+
+ /* Unlock while running the destructor. */
+ pg_rdlock_unlock(&dtor_table_lock);
+ function(value);
+ pg_rwlock_rdlock(&dtor_table_lock);
+ }
+ }
+
+ /* If we didn't see any values, we're finished. */
+ if (!seen_non_null_value)
+ break;
+ }
+ pg_rdlock_unlock(&dtor_table_lock);
+}
+
+static void
+pg_tss_install_run_destructors(void)
+{
+ /*
+ * We need a way to make sure our TSS destructors run at thread exit, even
+ * if the thread exits via native calls instead of our own pg_thrd_exit()
+ * or trampoline function. So we register one real native 'hook'
+ * destructor that will then call all the destructors in our own
+ * destructor table.
+ *
+ * On Windows, we use FlsAlloc(), not TlsAlloc(), because that supports
+ * destructors. Unforunately they have the wrong calling convention, or
+ * we could simply use them directly instead of doing all this extra work.
+ */
+#ifdef PG_THREADS_WIN32
+ pg_tss_destructor_hook = FlsAlloc(pg_tss_run_destructors);
+ if (pg_tss_destructor_hook == FLS_OUT_OF_INDEXES)
+ return;
+#else
+ if (pthread_key_create(&pg_tss_destructor_hook, pg_tss_run_destructors) != 0)
+ return;
+#endif
+ pg_write_barrier();
+ pg_tss_run_destructors_installed = true;
+
+ /*
+ * Make sure that any thread that receives a pg_tss_t and might store a
+ * value can see that there is now potentially a registered destructor.
+ */
+ pg_write_barrier();
+}
+
+/*
+ * Called every time pg_tss_set() installs a non-NULL value.
+ */
+void
+pg_tss_ensure_destructors_in_this_thread(void)
+{
+ /*
+ * Pairs with pg_tss_install_run_destructors(), called by pg_tss_create().
+ * This makes sure that we know if the tss_id being set could possibly
+ * have a destructor. We don't want to pay the cost of checking, but we
+ * can check with a simple load if *any* tss_id has a destructor. If so,
+ * we make sure that pg_tss_destructor_hook has a non-NULL value in *this*
+ * thread, because both Windows and POSIX will only call a destructor for
+ * a non-NULL value.
+ */
+ pg_read_barrier();
+ if (pg_tss_run_destructors_installed)
+ {
+#ifdef PG_THREADS_WIN32
+ if (FlsGetValue(pg_tss_destructor_hook) == NULL)
+ FlsSetValue(pg_tss_destructor_hook, (void *) 1);
+#else
+ if (pthread_getspecific(pg_tss_destructor_hook) == NULL)
+ pthread_setspecific(pg_tss_destructor_hook, (void *) 1);
+#endif
+ }
+}
+#endif
+
+int
+pg_tss_create(pg_tss_t *tss_id, pg_tss_dtor_t destructor)
+{
+#ifdef PG_THREADS_NEED_DESTRUCTOR_TABLE
+ static pg_once_flag destructor_cleanup_once;
+
+ /*
+ * Make sure our destructor hook is registered with the operating system
+ * in this process. This happens only once in the whole process. Making
+ * sure it will run actually in each thread happens in
+ * pg_tss_ensure_destructors_will_run().
+ */
+ pg_call_once(&destructor_cleanup_once, pg_tss_install_run_destructors);
+ if (!pg_tss_run_destructors_installed)
+ return pg_thrd_error;
+#endif
+
+#ifdef PG_THREADS_WIN32
+ /* Windows native TSL, our own destructors machinery. */
+ *tss_id = TlsAlloc();
+ if (*tss_id == TLS_OUT_OF_INDEXES)
+ return pg_thrd_error;
+#elif defined(PG_THREADS_NEED_DESTRUCTOR_TABLE)
+ /* POSIX, but testing our own destructor machinery. */
+ if (pthread_key_create(tss_id, NULL) != 0)
+ return pg_thrd_error;
+#else
+ /* POSIX handles destructors. */
+ return pg_thrd_maperror(pthread_key_create(tss_id, destructor));
+#endif
+
+#ifdef PG_THREADS_NEED_DESTRUCTOR_TABLE
+ /* Allocate destructor table entry, or fail and clean up. */
+ if (destructor &&!pg_tss_dtor_set(*tss_id, destructor))
+ {
+#ifdef PG_THREADS_WIN32
+ TlsFree(*tss_id);
+#else
+ pthread_key_delete(*tss_id);
+#endif
+ return pg_thrd_error;
+ }
+#endif
+
+ return pg_thrd_success;
+}
+
+void
+pg_tss_dtor_delete(pg_tss_t tss_id)
+{
+#ifdef PG_THREADS_NEED_DESTRUCTOR_TABLE
+ /*
+ * We have to search the destructor table linearly, but deleting IDs is
+ * probably very rare so that's OK.
+ */
+ pg_rwlock_wrlock(&dtor_table_lock);
+ for (size_t i = 0; i < dtor_table_count; ++i)
+ {
+ if (dtor_table[i].tss_id == tss_id)
+ {
+ /* Move the other values to compact the table. */
+ if (i < dtor_table_count - 1)
+ memmove(&dtor_table[i],
+ &dtor_table[i + 1],
+ sizeof(dtor_table[i]) * (dtor_table_count - i - 1));
+ dtor_table_count--;
+ break;
+ }
+ }
+ pg_wrlock_unlock(&dtor_table_lock);
+#endif
+
+#ifdef PG_THREADS_WIN32
+ TlsFree(tss_id);
+#else
+ pthread_key_delete(tss_id);
+#endif
+}
diff --git a/src/port/pthread-win32.h b/src/port/pthread-win32.h
deleted file mode 100644
index 5f33269057..0000000000
--- a/src/port/pthread-win32.h
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * src/port/pthread-win32.h
- */
-#ifndef __PTHREAD_H
-#define __PTHREAD_H
-
-typedef ULONG pthread_key_t;
-
-typedef struct pthread_mutex_t
-{
- /* initstate = 0: not initialized; 1: init done; 2: init in progress */
- LONG initstate;
- CRITICAL_SECTION csection;
-} pthread_mutex_t;
-
-#define PTHREAD_MUTEX_INITIALIZER { 0 }
-
-typedef int pthread_once_t;
-
-DWORD pthread_self(void);
-
-void pthread_setspecific(pthread_key_t, void *);
-void *pthread_getspecific(pthread_key_t);
-
-int pthread_mutex_init(pthread_mutex_t *, void *attr);
-int pthread_mutex_lock(pthread_mutex_t *);
-
-/* blocking */
-int pthread_mutex_unlock(pthread_mutex_t *);
-
-#endif
diff --git a/src/port/pthread_barrier_wait.c b/src/port/pthread_barrier_wait.c
deleted file mode 100644
index 835dbf1c7a..0000000000
--- a/src/port/pthread_barrier_wait.c
+++ /dev/null
@@ -1,77 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * pthread_barrier_wait.c
- * Implementation of pthread_barrier_t support for platforms lacking it.
- *
- * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
- *
- * IDENTIFICATION
- * src/port/pthread_barrier_wait.c
- *
- *-------------------------------------------------------------------------
- */
-
-#include "c.h"
-
-#include "port/pg_pthread.h"
-
-int
-pthread_barrier_init(pthread_barrier_t *barrier, const void *attr, int count)
-{
- int error;
-
- barrier->sense = false;
- barrier->count = count;
- barrier->arrived = 0;
- if ((error = pthread_cond_init(&barrier->cond, NULL)) != 0)
- return error;
- if ((error = pthread_mutex_init(&barrier->mutex, NULL)) != 0)
- {
- pthread_cond_destroy(&barrier->cond);
- return error;
- }
-
- return 0;
-}
-
-int
-pthread_barrier_wait(pthread_barrier_t *barrier)
-{
- bool initial_sense;
-
- pthread_mutex_lock(&barrier->mutex);
-
- /* We have arrived at the barrier. */
- barrier->arrived++;
- Assert(barrier->arrived <= barrier->count);
-
- /* If we were the last to arrive, release the others and return. */
- if (barrier->arrived == barrier->count)
- {
- barrier->arrived = 0;
- barrier->sense = !barrier->sense;
- pthread_mutex_unlock(&barrier->mutex);
- pthread_cond_broadcast(&barrier->cond);
-
- return PTHREAD_BARRIER_SERIAL_THREAD;
- }
-
- /* Wait for someone else to flip the sense. */
- initial_sense = barrier->sense;
- do
- {
- pthread_cond_wait(&barrier->cond, &barrier->mutex);
- } while (barrier->sense == initial_sense);
-
- pthread_mutex_unlock(&barrier->mutex);
-
- return 0;
-}
-
-int
-pthread_barrier_destroy(pthread_barrier_t *barrier)
-{
- pthread_cond_destroy(&barrier->cond);
- pthread_mutex_destroy(&barrier->mutex);
- return 0;
-}
diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck
index 436e2b92a3..03faaefb77 100755
--- a/src/tools/pginclude/headerscheck
+++ b/src/tools/pginclude/headerscheck
@@ -102,7 +102,6 @@ do
test "$f" = src/include/port/win32_msvc/dirent.h && continue
test "$f" = src/include/port/win32_msvc/utime.h && continue
test "$f" = src/include/port/win32ntdll.h && continue
- test "$f" = src/port/pthread-win32.h && continue
# Likewise, these files are platform-specific, and the one
# relevant to our platform will be included by atomics.h.
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 547d14b3e7..b4e5356468 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3676,12 +3676,14 @@ pgParameterStatus
pg_atomic_flag
pg_atomic_uint32
pg_atomic_uint64
+pg_barrier_t
pg_be_sasl_mech
pg_case_map
pg_category_range
pg_checksum_context
pg_checksum_raw_context
pg_checksum_type
+pg_cnd_t
pg_compress_algorithm
pg_compress_specification
pg_conn_host
@@ -3706,7 +3708,10 @@ pg_local_to_utf_combined
pg_locale_t
pg_mb_radix_tree
pg_md5_ctx
+pg_mtx_t
+pg_mtx_type_t
pg_on_exit_callback
+pg_once_flag
pg_prng_state
pg_re_flags
pg_regex_t
@@ -3720,8 +3725,12 @@ pg_sha384_ctx
pg_sha512_ctx
pg_snapshot
pg_stack_base_t
+pg_thrd_error_t
+pg_thrd_t
+pg_thrd_thunk
pg_time_t
pg_time_usec_t
+pg_tss_t
pg_tz
pg_tz_cache
pg_tzenum
--
2.39.2
^ permalink raw reply [nested|flat] 35+ messages in thread
* Re: Cleaning up threading code
@ 2024-08-26 09:59 Heikki Linnakangas <[email protected]>
parent: Thomas Munro <[email protected]>
0 siblings, 0 replies; 35+ messages in thread
From: Heikki Linnakangas @ 2024-08-26 09:59 UTC (permalink / raw)
To: Thomas Munro <[email protected]>; +Cc: Peter Eisentraut <[email protected]>; Andres Freund <[email protected]>; pgsql-hackers
On 21/08/2024 15:51, Thomas Munro wrote:
> On Sun, Apr 14, 2024 at 3:16 PM Thomas Munro <[email protected]> wrote:
>> So I'll go ahead and add the storage class to the next version, and
>> contemplate a couple of different options for the tss stuff, including
>> perhaps leaving it out if that seems doable.
>
> Here is a new attempt at pg_threads.h. Still work in progress, but
> passing tests, with storage class and working TSS, showing various
> users.
Looks good, thanks!
> I eventually understood first that my TSS destructor problems on
> Windows came from mismatched calling conventions, and that you can't
> really trampoline your way around that, at least not without doing
> some pretty unreasonable things, and that is why nobody can emulate
> either tss_create() or pthread_key_create() directly with Windows'
> FlsAlloc(), so everybody who tries finishes up building their own
> infrastructure to track destructors, or in ECPG's case just leaks all
> the memory instead.
>
> Here's the simplest implementation I could come up with so far. I
> don't have Windows so I made it possible to use emulated TSS
> destructors on my local machine with a special macro for testing, and
> then argued with CI for a while until the other machines agreed.
> Otherwise, it's all a fairly thin wrapper and hopefully not suprising.
In the Windows implementation of pg_tss_create(), how do threads and
FlsAlloc() interact? If I understand correctly, one thread can run
multiple "fibers" in a co-operative fashion. It's a legacy feature, not
widely used nowadays, but what will happen if someone does try to use
fibers? I think that will result in chaos. You're using FlsAlloc() to
install the destructor callback but TlsAlloc() for allocating the actual
thread-local values. So all fibers running on the same thread will share
the storage, but the destructor will be called whenever any of the
fibers exit, clearing the TSS storage for all fibers.
I don't know much about Windows or fibers but mixing the Tls* and Fls*
functions seems unwise.
To be honest, I think we should just accept the limitation that TSS
destructors don't run on Windows. Yes, it means we'll continue to leak
memory on ecpg, but that's not a new issue. We can address that as a
separate patch later, if desired. Or more likely, do nothing until C11
threads with proper destructors become widely available on Windows.
> One thing this would need to be complete, at least the way I've
> implemented it, is memory barriers, for non-TSO hardware, which would
> require lifting the ban on atomics.h in frontend code, or at least
> parts of it.
+1 on doing that. Although it becomes moot if you just give up on the
destructors on Windows.
> Only 64 bit emulation is actually tied to the backend
> now (because it calls spinlock stuff, that itself is backend-only, but
> also it doesn't actually need to be either). Or maybe I can figure
> out a different scheme that doesn't need that. Or something...
You could use a pg_mtx now instead of a spinlock. I wonder if there are
any supported platforms left that don't have 64-bit atomics though.
> + * We have some extensions of our own, not present in C11:
> + *
> + * - pg_rwlock_t for read/write locks
> + * - pg_mtx_t static initializer PG_MTX_STATIC_INIT
> + * - pg_barrier_t
Hmm, I don't see anything wrong with those extensions as such, but I
wonder how badly we really need them?
pg_rwlock is used by:
- Windows implementation of pg_mtx_t, to provide PG_MTX_STATIC_INIT
- the custom Thread-Specific Storage implementation (i.e. Windows)
PG_MTX_STATIC_INIT is used by:
- ecpg
- libpq
pg_barrier_t is used by:
- pgbench
pg_rwlock goes away if you give up on the TSS destructors on Windows,
and implement pg_mtx directly with SRWLOCK on Windows.
The barrier implementation could easily be moved into pgbench, if we
don't expect to need it in other places soon. Having a generic
implementation seems fine too though, it's pretty straightforward.
PG_MTX_STATIC_INIT seems hard to get rid of. I suppose you could use
call_once() to ensure it's initialized only once, but a static
initializer is a lot nicer to use.
> diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
> index 01b4309a71..d8416b19e3 100644
> --- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
> +++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
> @@ -169,7 +169,7 @@ bool ecpg_get_data(const PGresult *, int, int, int, enum ECPGttype type,
> enum ECPGttype, char *, char *, long, long, long,
> enum ARRAY_TYPE, enum COMPAT_MODE, bool);
>
> -void ecpg_pthreads_init(void);
> +#define ecpg_pthreads_init()
> struct connection *ecpg_get_connection(const char *connection_name);
> char *ecpg_alloc(long size, int lineno);
> char *ecpg_auto_alloc(long size, int lineno);
Is it safe to remove the function, or might it be referenced by existing
binaries that were built against an older ecpglib version? I believe
it's OK to remove, it's not part of the public API and should not be
called directly from a binary. But in that case, we don't need the dummy
#define either.
> +/*-------------------------------------------------------------------------
> + *
> + * Barriers. Not in C11. Apple currently lacks the POSIX version.
> + * We assume that the OS might know a better way to implement it that
> + * we do, so we only provide our own if we have to.
> + *
> + *-------------------------------------------------------------------------
> + */
> +
> +#ifdef PG_THREADS_WIN32
> +typedef SYNCHRONIZATION_BARRIER pg_barrier_t;
> +#elif defined(HAVE_PTHREAD_BARRIER)
> +typedef pthread_barrier_t pg_barrier_t;
> +#else
> +typedef struct pg_barrier_t
> +{
> + bool sense;
> + int expected;
> + int arrived;
> + pg_mtx_t mutex;
> + pg_cnd_t cond;
> +} pg_barrier_t;
> +#endif
I'd suggest calling this pg_thrd_barrier_t or even pg_thread_barrier_t.
It's a little more verbose, but would rhyme with pthread_barrier_t. And
to avoid confusing it with memory barriers and ProcSignal barriers.
Comments on the TSS implementation follow, which largely become moot if
you give up on the destructor support on Windows:
> +void
> +pg_tss_dtor_delete(pg_tss_t tss_id)
Should this be pg_tss_delete(), since the C11 function is tss_delete()?
Or is this different? There are actually no callers currently, so maybe
just leave it out. pg_tss_dtor_delete() also seems racy, if run
concurrently with pg_tss_run_destructors().
> + * Make sure our destructor hook is registered with the operating system
> + * in this process. This happens only once in the whole process. Making
> + * sure it will run actually in each thread happens in
> + * pg_tss_ensure_destructors_will_run().
the function is called pg_tss_ensure_destructors_in_this_thread(), not
pg_tss_ensure_destructors_will_run().
> +/*
> + * Called every time pg_tss_set() installs a non-NULL value.
> + */
> +void
> +pg_tss_ensure_destructors_in_this_thread(void)
> +{
> + /*
> + * Pairs with pg_tss_install_run_destructors(), called by pg_tss_create().
> + * This makes sure that we know if the tss_id being set could possibly
> + * have a destructor. We don't want to pay the cost of checking, but we
> + * can check with a simple load if *any* tss_id has a destructor. If so,
> + * we make sure that pg_tss_destructor_hook has a non-NULL value in *this*
> + * thread, because both Windows and POSIX will only call a destructor for
> + * a non-NULL value.
> + */
> + pg_read_barrier();
> + if (pg_tss_run_destructors_installed)
> + {
> +#ifdef PG_THREADS_WIN32
> + if (FlsGetValue(pg_tss_destructor_hook) == NULL)
> + FlsSetValue(pg_tss_destructor_hook, (void *) 1);
> +#else
> + if (pthread_getspecific(pg_tss_destructor_hook) == NULL)
> + pthread_setspecific(pg_tss_destructor_hook, (void *) 1);
> +#endif
> + }
> +}
> +#endif
I believe this cannot get called if !pg_tss_run_destructors_installed,
no need to check that. Because pg_tss_create() will fail
pg_tss_run_destructors is set. Maybe turn it into an Assert. Or
alternatively, skip the call to pg_tss_install_run_destructors when
pg_tss_create() is called with destructor==NULL. I think that may have
been the idea here, based on the comment above.
The write barriers in pg_tss_install_run_destructors() seem excessive.
The function will only ever run in one thread, protected by
pg_call_once(). pg_call_once() surely provides all the necessary barriers.
--
Heikki Linnakangas
Neon (https://neon.tech)
^ permalink raw reply [nested|flat] 35+ messages in thread
end of thread, other threads:[~2024-08-26 09:59 UTC | newest]
Thread overview: 35+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-12-13 16:53 Change pgarch_readyXlog() to return .history files first David Steele <[email protected]>
2018-12-13 18:45 ` David Steele <[email protected]>
2018-12-14 00:15 ` Michael Paquier <[email protected]>
2018-12-14 13:43 ` David Steele <[email protected]>
2018-12-15 00:10 ` Michael Paquier <[email protected]>
2018-12-20 11:57 ` David Steele <[email protected]>
2018-12-21 03:19 ` Michael Paquier <[email protected]>
2018-12-21 04:49 ` Kyotaro HORIGUCHI <[email protected]>
2018-12-21 05:17 ` Michael Paquier <[email protected]>
2018-12-21 05:23 ` Kyotaro HORIGUCHI <[email protected]>
2018-12-21 06:17 ` David Steele <[email protected]>
2018-12-21 23:55 ` Michael Paquier <[email protected]>
2018-12-24 11:31 ` Michael Paquier <[email protected]>
2018-12-26 14:45 ` David Steele <[email protected]>
2020-03-25 23:48 [PATCH v11 3/3] Rework wal_receiver_create_temp_slot using new infrastructure Alvaro Herrera <[email protected]>
2022-12-30 16:56 MERGE ... WHEN NOT MATCHED BY SOURCE Dean Rasheed <[email protected]>
2023-06-10 05:26 Re: Cleaning up threading code Andres Freund <[email protected]>
2023-06-10 15:19 ` Re: Cleaning up threading code Joe Conway <[email protected]>
2023-07-03 07:29 ` Re: Cleaning up threading code Peter Eisentraut <[email protected]>
2023-07-03 08:43 ` Re: Cleaning up threading code Heikki Linnakangas <[email protected]>
2023-07-09 22:45 ` Re: Cleaning up threading code Thomas Munro <[email protected]>
2023-07-11 20:58 ` Re: Cleaning up threading code Thomas Munro <[email protected]>
2023-07-11 22:49 ` Re: Cleaning up threading code Andres Freund <[email protected]>
2023-07-12 03:21 ` Re: Cleaning up threading code Thomas Munro <[email protected]>
2023-07-12 03:34 ` Re: Cleaning up threading code Thomas Munro <[email protected]>
2023-07-12 05:27 ` Re: Cleaning up threading code Thomas Munro <[email protected]>
2024-04-14 03:16 ` Re: Cleaning up threading code Thomas Munro <[email protected]>
2024-08-21 12:51 ` Re: Cleaning up threading code Thomas Munro <[email protected]>
2024-08-26 09:59 ` Re: Cleaning up threading code Heikki Linnakangas <[email protected]>
2024-01-22 02:10 ` Re: MERGE ... WHEN NOT MATCHED BY SOURCE Peter Smith <[email protected]>
2024-01-26 15:48 ` Re: MERGE ... WHEN NOT MATCHED BY SOURCE Dean Rasheed <[email protected]>
2024-01-26 15:57 ` Re: MERGE ... WHEN NOT MATCHED BY SOURCE Alvaro Herrera <[email protected]>
2024-01-29 10:07 ` Re: MERGE ... WHEN NOT MATCHED BY SOURCE Dean Rasheed <[email protected]>
2024-01-26 14:59 ` Re: MERGE ... WHEN NOT MATCHED BY SOURCE vignesh C <[email protected]>
2024-01-26 15:48 ` Re: MERGE ... WHEN NOT MATCHED BY SOURCE Dean Rasheed <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox