pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
24+ messages / 4 participants
[nested] [flat]
* pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-14 01:05 Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2018-09-14 01:05 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Allow concurrent-safe open() and fopen() in frontend code for Windows
PostgreSQL uses a custom wrapper for open() and fopen() which is
concurrent-safe, allowing multiple processes to open and work on the
same file. This has a couple of advantages:
- pg_test_fsync does not handle O_DSYNC correctly otherwise, leading to
false claims that disks are unsafe.
- TAP tests can run into race conditions when a postmaster and pg_ctl
open postmaster.pid, fixing some random failures in the buildfam.
pg_upgrade is one frontend tool using workarounds to bypass file locking
issues with the log files it generates, however the interactions with
pg_ctl are proving to be tedious to get rid of, so this is left for
later.
Author: Laurenz Albe
Reviewed-by: Michael Paquier, Kuntal Ghosh
Discussion: https://postgr.es/m/1527846213.2475.31.camel@cybertec.at
Discussion: https://postgr.es/m/16922.1520722108@sss.pgh.pa.us
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/0ba06e0bfb8cfd24ff17aca92aa72245ddd6c4d7
Modified Files
--------------
src/bin/initdb/initdb.c | 8 ++++++++
src/bin/pg_basebackup/pg_receivewal.c | 2 +-
src/bin/pg_verify_checksums/pg_verify_checksums.c | 2 +-
src/common/file_utils.c | 4 ++--
src/include/port.h | 3 ---
5 files changed, 12 insertions(+), 7 deletions(-)
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-14 15:22 Tom Lane <tgl@sss.pgh.pa.us>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 24+ messages in thread
From: Tom Lane @ 2018-09-14 15:22 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-committers@lists.postgresql.org
Michael Paquier <michael@paquier.xyz> writes:
> Allow concurrent-safe open() and fopen() in frontend code for Windows
I'm surprised you didn't back-patch this --- isn't it a bug fix?
A compromise might be to push it to v11 but not further.
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-14 22:49 Michael Paquier <michael@paquier.xyz>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2018-09-14 22:49 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org
On Fri, Sep 14, 2018 at 11:22:55AM -0400, Tom Lane wrote:
> I'm surprised you didn't back-patch this --- isn't it a bug fix?
>
> A compromise might be to push it to v11 but not further.
Yes, that's clearly a bug fix from pg_ctl point of view with TAP tests.
However, I have rather cold feet about back-patching for two reasons as
this introduces a behavior change:
- The concurrency part disappears.
- Caller of open() needs to enforce text mode to strip correctly CRLF
characters.
I would be fine to get that into v11 though, as that is not released
yet. You will have to wait for a couple of days though, there is a long
week-end here away from laptops and electronic devices ;)
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../../20180914224939.GA1178@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 11:39 Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 24+ messages in thread
From: Michael Paquier @ 2018-09-17 11:39 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Allow concurrent-safe open() and fopen() in frontend code for Windows
PostgreSQL uses a custom wrapper for open() and fopen() which is
concurrent-safe, allowing multiple processes to open and work on the
same file. This has a couple of advantages:
- pg_test_fsync does not handle O_DSYNC correctly otherwise, leading to
false claims that disks are unsafe.
- TAP tests can run into race conditions when a postmaster and pg_ctl
open postmaster.pid, fixing some random failures in the buildfam.
pg_upgrade is one frontend tool using workarounds to bypass file locking
issues with the log files it generates, however the interactions with
pg_ctl are proving to be tedious to get rid of, so this is left for
later.
Author: Laurenz Albe
Reviewed-by: Michael Paquier, Kuntal Ghosh
Discussion: https://postgr.es/m/1527846213.2475.31.camel@cybertec.at
Discussion: https://postgr.es/m/16922.1520722108@sss.pgh.pa.us
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/f02259fe93e75d5443a2fabe2f2f38b81924ab36
Modified Files
--------------
src/bin/initdb/initdb.c | 8 ++++++++
src/bin/pg_basebackup/pg_receivewal.c | 2 +-
src/bin/pg_verify_checksums/pg_verify_checksums.c | 2 +-
src/common/file_utils.c | 4 ++--
src/include/port.h | 3 ---
5 files changed, 12 insertions(+), 7 deletions(-)
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 11:41 Michael Paquier <michael@paquier.xyz>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2018-09-17 11:41 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org
On Sat, Sep 15, 2018 at 07:49:39AM +0900, Michael Paquier wrote:
> I would be fine to get that into v11 though, as that is not released
> yet. You will have to wait for a couple of days though, there is a long
> week-end here away from laptops and electronic devices ;)
OK, REL_11_STABLE has been patched as well, after doing a couple of
extra tests on Windows.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../../20180917114113.GA31460@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 13:41 Tom Lane <tgl@sss.pgh.pa.us>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 24+ messages in thread
From: Tom Lane @ 2018-09-17 13:41 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-committers@lists.postgresql.org, Andrew Dunstan <andrew@dunslane.net>
Michael Paquier <michael@paquier.xyz> writes:
> OK, REL_11_STABLE has been patched as well, after doing a couple of
> extra tests on Windows.
BTW, I'm a bit concerned by the fact that bowerbird has failed its
last couple of HEAD runs at the pgbench step. The first such
failure was here:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&dt=2018-09-15%2014%3A19%3A58
Looking at the set of commits between the prior run and that one,
it's hard to see anything that could have triggered the test failures
other than this patch --- but I also don't see how this patch would've
blown up pgbench without breaking earlier tests. Ideas?
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 14:02 Michael Paquier <michael@paquier.xyz>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2018-09-17 14:02 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org, Andrew Dunstan <andrew@dunslane.net>
On Mon, Sep 17, 2018 at 09:41:37AM -0400, Tom Lane wrote:
> BTW, I'm a bit concerned by the fact that bowerbird has failed its
> last couple of HEAD runs at the pgbench step. The first such
> failure was here:
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&dt=2018-09-15%2014%3A19%3A58
>
> Looking at the set of commits between the prior run and that one,
> it's hard to see anything that could have triggered the test failures
> other than this patch --- but I also don't see how this patch would've
> blown up pgbench without breaking earlier tests. Ideas?
Thanks, I have been looking at the build farm but I missed this one.
dory, which uses VS 2015 is not complaining because it does not run
bincheck. At quick glance, it seems to be caused by process_file() in
pgbench.c which would need to open files in text mode, and the input
file parsing fails at the first '\' character found. I'll test that
stuff on tomorrow morning manually.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../../20180917140202.GF31460@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 14:48 Tom Lane <tgl@sss.pgh.pa.us>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 24+ messages in thread
From: Tom Lane @ 2018-09-17 14:48 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-committers@lists.postgresql.org, Andrew Dunstan <andrew@dunslane.net>
Michael Paquier <michael@paquier.xyz> writes:
> On Mon, Sep 17, 2018 at 09:41:37AM -0400, Tom Lane wrote:
>> Looking at the set of commits between the prior run and that one,
>> it's hard to see anything that could have triggered the test failures
>> other than this patch --- but I also don't see how this patch would've
>> blown up pgbench without breaking earlier tests. Ideas?
> Thanks, I have been looking at the build farm but I missed this one.
> dory, which uses VS 2015 is not complaining because it does not run
> bincheck. At quick glance, it seems to be caused by process_file() in
> pgbench.c which would need to open files in text mode, and the input
> file parsing fails at the first '\' character found.
Oh, you're thinking pgbench isn't robust against finding \r's visible
in its input? Could be.
> I'll test that stuff on tomorrow morning manually.
We've got a bit of a timing problem because we want to wrap 11beta4/rc1
(still TBD) in a few hours. I'll take a look and see if I can push a
quick fix before that.
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 15:13 Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 24+ messages in thread
From: Andrew Dunstan @ 2018-09-17 15:13 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-committers@lists.postgresql.org
On 09/17/2018 10:48 AM, Tom Lane wrote:
> Michael Paquier <michael@paquier.xyz> writes:
>> On Mon, Sep 17, 2018 at 09:41:37AM -0400, Tom Lane wrote:
>>> Looking at the set of commits between the prior run and that one,
>>> it's hard to see anything that could have triggered the test failures
>>> other than this patch --- but I also don't see how this patch would've
>>> blown up pgbench without breaking earlier tests. Ideas?
>> Thanks, I have been looking at the build farm but I missed this one.
>> dory, which uses VS 2015 is not complaining because it does not run
>> bincheck. At quick glance, it seems to be caused by process_file() in
>> pgbench.c which would need to open files in text mode, and the input
>> file parsing fails at the first '\' character found.
> Oh, you're thinking pgbench isn't robust against finding \r's visible
> in its input? Could be.
>
>> I'll test that stuff on tomorrow morning manually.
> We've got a bit of a timing problem because we want to wrap 11beta4/rc1
> (still TBD) in a few hours. I'll take a look and see if I can push a
> quick fix before that.
When you do I'll start a bowerbird run to check it.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 16:13 Tom Lane <tgl@sss.pgh.pa.us>
parent: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
0 siblings, 2 replies; 24+ messages in thread
From: Tom Lane @ 2018-09-17 16:13 UTC (permalink / raw)
To: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>; +Cc: Michael Paquier <michael@paquier.xyz>; pgsql-committers@lists.postgresql.org
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 09/17/2018 10:48 AM, Tom Lane wrote:
>> We've got a bit of a timing problem because we want to wrap 11beta4/rc1
>> (still TBD) in a few hours. I'll take a look and see if I can push a
>> quick fix before that.
> When you do I'll start a bowerbird run to check it.
Pushed, please test.
I think there's a general issue here of exactly how we want pgwin32_fopen
to behave, but the immediate problem is best fixed by making pgbench deal
with Windows newlines more thoroughly.
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 18:30 Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 24+ messages in thread
From: Andrew Dunstan @ 2018-09-17 18:30 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Michael Paquier <michael@paquier.xyz>; pgsql-committers@lists.postgresql.org
On 09/17/2018 12:13 PM, Tom Lane wrote:
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>> On 09/17/2018 10:48 AM, Tom Lane wrote:
>>> We've got a bit of a timing problem because we want to wrap 11beta4/rc1
>>> (still TBD) in a few hours. I'll take a look and see if I can push a
>>> quick fix before that.
>> When you do I'll start a bowerbird run to check it.
> Pushed, please test.
>
> I think there's a general issue here of exactly how we want pgwin32_fopen
> to behave, but the immediate problem is best fixed by making pgbench deal
> with Windows newlines more thoroughly.
>
>
Tests are still running, but it's past the pgbench stage on HEAD, so I
think we're good.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 22:00 Tom Lane <tgl@sss.pgh.pa.us>
parent: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
0 siblings, 0 replies; 24+ messages in thread
From: Tom Lane @ 2018-09-17 22:00 UTC (permalink / raw)
To: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>; +Cc: Michael Paquier <michael@paquier.xyz>; pgsql-committers@lists.postgresql.org
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 09/17/2018 12:13 PM, Tom Lane wrote:
>> Pushed, please test.
> Tests are still running, but it's past the pgbench stage on HEAD, so I
> think we're good.
I was more concerned about whether any of the post-pgbench steps would
show a failure; but it looks like HEAD's green, so probably v11 is too.
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-17 23:38 Tom Lane <tgl@sss.pgh.pa.us>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 24+ messages in thread
From: Tom Lane @ 2018-09-17 23:38 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-hackers@lists.postgresql.org
So we seem to be out of the woods in terms of 0ba06e0bf breaking the
regression tests, but I'm not very happy about the whole thing, because
that patch wasn't supposed to change the behavior of open/fopen in any
way other than allowing concurrent file access. Obviously, it did.
After looking at src/port/open.c a bit, it seems like the problem is
that our fopen() requires a nonstandard "t" option to select text mode.
I'd always supposed that you got binary mode with "b" and text mode
otherwise; is there some third possibility on Windows?
Anyway, I'm inclined to propose that we ought to do something like
the attached in HEAD and v11 in order to reduce the risk that there
are other unexpected behavioral changes. This should also let us
revert 0ba06e0bf's change in initdb.c.
I wonder whether pgwin32_open() also ought to enforce that either
O_BINARY or O_TEXT mode gets selected.
regards, tom lane
Attachments:
[text/x-diff] force-either-binary-or-text-open-mode.patch (920B, ../../11148.1537227504@sss.pgh.pa.us/2-force-either-binary-or-text-open-mode.patch)
download | inline diff:
diff --git a/src/port/open.c b/src/port/open.c
index a3ad946..85ab06e 100644
--- a/src/port/open.c
+++ b/src/port/open.c
@@ -154,7 +154,7 @@ pgwin32_fopen(const char *fileName, const char *mode)
if (strchr(mode, 'b'))
openmode |= O_BINARY;
- if (strchr(mode, 't'))
+ else
openmode |= O_TEXT;
fd = pgwin32_open(fileName, openmode);
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index b53d6eb..cb8c745 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -491,15 +491,7 @@ readfile(const char *path)
char *buffer;
int c;
-#ifdef WIN32
- /*
- * On Windows, we have to open the file in text mode so that carriage
- * returns are stripped.
- */
- if ((infile = fopen(path, "rt")) == NULL)
-#else
if ((infile = fopen(path, "r")) == NULL)
-#endif
{
fprintf(stderr, _("%s: could not open file \"%s\" for reading: %s\n"),
progname, path, strerror(errno));
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-18 00:11 Michael Paquier <michael@paquier.xyz>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2018-09-18 00:11 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@lists.postgresql.org
On Mon, Sep 17, 2018 at 07:38:24PM -0400, Tom Lane wrote:
> So we seem to be out of the woods in terms of 0ba06e0bf breaking the
> regression tests, but I'm not very happy about the whole thing, because
> that patch wasn't supposed to change the behavior of open/fopen in any
> way other than allowing concurrent file access. Obviously, it did.
Thanks! I can see that as well.
> After looking at src/port/open.c a bit, it seems like the problem is
> that our fopen() requires a nonstandard "t" option to select text mode.
> I'd always supposed that you got binary mode with "b" and text mode
> otherwise; is there some third possibility on Windows?
There is a sort of automatic mode... Please see below.
> Anyway, I'm inclined to propose that we ought to do something like
> the attached in HEAD and v11 in order to reduce the risk that there
> are other unexpected behavioral changes. This should also let us
> revert 0ba06e0bf's change in initdb.c.
>
> I wonder whether pgwin32_open() also ought to enforce that either
> O_BINARY or O_TEXT mode gets selected.
There is no need to go down that road I think, a lot of code paths
already call setmode to make sure that binary or text modes are used.
> diff --git a/src/port/open.c b/src/port/open.c
> index a3ad946..85ab06e 100644
> --- a/src/port/open.c
> +++ b/src/port/open.c
> @@ -154,7 +154,7 @@ pgwin32_fopen(const char *fileName, const char *mode)
>
> if (strchr(mode, 'b'))
> openmode |= O_BINARY;
> - if (strchr(mode, 't'))
> + else
> openmode |= O_TEXT;
Hm. I don't think that this is correct either. The whole logic behind
the mode selected depends on how setmode() is being set to. Please see
here:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setmode?view=vs-2017
And the point is that if neither 'b' nor 't' are set, then open() would
use the value set by the process, which is 't' by default if not set.
And initdb does not set that, so it would use 't'. miscinit.c sets it
to 'b', and your patch would likely cause some backend code to be
broken. So it seems to me that if 'b' or 't' are not defined by the
caller, then we ought to use what get_fmode() returns:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/get-fmode?view=vs-2017
What I think I broke is that CreateFile ignores what _fmode uses, which
has caused the breakage, while calling directly open() or fopen() does
the work. There are also other things assuming that binary mode is
used, you can grep for "setmode" and see how miscinit.c or pg_basebackup
do the job.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../../20180918001143.GG31460@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-18 02:42 Michael Paquier <michael@paquier.xyz>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2018-09-18 02:42 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@lists.postgresql.org
On Tue, Sep 18, 2018 at 09:11:43AM +0900, Michael Paquier wrote:
> What I think I broke is that CreateFile ignores what _fmode uses, which
> has caused the breakage, while calling directly open() or fopen() does
> the work. There are also other things assuming that binary mode is
> used, you can grep for "setmode" and see how miscinit.c or pg_basebackup
> do the job.
I have been playing with this stuff, and hacked the attached. Now, while
TAP tests of initdb and pgbench are happy (I can actually see the past
failure as well), pg_dump complains at authentication time when using
plain-text mode when using databases with all ASCII characters. That's
not something I expected first, but _get_fmode also influences
operations like pipe(), which is something that pg_dump uses, and
setmode is enforced to binary mode only when adapted.
I am getting to wonder if what's present on HEAD represents actually the
best deal we could get. Attached is the patch I used for reference.
Thoughts?
--
Michael
Attachments:
[text/x-diff] win32-open-mode-v1.patch (1.4K, ../../20180918024227.GI31460@paquier.xyz/2-win32-open-mode-v1.patch)
download | inline diff:
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index b53d6eb9cc..cb8c7450d9 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -491,15 +491,7 @@ readfile(const char *path)
char *buffer;
int c;
-#ifdef WIN32
- /*
- * On Windows, we have to open the file in text mode so that carriage
- * returns are stripped.
- */
- if ((infile = fopen(path, "rt")) == NULL)
-#else
if ((infile = fopen(path, "r")) == NULL)
-#endif
{
fprintf(stderr, _("%s: could not open file \"%s\" for reading: %s\n"),
progname, path, strerror(errno));
diff --git a/src/port/open.c b/src/port/open.c
index a3ad946a60..f7a296ef28 100644
--- a/src/port/open.c
+++ b/src/port/open.c
@@ -71,6 +71,28 @@ pgwin32_open(const char *fileName, int fileFlags,...)
_O_SHORT_LIVED | O_DSYNC | O_DIRECT |
(O_CREAT | O_TRUNC | O_EXCL) | (O_TEXT | O_BINARY))) == fileFlags);
+ /*
+ * If caller has set neither O_BINARY nor O_TEXT, then look for what
+ * the default mode is for this process, then enforce it.
+ */
+ if ((fileFlags & (O_BINARY | O_TEXT)) == 0)
+ {
+ int pmode = 0;
+
+ /* only MSVC newer than 2015 support _get_fmode */
+#if (_MSC_VER >= 1900)
+ if (_get_fmode(&pmode) < 0)
+ {
+ /* get_fmode sets errno */
+ return -1;
+ }
+#else
+ pmode = _fmode;
+#endif
+
+ fileFlags |= pmode;
+ }
+
sa.nLength = sizeof(sa);
sa.bInheritHandle = TRUE;
sa.lpSecurityDescriptor = NULL;
[application/pgp-signature] signature.asc (832B, ../../20180918024227.GI31460@paquier.xyz/3-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-18 14:45 Tom Lane <tgl@sss.pgh.pa.us>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 2 replies; 24+ messages in thread
From: Tom Lane @ 2018-09-18 14:45 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-hackers@lists.postgresql.org
Michael Paquier <michael@paquier.xyz> writes:
> On Tue, Sep 18, 2018 at 09:11:43AM +0900, Michael Paquier wrote:
>> What I think I broke is that CreateFile ignores what _fmode uses, which
>> has caused the breakage, while calling directly open() or fopen() does
>> the work. There are also other things assuming that binary mode is
>> used, you can grep for "setmode" and see how miscinit.c or pg_basebackup
>> do the job.
> I have been playing with this stuff, and hacked the attached. Now, while
> TAP tests of initdb and pgbench are happy (I can actually see the past
> failure as well), pg_dump complains at authentication time when using
> plain-text mode when using databases with all ASCII characters. That's
> not something I expected first, but _get_fmode also influences
> operations like pipe(), which is something that pg_dump uses, and
> setmode is enforced to binary mode only when adapted.
> I am getting to wonder if what's present on HEAD represents actually the
> best deal we could get. Attached is the patch I used for reference.
> Thoughts?
Well, we have to do something. I have a report from EDB's packagers
that in 11beta4, "initdb --pwfile" is failing on Windows (ie, one can't
connect afterwards using the specified password). It seems nearly
certain to me that the reason is that the file is read with
FILE *pwf = fopen(pwfilename, "r");
and so the \r isn't getting stripped from what's used as the password.
So my opinion is that it's not negotiable that we have to restore
the previous behavior in this realm. I don't want to be running
around finding other cases one at a time, and I absolutely won't
hold still for putting an "#ifdef WIN32" around each such case.
(Even if we fixed all our own code, we'd still be breaking 3rd-party
code.)
What I don't understand yet is why your latest patch doesn't restore
the previous behavior. What exactly is still different?
In the meantime, we might be well advised to revert this patch in
v11 and just continue to work on the problem in HEAD. I see now
that this wasn't something to cram in during late beta ...
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-18 15:28 Laurenz Albe <laurenz.albe@cybertec.at>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 24+ messages in thread
From: Laurenz Albe @ 2018-09-18 15:28 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-hackers@lists.postgresql.org
Tom Lane wrote:
> Well, we have to do something. I have a report from EDB's packagers
> that in 11beta4, "initdb --pwfile" is failing on Windows (ie, one can't
> connect afterwards using the specified password). It seems nearly
> certain to me that the reason is that the file is read with
>
> FILE *pwf = fopen(pwfilename, "r");
>
> and so the \r isn't getting stripped from what's used as the password.
Perhaps there is something obvious that I'm missing, but it seems that
all the problems we observe are caused by frontend code suddenly defaulting
to binary mode when it was text mode before.
Would it be an option to have pgwin32_open default to text mode in
frontend code and to binary mode in backend code?
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-18 15:38 Tom Lane <tgl@sss.pgh.pa.us>
parent: Laurenz Albe <laurenz.albe@cybertec.at>
0 siblings, 1 reply; 24+ messages in thread
From: Tom Lane @ 2018-09-18 15:38 UTC (permalink / raw)
To: Laurenz Albe <laurenz.albe@cybertec.at>; +Cc: Michael Paquier <michael@paquier.xyz>; pgsql-hackers@lists.postgresql.org
Laurenz Albe <laurenz.albe@cybertec.at> writes:
> Would it be an option to have pgwin32_open default to text mode in
> frontend code and to binary mode in backend code?
Well, the question is why Michael's latest proposed patch doesn't
accomplish that.
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-18 16:04 Laurenz Albe <laurenz.albe@cybertec.at>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 24+ messages in thread
From: Laurenz Albe @ 2018-09-18 16:04 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Michael Paquier <michael@paquier.xyz>; pgsql-hackers@lists.postgresql.org
Tom Lane wrote:
> Laurenz Albe <laurenz.albe@cybertec.at> writes:
> > Would it be an option to have pgwin32_open default to text mode in
> > frontend code and to binary mode in backend code?
>
> Well, the question is why Michael's latest proposed patch doesn't
> accomplish that.
I was thinking of something trivial like this:
--- a/src/port/open.c
+++ b/src/port/open.c
@@ -71,6 +71,12 @@ pgwin32_open(const char *fileName, int fileFlags,...)
_O_SHORT_LIVED | O_DSYNC | O_DIRECT |
(O_CREAT | O_TRUNC | O_EXCL) | (O_TEXT | O_BINARY))) == fileFlags);
+#ifdef FRONTEND
+ /* default to text mode in frontend code */
+ if (fileFlags & O_BINARY == 0)
+ fileFlags |= O_TEXT;
+#endif
+
sa.nLength = sizeof(sa);
sa.bInheritHandle = TRUE;
sa.lpSecurityDescriptor = NULL;
That wouldn't influence pipes, which was what Michael said was a
problem for pg_dump.
I currently have no Windows system close, so I cannot test...
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-18 21:38 Michael Paquier <michael@paquier.xyz>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 24+ messages in thread
From: Michael Paquier @ 2018-09-18 21:38 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@lists.postgresql.org
On Tue, Sep 18, 2018 at 10:45:09AM -0400, Tom Lane wrote:
> In the meantime, we might be well advised to revert this patch in
> v11 and just continue to work on the problem in HEAD. I see now
> that this wasn't something to cram in during late beta ...
I can see that you have reverted the change just before I was able to
reply. Thanks I was going to do the same. Also, if we cannot find a
clear solution for HEAD rather soon, say by tomorrow, let's also remove
it there as well and go ack to the blackboard. I still want to test a
couple of other solutions first.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../../20180918213807.GA1650@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-19 02:32 Michael Paquier <michael@paquier.xyz>
parent: Laurenz Albe <laurenz.albe@cybertec.at>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2018-09-19 02:32 UTC (permalink / raw)
To: Laurenz Albe <laurenz.albe@cybertec.at>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-hackers@lists.postgresql.org
On Tue, Sep 18, 2018 at 06:04:58PM +0200, Laurenz Albe wrote:
> That wouldn't influence pipes, which was what Michael said was a
> problem for pg_dump.
Yeah, the authentication blows up badly on that.. You can see all the
tests using user and database names with all ASCII range turning red.
> I currently have no Windows system close, so I cannot test...
I have spent a large portion of my morning trying to test all the
solutions proposed, and a winner shows up. Attached are three patches
which present all the solutions mentioned, attached here for visibility:
- win32-open-michael.patch, or the solution I was working on. This has
led me actually nowhere. Even trying to enforce _fmode to happen only
on the frontend has caused breakages of pg_dump for authentication.
Trying to be smarter than what other binaries do is nice and consistent
with the Windows experience I think, still it looks that this can lead
to breakages when a utility uses setmode() for some of the output
files. I noticed that pgwin32_open missed to enforce the mode used when
calling _fdmode, still that solves nothing.
- win32-open-tom.patch, which switches pgwin32_fopen() to use text mode
all the time if binary is not specified. While this looked promising,
I have noticed that this has been causing the same set of errors as what
my previous patch has been doing in pg_dump TAP tests. Anyway, a
solution needs also to happen for pgwin32_open() as we want direct calls
to it to get the right call.
- win32-open-laurenz.patch, which enforces to text mode only if binary
mode is not defined, which maps strictly to what pre-11 is doing when
calling the system _open or _fopen. And surprisingly, this is proving
to pass all the tests I ran: bincheck (including pgbench and pg_dump),
upgradecheck, recoverycheck, check, etc. initdb --pwfile is not
complaining to me either.
So the odds are that Laurenz's solution is what we are looking for.
Laurenz, Tom, any thoughts to share? I won't back-patch that ;)
--
Michael
Attachments:
[text/x-diff] win32-open-michael.patch (2.7K, ../../20180919023211.GC1650@paquier.xyz/2-win32-open-michael.patch)
download | inline diff:
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index b53d6eb9cc..cb8c7450d9 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -491,15 +491,7 @@ readfile(const char *path)
char *buffer;
int c;
-#ifdef WIN32
- /*
- * On Windows, we have to open the file in text mode so that carriage
- * returns are stripped.
- */
- if ((infile = fopen(path, "rt")) == NULL)
-#else
if ((infile = fopen(path, "r")) == NULL)
-#endif
{
fprintf(stderr, _("%s: could not open file \"%s\" for reading: %s\n"),
progname, path, strerror(errno));
diff --git a/src/port/open.c b/src/port/open.c
index a3ad946a60..9174cdce66 100644
--- a/src/port/open.c
+++ b/src/port/open.c
@@ -122,11 +122,35 @@ pgwin32_open(const char *fileName, int fileFlags,...)
return -1;
}
+ /*
+ * If caller has set neither O_BINARY nor O_TEXT, then look for what
+ * the default mode is for this process, then enforce it. This can
+ * be done only after opening the file and before switching the file
+ * mode.
+ */
+ if ((fileFlags & (O_BINARY | O_TEXT)) == 0)
+ {
+ int pmode = 0;
+
+ /* only MSVC newer than 2015 support _get_fmode */
+#if (_MSC_VER >= 1900)
+ if (_get_fmode(&pmode) < 0)
+ {
+ /* _get_fmode sets errno */
+ CloseHandle(h);
+ return -1;
+ }
+#else
+ pmode = _fmode;
+#endif
+
+ fileFlags |= pmode;
+ }
+
/* _open_osfhandle will, on error, set errno accordingly */
if ((fd = _open_osfhandle((intptr_t) h, fileFlags & O_APPEND)) < 0)
CloseHandle(h); /* will not affect errno */
- else if (fileFlags & (O_TEXT | O_BINARY) &&
- _setmode(fd, fileFlags & (O_TEXT | O_BINARY)) < 0)
+ else if (_setmode(fd, fileFlags & (O_TEXT | O_BINARY)) < 0)
{
_close(fd);
return -1;
@@ -138,6 +162,7 @@ pgwin32_open(const char *fileName, int fileFlags,...)
FILE *
pgwin32_fopen(const char *fileName, const char *mode)
{
+ char fdmode[32];
int openmode = 0;
int fd;
@@ -160,7 +185,36 @@ pgwin32_fopen(const char *fileName, const char *mode)
fd = pgwin32_open(fileName, openmode);
if (fd == -1)
return NULL;
- return _fdopen(fd, mode);
+
+ strcpy(fdmode, mode);
+
+ /*
+ * Like pgwin32_open, look for the default mode to be used for this
+ * file descriptor. Note that it is important to do that again here
+ * for _fdopen below.
+ */
+ if ((openmode & (O_BINARY | O_TEXT)) == 0)
+ {
+ int pmode = 0;
+
+ /* only MSVC newer than 2015 support _get_fmode */
+#if (_MSC_VER >= 1900)
+ if (_get_fmode(&pmode) < 0)
+ {
+ /* get_fmode sets errno */
+ _close(fd);
+ return NULL;
+ }
+#else
+ pmode = _fmode;
+#endif
+
+ snprintf(fdmode, sizeof(fdmode), "%s%s%s", fdmode,
+ (pmode & O_BINARY) != 0 ? "b" : "",
+ (pmode & O_TEXT) != 0 ? "t" : "");
+ }
+
+ return _fdopen(fd, fdmode);
}
#endif
[text/x-diff] win32-open-laurenz.patch (1.5K, ../../20180919023211.GC1650@paquier.xyz/3-win32-open-laurenz.patch)
download | inline diff:
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index b53d6eb9cc..cb8c7450d9 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -491,15 +491,7 @@ readfile(const char *path)
char *buffer;
int c;
-#ifdef WIN32
- /*
- * On Windows, we have to open the file in text mode so that carriage
- * returns are stripped.
- */
- if ((infile = fopen(path, "rt")) == NULL)
-#else
if ((infile = fopen(path, "r")) == NULL)
-#endif
{
fprintf(stderr, _("%s: could not open file \"%s\" for reading: %s\n"),
progname, path, strerror(errno));
diff --git a/src/port/open.c b/src/port/open.c
index a3ad946a60..63a7a1d2fb 100644
--- a/src/port/open.c
+++ b/src/port/open.c
@@ -71,6 +71,19 @@ pgwin32_open(const char *fileName, int fileFlags,...)
_O_SHORT_LIVED | O_DSYNC | O_DIRECT |
(O_CREAT | O_TRUNC | O_EXCL) | (O_TEXT | O_BINARY))) == fileFlags);
+#ifdef FRONTEND
+ /*
+ * Since PostgreSQL 12, those concurrent-safe versions of open() and
+ * fopen() can be used by frontends, having as side-effect to switch
+ * the file-transaction mode from O_TEXT to O_BINARY if none is
+ * specified. Caller may want to enforce the binary or text mode,
+ * but if nothing is defined make sure that the default mode maps
+ * with what versions older than 12 have been doing.
+ */
+ if ((fileFlags & O_BINARY) == 0)
+ fileFlags |= O_TEXT;
+#endif
+
sa.nLength = sizeof(sa);
sa.bInheritHandle = TRUE;
sa.lpSecurityDescriptor = NULL;
[text/x-diff] win32-open-tom.patch (932B, ../../20180919023211.GC1650@paquier.xyz/4-win32-open-tom.patch)
download | inline diff:
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
index b53d6eb9cc..cb8c7450d9 100644
--- a/src/bin/initdb/initdb.c
+++ b/src/bin/initdb/initdb.c
@@ -491,15 +491,7 @@ readfile(const char *path)
char *buffer;
int c;
-#ifdef WIN32
- /*
- * On Windows, we have to open the file in text mode so that carriage
- * returns are stripped.
- */
- if ((infile = fopen(path, "rt")) == NULL)
-#else
if ((infile = fopen(path, "r")) == NULL)
-#endif
{
fprintf(stderr, _("%s: could not open file \"%s\" for reading: %s\n"),
progname, path, strerror(errno));
diff --git a/src/port/open.c b/src/port/open.c
index a3ad946a60..85ab06e518 100644
--- a/src/port/open.c
+++ b/src/port/open.c
@@ -154,7 +154,7 @@ pgwin32_fopen(const char *fileName, const char *mode)
if (strchr(mode, 'b'))
openmode |= O_BINARY;
- if (strchr(mode, 't'))
+ else
openmode |= O_TEXT;
fd = pgwin32_open(fileName, openmode);
[application/pgp-signature] signature.asc (832B, ../../20180919023211.GC1650@paquier.xyz/5-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-19 15:55 Tom Lane <tgl@sss.pgh.pa.us>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 24+ messages in thread
From: Tom Lane @ 2018-09-19 15:55 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Laurenz Albe <laurenz.albe@cybertec.at>; pgsql-hackers@lists.postgresql.org
Michael Paquier <michael@paquier.xyz> writes:
> I have spent a large portion of my morning trying to test all the
> solutions proposed, and a winner shows up. ...
> - win32-open-laurenz.patch, which enforces to text mode only if binary
> mode is not defined, which maps strictly to what pre-11 is doing when
> calling the system _open or _fopen. And surprisingly, this is proving
> to pass all the tests I ran: bincheck (including pgbench and pg_dump),
> upgradecheck, recoverycheck, check, etc. initdb --pwfile is not
> complaining to me either.
I'm OK with this approach. I wonder though what happens if you take
away the "#ifdef FRONTEND" and just enforce that one or the other mode
is selected always. That would seem like a sensible solution rather
than a wart to me ...
regards, tom lane
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-19 23:58 Michael Paquier <michael@paquier.xyz>
parent: Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 24+ messages in thread
From: Michael Paquier @ 2018-09-19 23:58 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Laurenz Albe <laurenz.albe@cybertec.at>; pgsql-hackers@lists.postgresql.org
On Wed, Sep 19, 2018 at 11:55:01AM -0400, Tom Lane wrote:
> I'm OK with this approach. I wonder though what happens if you take
> away the "#ifdef FRONTEND" and just enforce that one or the other mode
> is selected always. That would seem like a sensible solution rather
> than a wart to me ...
Thanks, I have pushed the solution from Laurenz to maintain pure
compatibility. The origin of the failures reported by pg_dump actually
comes from a mismatch with pg_hba.conf in the way users and/or databases
are parsed. I am glad that we have tests for the full range of ASCII
characters in TAP so as it is easy to detect regressions at early
stages. We could try to manipulate more setmode calls like the one in
miscinit.c so as authentication gets the right call. I am not sure of
other side effects though...
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../../20180919235820.GA1338@paquier.xyz/2-signature.asc)
download
^ permalink raw reply [nested|flat] 24+ messages in thread
* Re: pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi
@ 2018-09-20 03:51 Laurenz Albe <laurenz.albe@cybertec.at>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 24+ messages in thread
From: Laurenz Albe @ 2018-09-20 03:51 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-hackers@lists.postgresql.org
Michael Paquier wrote:
> Thanks, I have pushed the solution from Laurenz to maintain pure
> compatibility.
Thanks for all the work!
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 24+ messages in thread
end of thread, other threads:[~2018-09-20 03:51 UTC | newest]
Thread overview: 24+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2018-09-14 01:05 pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi Michael Paquier <michael@paquier.xyz>
2018-09-14 15:22 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-14 22:49 ` Michael Paquier <michael@paquier.xyz>
2018-09-17 11:41 ` Michael Paquier <michael@paquier.xyz>
2018-09-17 13:41 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-17 14:02 ` Michael Paquier <michael@paquier.xyz>
2018-09-17 14:48 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-17 15:13 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-09-17 16:13 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-17 18:30 ` Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-09-17 22:00 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-17 23:38 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-18 00:11 ` Michael Paquier <michael@paquier.xyz>
2018-09-18 02:42 ` Michael Paquier <michael@paquier.xyz>
2018-09-18 14:45 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-18 15:28 ` Laurenz Albe <laurenz.albe@cybertec.at>
2018-09-18 15:38 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-18 16:04 ` Laurenz Albe <laurenz.albe@cybertec.at>
2018-09-19 02:32 ` Michael Paquier <michael@paquier.xyz>
2018-09-19 15:55 ` Tom Lane <tgl@sss.pgh.pa.us>
2018-09-19 23:58 ` Michael Paquier <michael@paquier.xyz>
2018-09-20 03:51 ` Laurenz Albe <laurenz.albe@cybertec.at>
2018-09-18 21:38 ` Michael Paquier <michael@paquier.xyz>
2018-09-17 11:39 pgsql: Allow concurrent-safe open() and fopen() in frontend code for Wi Michael Paquier <michael@paquier.xyz>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox