public inbox for [email protected]  

help / color / mirror / Atom feed

From: Thomas Munro <[email protected]>
To: Andres Freund <[email protected]>
Cc: Noah Misch <[email protected]>
Cc: [email protected]
Subject: Re: BackgroundPsql swallowing errors on windows
Date: Mon, 17 Feb 2025 15:18:01 +1300
Message-ID: <CA+hUKGLCeNBwFwWiJ2yk75Oz4G_+DeQd-xaa4CsumWg129SfKw@mail.gmail.com> (raw)
In-Reply-To: <h6t2voqv6vm4xk4oh5q6og676h7ehv2opbvr2id7m2xot3adh5@t7su2dmuo3bs>
References: <wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft>
	<[email protected]>
	<h6t2voqv6vm4xk4oh5q6og676h7ehv2opbvr2id7m2xot3adh5@t7su2dmuo3bs>

On Mon, Feb 17, 2025 at 7:02 AM Andres Freund <[email protected]> wrote:
> I don't really know enough about IPC::Run's internals to answer. My
> interpretation of how it might work, purely from observation, is that it opens
> one tcp connection for each "pipe" and that that's what's introducing the
> potential of reordering, as the different sockets can have different delivery
> timeframes.  If that's it, it seems proxying all the pipes through one
> connection might be an option.

I had a couple of ideas about how to get rid of the intermediate
subprocess.  Obviously it can't convert "two pipes are ready" into two
separate socket send() calls that preserve the original order, as it
doesn't know them (unless perhaps it switches to completion-based
I/O).  But really, the whole design is ugly and slow.  If we have some
capacity to improve Run::IPC, I think we should try to get rid of the
pipe/socket bridge and plug either a pipe or a socket directly into
the target subprocess.  But which one?

1.  Pipes only:  Run::IPC could use IOCP or WaitForMultipleEvents()
instead of select()/poll().
2.  Sockets only: Apparently you can give sockets directly to
subprocesses as stdin/stdout/stderr:

https://stackoverflow.com/questions/4993119/redirect-io-of-process-to-windows-socket

The Run::IPC comments explain that the extra process was needed to be
able to forward all data even if the target subprocess exits without
closing the socket (the linger stuff we have met before in PostgreSQL
itself).  I suspect that if we went that way, maybe asynchronous I/O
would fix that too (see my other thread with guesses and demos on that
topic), but it might not be race-free.  I don't know.  I'd like to
know for PostgreSQL's own sake, but for Run::IPC I think I'd prefer
option 1 anyway: if you have to write new native Windows API
interactions either way, you might as well go with the normal native
way for Windows processes to connect standard I/O streams.

view thread (21+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: BackgroundPsql swallowing errors on windows
  In-Reply-To: <CA+hUKGLCeNBwFwWiJ2yk75Oz4G_+DeQd-xaa4CsumWg129SfKw@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox