Date: Sun, 16 Feb 2025 10:47:40 -0800
From: Noah Misch <noah@leadboat.com>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers@postgresql.org
Subject: Re: BackgroundPsql swallowing errors on windows
Message-ID: <20250216184740.55.nmisch@google.com>
References: <wmovm6xcbwh7twdtymxuboaoarbvwj2haasd3sikzlb3dkgz76@n45rzycluzft>
 <20250216173943.b6.nmisch@google.com>
 <h6t2voqv6vm4xk4oh5q6og676h7ehv2opbvr2id7m2xot3adh5@t7su2dmuo3bs>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <h6t2voqv6vm4xk4oh5q6og676h7ehv2opbvr2id7m2xot3adh5@t7su2dmuo3bs>
User-Agent: Mutt/2.2.12 (2023-09-09)
Archived-At: <https://www.postgresql.org/message-id/20250216184740.55.nmisch%40google.com>
Precedence: bulk

On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote:
> On 2025-02-16 09:39:43 -0800, Noah Misch wrote:
> > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote:
> > > I suspect what's happening is that the communication with the
> > > external process allows for reordering between stdout/stderr.
> > > 
> > > And indeed, changing BackgroundPsql::query() to emit the banner on both stdout
> > > and stderr and waiting on both seems to fix the issue.
> > 
> > That makes sense.  I wondered how one might fix IPC::Run to preserve the
> > relative timing of stdout and stderr, not perturbing the timing the way that
> > disrupted your test run.  I can think of two strategies:
> > 
> > - Remove the proxy.
> > 
> > - Make pipe data visible to Perl variables only when at least one of the
> >   proxy<-program_under_test pipes had no data ready to read.  In other words,
> >   if both pipes have data ready, make all that data visible to Perl code
> >   simultaneously.  (When both the stdout pipe and the stderr pipe have data
> >   ready, one can't determine data arrival order.)
> > 
> > Is there a possibly-less-invasive change that might work?
> 
> I don't really know enough about IPC::Run's internals to answer. My
> interpretation of how it might work, purely from observation, is that it opens
> one tcp connection for each "pipe" and that that's what's introducing the
> potential of reordering, as the different sockets can have different delivery
> timeframes.

Right.

> If that's it, it seems proxying all the pipes through one
> connection might be an option.

It would.  Thanks.  However, I think that would entail modifying the program
under test to cooperate with the arrangement.  When running an ordinary
program that does write(1, ...) and write(2, ...), the read end needs some way
to deal with the uncertainty about which write happened first.  dup2(1, 2)
solves the order ambiguity, but it loses other signal.

> I did briefly look at IPC::Run's code, but couldn't figure out how it all fits
> together quickly enough...

We can ignore what IPC::Run does today.  The reordering is a general problem
of proxying >1 pipe.  (Proxying on non-Windows would have the same problem.)