Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vzWor-0015U8-0p for pgpool-hackers@arkaria.postgresql.org; Mon, 09 Mar 2026 09:22:33 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vzWop-00FEpW-2O for pgpool-hackers@arkaria.postgresql.org; Mon, 09 Mar 2026 09:22:32 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vzWop-00FEpP-1f for pgpool-hackers@lists.postgresql.org; Mon, 09 Mar 2026 09:22:32 +0000 Received: from mail-yw1-x112b.google.com ([2607:f8b0:4864:20::112b]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.98.2) (envelope-from ) id 1vzWon-00000001m0B-1T6k for pgpool-hackers@lists.postgresql.org; Mon, 09 Mar 2026 09:22:31 +0000 Received: by mail-yw1-x112b.google.com with SMTP id 00721157ae682-79898953985so62718707b3.1 for ; Mon, 09 Mar 2026 02:22:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1773048148; cv=none; d=google.com; s=arc-20240605; b=JTV47L2DWsmBPTcj3Jf9MeSfnuCgQYQf1DL40LUp78hus+6ZC/Bdt96UhVE9ZCutZ2 GLpKLRRz5dMbnXReBzMAjljsi+fD/WJ7sx3MDgH0oLf50OLR2Ztdd7Xw6YNPn3nvMJr/ orfhMXQCrjqVXwk6F21tE2R7VhuDoVI4z5kcil9k8fL/guQ5otEJVYgjCS9ws7afJlCW ABtkS8S18htjtMZiV833FZJwygIjWH4O5HM3gb/SppYkxPtRv37Vg03cKvHw+1JBffuN zZn6U5Rfd83urikFcTBqQpceGjSn6p1wMV2DQE8wXgZDVo/wJ5clcBqcoegKE7g7xDi+ H7RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=GMl9hfLBjg2P/WKZcwjmRQlRnv3IcLMVRfF2hg5Ox28=; fh=m3oiE62fRN5J1XaE+5vpHZRIgCyFefxVI0mFw3d8yxA=; b=cbGXwnIhViURtRtNYYav9pq9J+FDiTfQ+uvMvcyH8uHnpK+Yjs1dgiu7B+jkDEvim8 LXPsZhSHa/ndagqeoA0uJ8aw/n41eCthiCQvqdbAAIc+qQxnAQAon8uDvjdBSjEnV1OO N5hkleHhLIp4zMieBqdVYbzt4fRC0eBzWOtmjoBS8OHGTStWKjuTEu7awF70Lc01lX5J hpF9y9Kw1jDX01Ev0tDCc/ko0K8YbZjo9Wj+JBCeJ89tF9fgCqf0UuuFXa34iTOKqUsc Ut60kqbHfd/hBudFtDwplAlUIsk/MwFohE06pcK2FF5mI4HlPVZrWvcLllXuWJwrZ4YG dFlw==; darn=lists.postgresql.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tailorbrands.com; s=google; t=1773048148; x=1773652948; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=GMl9hfLBjg2P/WKZcwjmRQlRnv3IcLMVRfF2hg5Ox28=; b=Y3dVEcvrQmVu+0wdtO+tr0SOl9SJ9iv8PPFRw2EIbAnA00K4bftmWjpXJ6FqqxuSig xEVkhHrCo+VYbXnY6puii6AkRu7s0g99iy4YE48fjgle4pwc1LMJ5v3LUDoDZH9CKnci nyRcgxUzNFJCnpY7D9QGM+KutqrBU57Gnai1r6HTnKjuI9AfSn9x+hrQQAsTBlpYbXEU P4uHMeppZsh4F5ziGZsmOOXbyONdZrpYq3+V4ULE36mQ96bYaz9j4+bIOPeBkjSO5LB+ wLQwv0sHL7aB3Yunn4GIA+PuNRR3pEAaabF5xMrgmolEMfonVO8rKi9++McfTuEso142 2H4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1773048148; x=1773652948; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=GMl9hfLBjg2P/WKZcwjmRQlRnv3IcLMVRfF2hg5Ox28=; b=qV4z27Ui6lLFuBHEM2h5/EK6RkJql7SeGeJZwvSNQu1BGB8ulvYosGv3s8EX9STVeq FZ+fPG3QdGhDtY/BUj0bTw7QS66YDMaV0Lepb68q31GSrILCcUkaXCzbfP5t58RLuOGt R/8VJ5tkIgi5vqRPDcPhiro8OL7iq2Wq/dWewgj6N+cSacb99SRhkL4BVXWDlonnODM6 SKJb3oPBhz1Vx9Mo/SrT4Pk2cxg1cxdus6VOYFilCAAw6zQktGT8pHhdz+8IHbUxcO2W M0W5oBEDSBjAVYGFHCznWX6noGcw2hgPRvxCi+JnWJvNo773sn2FMm/V7bvP6W4deGsx lX2Q== X-Gm-Message-State: AOJu0YyHxnHPmIG/azofg0GhEPpl3hlGJN7Yhh5IfdIMimrRPF6LFrpG AdWwz5WriNQlB3JouFxg5OdPLhm7pr8cziJjHwC5FxPj/4j29SDS/tIM6p9tM1BC5ghXg8u02SL hkiItHs7rZOyw2VSFrLFBd5L9U6bLm3DXMTKT9OMrRg== X-Gm-Gg: ATEYQzzEK8NCMlFxOR4j8+NG4hhaj4x346uZd0XfmIbrrCt+8H9AgEyyhMBPzf2UFjf Mxa4EKJk0CfKc18P81xBBRKl38B/ZrSYKgJwTENEpVjtDLmwl4PnB1QnYA7mX3+2Pntmr/ggvw6 a5as0KmTFBGb+BMf2XczXq/OgIUsIuSPO+UPmO4fYpc+LZLPO11U2326NwEDsUtzDbPbm0zbfXb RSHs3mkNrkONKQDsp0ZR9ACeV+BiKS6BWSxzxkgN0RcfvixOIaOPKdnDQng8BghzJupUrR619rP PlmPAfQO6qDB9TuaD1ypzymJDXbL6TSr2maY25aUb75MBjKr0Qg9UvMMxWsmllnFMEILqbr8o5Z LvEfL3aaMYVNJ23/rn86Xa5uCyJ2JdTlmgKIR2v2IZyK2znvh X-Received: by 2002:a05:690c:6d83:b0:796:6c1e:c07 with SMTP id 00721157ae682-798dd6ad9c4mr101257187b3.10.1773048147481; Mon, 09 Mar 2026 02:22:27 -0700 (PDT) MIME-Version: 1.0 References: <20260226.164742.1123136102374463742.ishii@postgresql.org> <20260309.141839.397327759559519126.ishii@postgresql.org> In-Reply-To: <20260309.141839.397327759559519126.ishii@postgresql.org> From: Nadav Shatz Date: Mon, 9 Mar 2026 11:22:15 +0200 X-Gm-Features: AaiRm51ceNk9AmADUh8XB5jzf-xXvAqj1Bdt8vFionJtH_2Cz5xwgRFJUMuR4rY Message-ID: Subject: Re: Proposal: Recent mutated table tracking in memory To: Tatsuo Ishii Cc: pgpool-hackers@lists.postgresql.org Content-Type: multipart/alternative; boundary="000000000000003998064c93f114" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000003998064c93f114 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thank you for looking into this, fixing it and getting back to me. Looking forward to your update. On Mon, Mar 9, 2026 at 6:18=E2=80=AFAM Tatsuo Ishii = wrote: > Hi Nadav, > > Sorry for late response. I have been working on this issue. > > > > Added some handling for possible causes - works now. > > > > Unfortunately this doesn't work here. Still 042 test fails if it is > > executed *after* 041. i.e. > > > > ./regress.sh 04[12] <-- 042 fails > > ./regress.sh 042 <-- Ok > > I ran following script to see if some sockets are left after the 041 test= . > > ./regress.sh '041';netstat -ap|grep 11000;./regress.sh 042 > : > testing 041.external_replication_delay...ok. > out of 1 ok:1 failed:0 timeout:0 > (=E4=B8=80=E9=83=A8=E3=81=AE=E3=83=97=E3=83=AD=E3=82=BB=E3=82=B9=E3=81=8C= =E8=AD=98=E5=88=A5=E3=81=95=E3=82=8C=E3=81=BE=E3=81=99=E3=81=8C, =E6=89=80= =E6=9C=89=E3=81=97=E3=81=A6=E3=81=84=E3=81=AA=E3=81=84=E3=83=97=E3=83=AD=E3= =82=BB=E3=82=B9=E3=81=AE=E6=83=85=E5=A0=B1=E3=81=AF > =E8=A1=A8=E7=A4=BA=E3=81=95=E3=82=8C=E3=81=BE=E3=81=9B=E3=82=93=E3=80=82= =E3=81=9D=E3=82=8C=E3=82=89=E5=85=A8=E3=81=A6=E3=82=92=E8=A6=8B=E3=82=8B=E3= =81=AB=E3=81=AF=E3=83=AB=E3=83=BC=E3=83=88=E3=81=AB=E3=81=AA=E3=82=8B=E5=BF= =85=E8=A6=81=E3=81=8C=E3=81=82=E3=82=8A=E3=81=BE=E3=81=99.) > tcp 0 0 0.0.0.0:11000 0.0.0.0:* > LISTEN 1401942/bash > tcp 0 0 localhost:36366 localhost:11000 > TIME_WAIT - > tcp 0 0 localhost:36380 localhost:11000 > TIME_WAIT - > tcp 0 0 localhost:36384 localhost:11000 > TIME_WAIT - > tcp 0 0 localhost:36390 localhost:11000 > TIME_WAIT - > tcp 0 0 localhost:43580 localhost:11000 > TIME_WAIT - > tcp 0 0 localhost:43596 localhost:11000 > TIME_WAIT - > tcp6 0 0 [::]:11000 [::]:* > LISTEN 1401942/bash > unix 2 [ ACC ] STREAM LISTENING 10164557 1401942/bash > /tmp/.s.PGSQL.11000 > creating pgpool-II temporary installation ... > moving pgpool_setup to temporary installation path ... > moving watchdog_setup to temporary installation path ... > using pgpool-II at > /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/temp/ins= talled > ************************* > REGRESSION MODE : install > Pgpool-II version : pgpool-II version 4.8devel (mitsukakeboshi) > Pgpool-II install path : > /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/regression/temp/ins= talled > PostgreSQL bin : /usr/local/pgsql/bin > PostgreSQL Major version : 18 > pgbench : /usr/local/pgsql/bin/pgbench > PostgreSQL jdbc : > /usr/local/pgsql/share/postgresql-9.2-1003.jdbc4.jar > ************************* > testing 042.track_table_mutation...failed. > out of 1 ok:0 failed:1 timeout:0 > > It seems the cause of the issue is the bash process: > > unix 2 [ ACC ] STREAM LISTENING 10164557 1401942/bash > /tmp/.s.PGSQL.11000 > > It keeps on listening to the socket even after the test and it > prevents pgpool in 042 test from binding the socket, which causes the > test failure. Possible solutions are: > > 1) fix the external replication delay checking to close the listening > socket before starting bash. > > 2) close the listening socket when the streaming replication check > worker process forks. > > While invensting the issue, I found similar problem is in other > places. For example, pcp process inherits pgpool listening sockets > which are not necessary for pcp. I posted a proposal to fix the issue: > > > https://www.postgresql.org/message-id/20260302.100028.1346768433787074248= .ishii%40postgresql.org > > This includes the fix #2. I plan to commit the patch today. After our > buildfarm reports no new problem (it will take 2- 3 days), I am going > to test your patch again. > > Best regards, > -- > Tatsuo Ishii > SRA OSS K.K. > English: http://www.sraoss.co.jp/index_en/ > Japanese:http://www.sraoss.co.jp > --=20 Nadav Shatz Tailor Brands | CTO --000000000000003998064c93f114 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thank you for looking into this, fixing it and getting bac= k to me.

Looking forward to your update.

=
On Mon, Mar 9, 2026 at 6:18=E2=80=AFAM Tatsuo Ishii <ishii@postgresql.org> wrote:
Hi Nadav,

Sorry for late response. I have been working on this issue.

> > Added some handling for possible causes - works now.
>
> Unfortunately this doesn't work here. Still 042 test fails if it i= s
> executed *after* 041.=C2=A0 i.e.
>
> ./regress.sh 04[12]=C2=A0 =C2=A0<-- 042 fails
> ./regress.sh 042=C2=A0 =C2=A0 =C2=A0 <-- Ok

I ran following script to see if some sockets are left after the 041 test.<= br>
./regress.sh '041';netstat -ap|grep 11000;./regress.sh 042
:
testing 041.external_replication_delay...ok.
out of 1 ok:1 failed:0 timeout:0
(=E4=B8=80=E9=83=A8=E3=81=AE=E3=83=97=E3=83=AD=E3=82=BB=E3=82=B9=E3=81=8C= =E8=AD=98=E5=88=A5=E3=81=95=E3=82=8C=E3=81=BE=E3=81=99=E3=81=8C, =E6=89=80= =E6=9C=89=E3=81=97=E3=81=A6=E3=81=84=E3=81=AA=E3=81=84=E3=83=97=E3=83=AD=E3= =82=BB=E3=82=B9=E3=81=AE=E6=83=85=E5=A0=B1=E3=81=AF
=E8=A1=A8=E7=A4=BA=E3=81=95=E3=82=8C=E3=81=BE=E3=81=9B=E3=82=93=E3=80=82=E3= =81=9D=E3=82=8C=E3=82=89=E5=85=A8=E3=81=A6=E3=82=92=E8=A6=8B=E3=82=8B=E3=81= =AB=E3=81=AF=E3=83=AB=E3=83=BC=E3=83=88=E3=81=AB=E3=81=AA=E3=82=8B=E5=BF=85= =E8=A6=81=E3=81=8C=E3=81=82=E3=82=8A=E3=81=BE=E3=81=99.)
tcp=C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 0 0.0.0.0:11000=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.0.0.0:*=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0LISTEN=C2=A0 =C2=A0 =C2=A0 1401942/bash=C2=A0 =C2= =A0 =C2=A0 =C2=A0
tcp=C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 0 localhost:36366=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0localhost:11000=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0TIME_WAIT=C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0
tcp=C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 0 localhost:36380=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0localhost:11000=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0TIME_WAIT=C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0
tcp=C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 0 localhost:36384=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0localhost:11000=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0TIME_WAIT=C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0
tcp=C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 0 localhost:36390=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0localhost:11000=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0TIME_WAIT=C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0
tcp=C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 0 localhost:43580=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0localhost:11000=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0TIME_WAIT=C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0
tcp=C2=A0 =C2=A0 =C2=A0 =C2=A0 0=C2=A0 =C2=A0 =C2=A0 0 localhost:43596=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0localhost:11000=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0TIME_WAIT=C2=A0 =C2=A0-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0
tcp6=C2=A0 =C2=A0 =C2=A0 =C2=A00=C2=A0 =C2=A0 =C2=A0 0 [::]:11000=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 [::]:*=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 LISTEN=C2=A0 =C2=A0 =C2=A0 1401942/bash= =C2=A0 =C2=A0 =C2=A0 =C2=A0
unix=C2=A0 2=C2=A0 =C2=A0 =C2=A0 [ ACC ]=C2=A0 =C2=A0 =C2=A0STREAM=C2=A0 = =C2=A0 =C2=A0LISTENING=C2=A0 =C2=A0 =C2=A010164557 1401942/bash=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0/tmp/.s.PGSQL.11000
creating pgpool-II temporary installation ...
moving pgpool_setup to temporary installation path ...
moving watchdog_setup to temporary installation path ...
using pgpool-II at /home/t-ishii/work/Pgpool-II/current/pgpool2/src/test/re= gression/temp/installed
*************************
REGRESSION MODE=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : install
Pgpool-II version=C2=A0 =C2=A0 =C2=A0 =C2=A0 : pgpool-II version 4.8devel (= mitsukakeboshi)
Pgpool-II install path=C2=A0 =C2=A0: /home/t-ishii/work/Pgpool-II/current/p= gpool2/src/test/regression/temp/installed
PostgreSQL bin=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0: /usr/local/pgsql/b= in
PostgreSQL Major version : 18
pgbench=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : /us= r/local/pgsql/bin/pgbench
PostgreSQL jdbc=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : /usr/local/pgsql/share/= postgresql-9.2-1003.jdbc4.jar
*************************
testing 042.track_table_mutation...failed.
out of 1 ok:0 failed:1 timeout:0

It seems the cause of the issue is the bash process:

unix=C2=A0 2=C2=A0 =C2=A0 =C2=A0 [ ACC ]=C2=A0 =C2=A0 =C2=A0STREAM=C2=A0 = =C2=A0 =C2=A0LISTENING=C2=A0 =C2=A0 =C2=A010164557 1401942/bash=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0/tmp/.s.PGSQL.11000

It keeps on listening to the socket even after the test and it
prevents pgpool in 042 test from binding the socket, which causes the
test failure. Possible solutions are:

1) fix the external replication delay checking to close the listening
socket before starting bash.

2) close the listening socket when the streaming replication check
worker process forks.

While invensting the issue, I found similar problem is in other
places. For example, pcp process inherits pgpool listening sockets
which are not necessary for pcp. I posted a proposal to fix the issue:

http= s://www.postgresql.org/message-id/20260302.100028.1346768433787074248.ishii= %40postgresql.org

This includes the fix #2.=C2=A0 I plan to commit the patch today. After our=
buildfarm reports no new problem (it will take 2- 3 days), I am going
to test your patch again.

Best regards,
--
Tatsuo Ishii
SRA OSS K.K.
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp


--
Nadav Shatz
<= font color=3D"#000000">Tailor Brands=C2=A0| CTO
--000000000000003998064c93f114--