Content-Type: multipart/alternative;
 boundary="------------V30t9L4axN7zjltRxzK0NGeV"
Message-ID: <fa5d7bf1-4faa-467c-8897-fa4ea4043f38@dunslane.net>
Date: Fri, 27 Feb 2026 07:12:52 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Non-text mode for pg_dumpall
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: jian he <jian.universality@gmail.com>,
 Mahendra Singh Thalor <mahi6run@gmail.com>,
 tushar <tushar.ahuja@enterprisedb.com>,
 Vaibhav Dalvi <vaibhav.dalvi@enterprisedb.com>,
 pgsql-hackers@lists.postgresql.org
References: <3f22a8bb-29e8-40cc-97a1-309181da2c13@dunslane.net>
 <CAC6VRoY-1WC_O5UDwKzNc18LXLwjuTFb+qMNY3gaF3cCov6M6g@mail.gmail.com>
 <CAKYtNArc3j8Akb+nB1QyJLKic7ej7vKZqKnv_mbB+iQAJRzLnA@mail.gmail.com>
 <CAC6VRoat6NPcS6vfGrPrwaQB3G1JHLSqDeAkf1AJ7npWrwhnUQ@mail.gmail.com>
 <CAKYtNAp=TFEZWQbTRZso3kNaD9BH=-JPNQ30mT60cJN_04hEog@mail.gmail.com>
 <4022765f-38ee-48a3-b246-615b3f8e1c23@dunslane.net>
 <CAKYtNAq73_UeL4-0wgu9rawctHdS2zhpzWtqVuOChCS3M+2Ymg@mail.gmail.com>
 <CACJufxFs9NXXTeb78i2MD+pnZootayAtiGcNHNeU35SmQgMjbA@mail.gmail.com>
 <59d3616f-6d6d-40d5-87e2-e019e350b52d@dunslane.net>
 <CACJufxEnmJ8otfmN7Kp110Wqi=M4YE0-zn-W6SK+eTpWuZw_fg@mail.gmail.com>
 <CAKYtNAqxjQMLVPfz6XESqszhxZKgigi7UEBzBhz_Wj_Bnasdag@mail.gmail.com>
 <7faf3a59-cfe1-45cb-a972-55e05560b414@dunslane.net>
 <CACJufxE-zJiBAaZABtJBJr8FZdj=xP5adf_excmu9S6Op=KpLA@mail.gmail.com>
 <2e7ada75-422c-4153-9437-ea0ce8d63521@dunslane.net>
 <84764ef6-2bcb-43f7-9037-d7! b589857da9@dunslane.net>
 <740033.1772142754@sss.pgh.pa.us>
From: Andrew Dunstan <andrew@dunslane.net>
Content-Language: en-US
Autocrypt: addr=andrew@dunslane.net; keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAlwQTAQgAQQIbAwIX
 gAIZAQULCQgHAwUVCgkICwUWAgMBAAIeBRYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNGd
 BQkdEO8nAAoJEJn6f8tZ/DuBq74H/jkTR4Zi3stbw+xC7v2u3QozssK7MYPL2AsVfh7OealS
 h182fiWXpfvmmAB7WUHbhk9GC2RAOnHI/2d2jgKaMLAHsGYOT0YopTVIwRY43fCw/mK67yxc
 wmDcX+zyKfLaivNbf5A7QPLNwda98bEAMSJ8Sn652Uc6cA8t3uKGsVzbRBQOoYzjgvBCfSrE
 9ql3PDNg0l4BfAqabd2f70ZUm9VAMEPrgv/v2xI7M2XiL4g5BVmqLCOwxLM8RMCotCuoweUr
 VO43DeBCIDwLxotMJKvGWDjBzQYlU1NPUAtNcz/gN9ITUe1VUGjyvGj4u1lxBOcQQUw7l1+T
 5moZ4iZxXzvOwE0ETspYWQEIANGc4zQULOxhbqO2dyD51YhqCNRmm9oKWaqf+wmW4tpDe/VV
 cxAnNizd4LWCHfzpb5cHAtGkOPePMfzWVf6nvdF7d3eglbtf59+zG7O7llV0xSSoFiieQBsr
 GvqDInXYX/4mRRXMtyhM353/tixC9RWLs1oofyYmCPPXXY7h9R7en3B8BoVrRFcdzlIY/NFN
 hFGW/9dkEiGjgna2Rk6e15kln4ZvFBWUg23p93w/pqXcxY6+k/8TEk+C4R+M6w7o2PLGOjdZ
 +kPiUcw5H85zf/yZJwQXzisXaNduwWB6Vads9YC9dj6kPR1c4VGRqAaYL++LAEOqrlvm2Tvq
 QqZRtnEAEQEAAcLAfAQYAQgAJgIbDBYhBOQ+WEYd/Hy/RGkVpZn6f8tZ/DuBBQJoGNI2BQkd
 EODdAAoJEJn6f8tZ/DuBfw0IAKTsfD40teP/pp+bsLLMSxPXUYrrprTj7WFB5v61p6dkpSr/
 qXmMlyahdxQFaPmfVgVirB1Vk/kHiWNnnGjfUV9nB2Zg9LI0Xb9/ts3LsUiRWXzG3tkMY6XL
 vsVOxW4XFRND9l2q+WW93aZ1DZl+fqWfYgMvsusFRhmGFOKTRfKPta2Pkv+AhA24N4+PrR5p
 bU4k2MO8PAGiK8eaYKGFG1bHKuAvoDoF7WXJ3FHxuWqLnKEt4dfOLm5pAe3zq1Lt6q8azT9i
 QWGpSAK5vQUWQHBHpiDjdPeqKZ6HiAXIIKfSmb+jrvXBqoP+D6/K7rUjG2aXiRtTIAXms9sm
 VRu7cmw=
In-Reply-To: <740033.1772142754@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/fa5d7bf1-4faa-467c-8897-fa4ea4043f38%40dunslane.net>
Precedence: bulk

This is a multi-part message in MIME format.
--------------V30t9L4axN7zjltRxzK0NGeV
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


On 2026-02-26 Th 4:52 PM, Tom Lane wrote:
> https://www.postgresql.org/message-id/2fb787be-79f2-4161-8ba4-24e8cab019ac%40dunslane.netAndrew Dunstan<andrew@dunslane.net> writes:
>> pushed with a slight tweak.
> BF member pollock isn't pleased with 007_pg_dumpall:
>
> 026-02-26 20:18:10.389 UTC [14469:1] FATAL:  could not create semaphores: No space left on device
> 2026-02-26 20:18:10.389 UTC [14469:2] DETAIL:  Failed system call was semget(668039, 17, 03600).
> 2026-02-26 20:18:10.389 UTC [14469:3] HINT:  This error does *not* mean that you have run out of disk space.  It occurs when either the system limit for the maximum number of semaphore sets (SEMMNI), or the system wide maximum number of semaphores (SEMMNS), would be exceeded.  You need to raise the respective kernel parameter.  Alternatively, reduce PostgreSQL's consumption of semaphores by reducing its "max_connections" parameter.
>
> It looks to me like this is happening because the script creates a
> boatload of postmasters and doesn't bother to shut any of them down
> (until that happens implicitly at script end).  That seems rather
> unfriendly to small BF machines in the first place, as well as for
> installations that might try to run multiple TAP scripts in parallel.
>
> It's probably eating an undue amount of disk space, as well.
>
> Is there a reason why the "foreach my $run (sort keys %pgdumpall_runs)"
> loop leaves the $target_nodes running, instead of cleaning each one
> up at the bottom of the loop?


No, I think it's just an oversight. I'll add this at the bottom of the loop:

+   $target_node->stop;
+   $target_node->clean_node;

More generally, perhaps we should have a DESTROY method in Cluster.pm 
that would call the stop method when an object goes out of scope.


cheers


andrew


--
Andrew Dunstan
EDB:https://www.enterprisedb.com

--------------V30t9L4axN7zjltRxzK0NGeV
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 2026-02-26 Th 4:52 PM, Tom Lane
      wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:740033.1772142754@sss.pgh.pa.us">
      <pre wrap="" class="moz-quote-pre"><a class="moz-txt-link-freetext" href="https://www.postgresql.org/message-id/2fb787be-79f2-4161-8ba4-24e8cab019ac%40dunslane.netAndrew">https://www.postgresql.org/message-id/2fb787be-79f2-4161-8ba4-24e8cab019ac%40dunslane.netAndrew</a> Dunstan <a class="moz-txt-link-rfc2396E" href="mailto:andrew@dunslane.net">&lt;andrew@dunslane.net&gt;</a> writes:
</pre>
      <blockquote type="cite">
        <pre wrap="" class="moz-quote-pre">pushed with a slight tweak.
</pre>
      </blockquote>
      <pre wrap="" class="moz-quote-pre">
BF member pollock isn't pleased with 007_pg_dumpall:

026-02-26 20:18:10.389 UTC [14469:1] FATAL:  could not create semaphores: No space left on device
2026-02-26 20:18:10.389 UTC [14469:2] DETAIL:  Failed system call was semget(668039, 17, 03600).
2026-02-26 20:18:10.389 UTC [14469:3] HINT:  This error does *not* mean that you have run out of disk space.  It occurs when either the system limit for the maximum number of semaphore sets (SEMMNI), or the system wide maximum number of semaphores (SEMMNS), would be exceeded.  You need to raise the respective kernel parameter.  Alternatively, reduce PostgreSQL's consumption of semaphores by reducing its "max_connections" parameter.

It looks to me like this is happening because the script creates a
boatload of postmasters and doesn't bother to shut any of them down
(until that happens implicitly at script end).  That seems rather
unfriendly to small BF machines in the first place, as well as for
installations that might try to run multiple TAP scripts in parallel.

It's probably eating an undue amount of disk space, as well.

Is there a reason why the "foreach my $run (sort keys %pgdumpall_runs)"
loop leaves the $target_nodes running, instead of cleaning each one
up at the bottom of the loop?</pre>
    </blockquote>
    <p><br>
    </p>
    <p>No, I think it's just an oversight. I'll add this at the bottom
      of the loop:</p>
    <p>+   $target_node-&gt;stop;<br>
      +   $target_node-&gt;clean_node;<br>
    </p>
    <p>More generally, perhaps we should have a DESTROY method in
      Cluster.pm that would call the stop method when an object goes out
      of scope.</p>
    <p><br>
    </p>
    <p>cheers</p>
    <p><br>
    </p>
    <p>andrew<br>
    </p>
    <p><br>
    </p>
    <p><span style="white-space: pre-wrap">
</span></p>
    <pre class="moz-signature"
    signature-switch-id="d0437855-2267-4610-80b3-83167ec45b0b" cols="72">--
Andrew Dunstan
EDB: <a class="moz-txt-link-freetext" href="https://www.enterprisedb.com">https://www.enterprisedb.com</a>
</pre>
  </body>
</html>

--------------V30t9L4axN7zjltRxzK0NGeV--