Content-Type: multipart/mixed; boundary="------------Hpww0925dyNys0fXxqtrWEdn"
Message-ID: <947e4acd-94b9-4990-83f5-874c71d69b17@dunslane.net>
Date: Tue, 4 Mar 2025 17:51:59 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: scalability bottlenecks with (many) partitions (and more)
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andres Freund <andres@anarazel.de>, Tomas Vondra <tomas@vondra.me>,
 Jakub Wartak <jakub.wartak@enterprisedb.com>,
 Robert Haas <robertmhaas@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
References: <7c1eeafb-2375-4ff6-8469-0640d52d44ed@vondra.me>
 <ffiwtzc6vedo6wb4gbwelon5nefqg675t5c7an2ta7pcz646cg@qwmkdb3l4ett>
 <bfca1010-e5f1-4e36-a461-61b6aad60c47@vondra.me>
 <5xahykogf2sg3ofdqk2li7xx3t2vh23m42jnvfwxoi5fhh7iya@girlhoxdxjzc>
 <486baeb2-2adc-48d2-ba9d-13c4ae43301c@vondra.me>
 <vgtms7b5egxdypmknxpobocqton3puhprk3ahhk45rcx4uyl33@6uetxejsc75r>
 <733e72fa-5fc2-48fc-ab1f-1d8f1ab387c3@vondra.me>
 <d5499660-1541-4749-aff4-6be88be76d99@vondra.me>
 <nubyc3twqd7mtuyc2rbbt2yazjzbat6mlpfgxtb2ppv2egci3g@uvptf7xmh46m>
 <193453.1741123834@sss.pgh.pa.us>
 <qdvrgej743lcddjnb4lzyakt3f5bs2oz34r274wbmgc2g4474q@utqqrqiih65h>
 <197167.1741125678@sss.pgh.pa.us>
 <8a1a5c10-0644-4ea0-a9e8-73d3a2908e18@dunslane.net>
 <200085.1741127335@sss.pgh.pa.us>
From: Andrew Dunstan <andrew@dunslane.net>
Content-Language: en-US
Autocrypt: addr=andrew@dunslane.net; keydata=
 xsBNBE7KWFkBCAClridxur2AIc7eW2AR7izbfp3EnNefie2HbLF0izW5Ik5UjX2HBXBx4syI
 gY6b0ugohXrr274+baoAlvSbq6cAoQuEVrk5IZFzt20b1Xkx65FwGSEj526yiKLocqkJceSq
 Xr9xcA5SGY+FZv441chh5SU92v4q6z+6LPpoHOh97ptAVXZYNTtU0LevyvD5lja0TzbvJm6C
 eFXitJfnm1pLEr0DGJCR/iUOl/N62Kh4855zZC7NHIjQHPOvV5Stz/l5ilDhvGVk+xkXFPys
 SjZoUr1rXhYLpiyi5sR0X9FHXT0KnGuz1F5ERO7ZTLSSQ6fJwPj6gOk9K+vvoKvoeql5ABEB
 AAHNJEFuZHJldyBEdW5zdGFuIDxhbmRyZXdAZHVuc2xhbmUubmV0PsLAmwQTAQgARQIbAwIX
 gAIZAQULCQgHAgMiAgEGFQoJCAsCBBYCAwECHgcWIQTkPlhGHfx8v0RpFaWZ+n/LWfw7gQUC
 ZFlxxwUJGVGAbgAKCRCZ+n/LWfw7gXikB/9ZdcUy6CTBFIIuL/bVsc1eLEW/gJBjJBF6HxNY
 xgEkAgXAp4Lg4A5U+QB9GouFr7+GYxF0BU4hzoGhNPUWltxnHdMWP8nC/38LAqgMi8L/bbsm
 HW5YPBdWYaAZAPJQVfOAgjTbRUb26KSprpyrrJKW0ZmrZfjhNPcQ72jpWzoPLQqx2X6B0fru
 1jq+cBh8lb6r1mJTim1T3JIn+F/v5VpdQS+EL8xqsHkfzKjIPsW3CIXpkypSk6saA55Rkkbl
 26AW8ftPVB0Q6Lnn6FLt9CP0MGNixBQ55yq8r1K+nCBvCCjvQjM8RDm0UUum0WNl+ifQgTLO
 E8TWEnwVtkBf+3QWzsBNBE7KWFkBCADRnOM0FCzsYW6jtncg+dWIagjUZpvaClmqn/sJluLa
 Q3v1VXMQJzYs3eC1gh386W+XBwLRpDj3jzH81lX+p73Re3d3oJW7X+ffsxuzu5ZVdMUkqBYo
 nkAbKxr6gyJ12F/+JkUVzLcoTN+d/7YsQvUVi7NaKH8mJgjz112O4fUe3p9wfAaFa0RXHc5S
 GPzRTYRRlv/XZBIho4J2tkZOnteZJZ+GbxQVlINt6fd8P6al3MWOvpP/ExJPguEfjOsO6Njy
 xjo3WfpD4lHMOR/Oc3/8mScEF84rF2jXbsFgelWnbPWAvXY+pD0dXOFRkagGmC/viwBDqq5b
 5tk76kKmUbZxABEBAAHCwHwEGAEIACYCGwwWIQTkPlhGHfx8v0RpFaWZ+n/LWfw7gQUCZFlx
 5wUJGVGAjgAKCRCZ+n/LWfw7gf+iB/4g8CPY5jihf5r/8EsoIGe2H+dpVmpPF8YGBzTIvCz/
 fQoOq8AX/pE76QEuFnFZWfjw+wgBXgCVmkox2Eflkk6z4ND3pcwGZ6CfCxTQCDk/dij+2DQ4
 6bmDCy/sBgcbz9mTpoLC11HLoPae6YN9nBNQRZDcEFEu54OaVOqlIdbA6m+POIBCXZdHOFc0
 WoDTgxHRzC1jgQNidyd6tKqcsVJs0dzF0oKTmFFmUAqTdJO12LBuNA1rlqrR3EtpYk8B/wtS
 5dIMD7Q8hwQpL+4C6GNpb6ZKnPkLi47pDOLhz2qBrqN+rqUEsT3YnExYpzj5yOBi+FlmV1Hw
 49QYe1sn2ZPs
In-Reply-To: <200085.1741127335@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/947e4acd-94b9-4990-83f5-874c71d69b17%40dunslane.net>
Precedence: bulk

This is a multi-part message in MIME format.
--------------Hpww0925dyNys0fXxqtrWEdn
Content-Type: multipart/alternative;
 boundary="------------eh1KAHojeBBk6AfzpSs0z1Mo"

--------------eh1KAHojeBBk6AfzpSs0z1Mo
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit


On 2025-03-04 Tu 5:28 PM, Tom Lane wrote:
> Andrew Dunstan<andrew@dunslane.net> writes:
>> I think I found a logic bug. Testing.
> Not sure what you are looking at, but I was trying to fix it
> by making the loop over test modules skip unbuilt modules,
> borrowing the test you added in v19 to skip unbuilt contrib
> modules.  It's a little more complicated for the other modules
> because some of them have no .c files to be built, and I could
> not get that to work.  I eventually concluded that there's
> something wrong with the "scalar glob()" idiom you used.
> A bit of googling suggested "grep -e, glob()" instead, and
> that seems to work for me.  sifaka seems happy with the
> attached patch.


I'm looking at something else, namely the attached.


Will check your patch out too.


--
Andrew Dunstan
EDB:https://www.enterprisedb.com

--------------eh1KAHojeBBk6AfzpSs0z1Mo
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 2025-03-04 Tu 5:28 PM, Tom Lane
      wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:200085.1741127335@sss.pgh.pa.us">
      <pre wrap="" class="moz-quote-pre">Andrew Dunstan <a class="moz-txt-link-rfc2396E" href="mailto:andrew@dunslane.net">&lt;andrew@dunslane.net&gt;</a> writes:
</pre>
      <blockquote type="cite">
        <pre wrap="" class="moz-quote-pre">I think I found a logic bug. Testing.
</pre>
      </blockquote>
      <pre wrap="" class="moz-quote-pre">
Not sure what you are looking at, but I was trying to fix it
by making the loop over test modules skip unbuilt modules,
borrowing the test you added in v19 to skip unbuilt contrib
modules.  It's a little more complicated for the other modules
because some of them have no .c files to be built, and I could
not get that to work.  I eventually concluded that there's
something wrong with the "scalar glob()" idiom you used.
A bit of googling suggested "grep -e, glob()" instead, and
that seems to work for me.  sifaka seems happy with the
attached patch.</pre>
    </blockquote>
    <p><br>
    </p>
    <p>I'm looking at something else, namely the attached.</p>
    <p><br>
    </p>
    <p>Will check your patch out too.<br>
    </p>
    <p><br>
    </p>
    <p><br>
    </p>
    <p><span style="white-space: pre-wrap">
</span><span style="white-space: pre-wrap">
</span></p>
    <pre class="moz-signature"
    signature-switch-id="d0437855-2267-4610-80b3-83167ec45b0b" cols="72">--
Andrew Dunstan
EDB: <a class="moz-txt-link-freetext" href="https://www.enterprisedb.com">https://www.enterprisedb.com</a>
</pre>
  </body>
</html>

--------------eh1KAHojeBBk6AfzpSs0z1Mo--
--------------Hpww0925dyNys0fXxqtrWEdn
Content-Type: text/x-patch; charset=UTF-8; name="bfdirfix.patch"
Content-Disposition: attachment; filename="bfdirfix.patch"
Content-Transfer-Encoding: base64

ZGlmZiAtLWdpdCBhL1BHQnVpbGQvVXRpbHMucG0gYi9QR0J1aWxkL1V0aWxzLnBtCmluZGV4
IGI5N2RlOTIuLmQ5NmRjZWMgMTAwNjQ0Ci0tLSBhL1BHQnVpbGQvVXRpbHMucG0KKysrIGIv
UEdCdWlsZC9VdGlscy5wbQpAQCAtNDE3LDcgKzQxNyw3IEBAIHN1YiBjaGVja19pbnN0YWxs
X2lzX2NvbXBsZXRlCiAJCSR0bXBfbG9jID0gIiR0bXBfbG9jLyRpbnN0YWxsX2RpciI7CiAJ
CSRiaW5kaXIgPSAiJHRtcF9sb2MvYmluIjsKIAkJJGxpYmRpciA9ICIkdG1wX2xvYy9saWIi
OwotCQkkbGliZGlyIC49ICcvcG9zdGdyZXNxbCcgdW5sZXNzICRsaWJkaXIgPX4gL3Bvc3Rn
cmVzfHBnc3FsLzsKKwkJJGxpYmRpciAuPSAnL3Bvc3RncmVzcWwnIHVubGVzcyAkaW5zdGFs
bF9kaXIgPX4gL3Bvc3RncmVzfHBnc3FsLzsKIAkJcmV0dXJuICgtZCAkYmluZGlyICYmIC1k
ICRsaWJkaXIpOwogCX0KIAllbHNpZiAoLWUgIiRidWlsZF9kaXIvc3JjL01ha2VmaWxlLmds
b2JhbCIpICAgICMgaS5lLiBub3QgbXN2YwpAQCAtNDI5LDcgKzQyOSw3IEBAIHN1YiBjaGVj
a19pbnN0YWxsX2lzX2NvbXBsZXRlCiAJCSR0bXBfbG9jID0gIiR0bXBfbG9jLyRpbnN0YWxs
X2RpciI7CiAJCSRiaW5kaXIgPSAiJHRtcF9sb2MvYmluIjsKIAkJJGxpYmRpciA9ICIkdG1w
X2xvYy9saWIiOwotCQkkbGliZGlyIC49ICcvcG9zdGdyZXNxbCcgdW5sZXNzICRsaWJkaXIg
PX4gL3Bvc3RncmVzfHBnc3FsLzsKKwkJJGxpYmRpciAuPSAnL3Bvc3RncmVzcWwnIHVubGVz
cyAkaW5zdGFsbF9kaXIgPX4gL3Bvc3RncmVzfHBnc3FsLzsKIAl9CiAKIAkjIHRoZXNlIGZp
bGVzIHNob3VsZCBiZSBwcmVzZW50IGlmIHdlJ3ZlIHRlbXBfaW5zdGFsbGVkIGV2ZXJ5dGhp
bmcsCg==

--------------Hpww0925dyNys0fXxqtrWEdn--