Content-Type: multipart/mixed; boundary="------------i7xs3ZtaYw1IyiUzW2PAMSH4"
Message-ID: <8160688e-db8e-4d7d-a7e7-ff366adf9717@gmail.com>
Date: Sun, 6 Apr 2025 23:00:00 +0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: AIO v2.5
To: Andres Freund <andres@anarazel.de>, Noah Misch <noah@leadboat.com>
Cc: pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>,
 Heikki Linnakangas <hlinnaka@iki.fi>, Robert Haas <robertmhaas@gmail.com>,
 Jakub Wartak <jakub.wartak@enterprisedb.com>,
 Jelte Fennema-Nio <postgres@jeltef.nl>, Antonin Houska <ah@cybertec.at>
References: <d4ff3qxudxn2ze3zk2xwhds5ysezfei3uqeuimymj624t5chsd@sgr2h3rmln5t>
 <wouqlwmsxbgak4kow7p4s6g4voh2bai7b4acy324v4ncgh2nnq@5r5o6nvghsko>
 <20250329212929.a6.nmisch@google.com>
 <usbwzckj7q3jhfx3ann3nrfnukmupbs35axvq5zfyeo6nvrzrm@onjhxs2du4st>
 <7s6fclfekpcxoaaorwrq67v4vmgixf2dcjcgznuj7vxs3ie3wq@okqbi5vabqbx>
 <20250401151159.51.nmisch@google.com>
 <kas47e54inkpgnirnr3txm5zywxkjobhbx6466sfaolmkjtnlu@xuucqvkchksl>
 <uc62i6vi5gd4bi6wtjj5poadqxolgy55e7ihkmf3mthjegb6zl@zqo7xez7sc2r>
 <q4hmkmjqxcrfx3xav5fgxwzelvwwlm6qshagvc45fbugtikf7v@hygsuz24g3lb>
 <20250402001324.e4.nmisch@google.com>
 <3fhulsvzks4khqahl6tngcnuda7pyrrl7223tmjh2x2spralm3@iisldh6pcowy>
Content-Language: en-US
From: Alexander Lakhin <exclusion@gmail.com>
In-Reply-To: <3fhulsvzks4khqahl6tngcnuda7pyrrl7223tmjh2x2spralm3@iisldh6pcowy>
Archived-At: 
 <https://www.postgresql.org/message-id/8160688e-db8e-4d7d-a7e7-ff366adf9717%40gmail.com>
Precedence: bulk

This is a multi-part message in MIME format.
--------------i7xs3ZtaYw1IyiUzW2PAMSH4
Content-Type: multipart/alternative;
 boundary="------------gmT8sUur20VooERp96nz4hTK"

--------------gmT8sUur20VooERp96nz4hTK
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hello Andres,

02.04.2025 14:58, Andres Freund wrote:
> Hi,
>
> I've pushed fixes for 1) and 2) and am working on 3).

When running multiple installcheck's against a single server (please find
the ready-to-use script attached (I use more sophisticated version with
additional patches to make installcheck pass cleanly, but that's not
required for this case)), I've encountered an interesting error related to
AIO/uring:
iteration 8: Sun Apr  6 19:22:39 UTC 2025
installchecks finished: Sun Apr  6 19:23:47 UTC 2025
2025-04-06 19:22:44.216 UTC [349525] LOG:  could not read blocks 0..0 in file "base/6179194/2606": Operation canceled
2025-04-06 19:22:44.216 UTC [349525] ERROR:  could not read blocks 0..0 in file "base/6179194/2606": Operation canceled

server.log contains:
2025-04-06 19:22:44.215 UTC [38231] LOG:  checkpoint complete: wrote ...
2025-04-06 19:22:44.216 UTC [38231] LOG:  checkpoint starting: immediate force wait flush-all
2025-04-06 19:22:44.216 UTC [349525] LOG:  could not read blocks 0..0 in file "base/6179194/2606": Operation canceled
2025-04-06 19:22:44.216 UTC [349525] STATEMENT:  alter table parted_copytest attach partition parted_copytest_a1 for 
values in(1);
2025-04-06 19:22:44.216 UTC [349525] ERROR:  could not read blocks 0..0 in file "base/6179194/2606": Operation canceled
2025-04-06 19:22:44.216 UTC [349525] STATEMENT:  alter table parted_copytest attach partition parted_copytest_a1 for 
values in(1);

It's reproduced better on tmpfs for me; probably you would need to increase
NUM_INSTALLCHECKS/NUM_ITERATIONS for your machine. I can reduce the testing
procedure to something trivial, if it makes sense for you. Probably, the
same effect can be also achieved with just pgbench...

Best regards,
Alexander Lakhin
Neon (https://neon.tech)
--------------gmT8sUur20VooERp96nz4hTK
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <div class="moz-cite-prefix">Hello Andres,<br>
      <br>
      02.04.2025 14:58, Andres Freund wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:3fhulsvzks4khqahl6tngcnuda7pyrrl7223tmjh2x2spralm3@iisldh6pcowy">
      <pre wrap="" class="moz-quote-pre">Hi,

I've pushed fixes for 1) and 2) and am working on 3).
</pre>
    </blockquote>
    <br>
    When running multiple installcheck's against a single server (please
    find<br>
    the ready-to-use script attached (I use more sophisticated version
    with<br>
    additional patches to make installcheck pass cleanly, but that's not<br>
    required for this case)), I've encountered an interesting error
    related to<br>
    AIO/uring:<br>
    iteration 8: Sun Apr  6 19:22:39 UTC 2025<br>
    installchecks finished: Sun Apr  6 19:23:47 UTC 2025<br>
    2025-04-06 19:22:44.216 UTC [349525] LOG:  could not read blocks
    0..0 in file "base/6179194/2606": Operation canceled<br>
    2025-04-06 19:22:44.216 UTC [349525] ERROR:  could not read blocks
    0..0 in file "base/6179194/2606": Operation canceled<br>
    <br>
    server.log contains:<br>
    2025-04-06 19:22:44.215 UTC [38231] LOG:  checkpoint complete: wrote
    ...<br>
    2025-04-06 19:22:44.216 UTC [38231] LOG:  checkpoint starting:
    immediate force wait flush-all<br>
    2025-04-06 19:22:44.216 UTC [349525] LOG:  could not read blocks
    0..0 in file "base/6179194/2606": Operation canceled<br>
    2025-04-06 19:22:44.216 UTC [349525] STATEMENT:  alter table
    parted_copytest attach partition parted_copytest_a1 for values
    in(1);<br>
    2025-04-06 19:22:44.216 UTC [349525] ERROR:  could not read blocks
    0..0 in file "base/6179194/2606": Operation canceled<br>
    2025-04-06 19:22:44.216 UTC [349525] STATEMENT:  alter table
    parted_copytest attach partition parted_copytest_a1 for values
    in(1);<br>
    <br>
    It's reproduced better on tmpfs for me; probably you would need to
    increase<br>
    NUM_INSTALLCHECKS/NUM_ITERATIONS for your machine. I can reduce the
    testing<br>
    procedure to something trivial, if it makes sense for you. Probably,
    the<br>
    same effect can be also achieved with just pgbench...<br>
    <br>
    Best regards,<br>
    Alexander Lakhin<br>
    Neon (<a class="moz-txt-link-freetext" href="https://neon.tech">https://neon.tech</a>)
  </body>
</html>

--------------gmT8sUur20VooERp96nz4hTK--

--------------i7xs3ZtaYw1IyiUzW2PAMSH4
Content-Type: application/gzip; name="test-icx+.tar.gz"
Content-Disposition: attachment; filename="test-icx+.tar.gz"
Content-Transfer-Encoding: base64

H4sIAGbW8mcAA+1ae2/bOBLfv/UpuKq3eRSy7KTZos05QBK7aXBpEtgpbhd1TytLtM2NJKok
lcfV/e43Qz2dxLbSIDgcUAGpJXI4nMdvhkOyikplMe/mVVNOf3mmp9VqvXmzQ+C3/WanVf3F
Z6vVet0m7ddv3rze2dlut3ZIq731+87WL6T1XAJVn0QqV4AogXu9lG5Vf6oMKX7/T54Xv9oj
FtkjV04NSRWxqGG8IIdTziUlkoorKgjzjc+EWBExG22TfCEvX5LBcbfTaJPZTL+12jhoAMNj
V7ghVVRI4/TTR+f4dHCxf3Jy+KF3+M9BZ7uVNl70+vsXx2eng067ZRjnR+f93vvjPzpmYz2+
9jdsBZB0WAR+CQLTMAYXZ/39o14n7TUG/cPDs/M/u8d9GJD12QBgU3M66190dl5vb+0SehNz
oUjaZoACQH3cNUmHmK1ci/Oj7kE5bRj7IxNVut+shxoFy+4BivXxvFMIIJHChlksHyeCvmwK
ERJLjPM2I7z0mSi+DDYmn8mv6aDcDjBwl6gpjQxCQveSEksSn0nlBdSNyNae7dMrO0qCAEVV
IgGHEUK9KSfmQcICn0UTEnOpJoLKZrNpQu/h+fn7k/2jASiVvxLrbEKsf4GJ4d+IW0mUSOpb
hf/SVskmkeXxEJqpSYbAqml7PBqzSSJAsK/EsmJBx+ymU5XfsmjkjgJq+XSUTMpPz5WAKMCY
dc3U1ArYKBEgbUVP628wexQL7m2Y5JqLwLcAnWjHBymsQyKFpwFTpcmwYz3IIOtcMbbabYfc
TwIq0d70hinSLgze5REFC4+ZUYLj4Pi0Yg2MLrPs3b/4oDuB6F0DvwpUnXSdk+OD/n7/T6eg
yliAoSosjj4Nev1O7mEjxxyiEjEHQuYgS5t0aCoeE/DUFeNJpmPkUXgh4BP4VYRJEtCxImPB
Q+KWtCKJjHjieApMck0sRbYhuVndgrtEzlZIWBhSn7mKlqh8QfYjnyBsBfFd5RIXPgM+kUYR
FCmPMo7ThIOx1ARC6Mnc4U2pd4nNlrmJPeUkLGLKHxHrsiqUZQGYRQRILuxE9lLKnDcE0su2
kXrR0IbtkEaWLALPCamU7gSGdcja4ZoxlreRB+98PDbcRPEr10uSMGsA7rCEKMcdQ9w4noBU
mvUYjDuhe+NAyHiJEDTybqFnK22nasr9DrylQWBoHWPOIuUoFlKeoEjbLWmYZG8vU83O1fka
NFGOJgajiaY+FxSDlCgaxgG6ASBAPcXFbZGUisR5LzWVPWWCWtpm59OYhgfujyEtbC4gIJWE
hTYvXJfaXgdUC6geHp5lmyZYWCVyqSg2xGsuDobuAoGQzF4sVTreEmVyWMJlnlqjqDY1Jpwx
w7SyRBvILBAzTVh7AcvkYzYEuABkFMQhz3qOTj+Fdztn5HqK38fvBx0iqOvjzICEtTUS7hKf
Y9rV1gSZHhahsQ7dmr/ZCM0NXElSFfBz0ZjQNHzIigZUEzAjJWvS/jdE8Dv7hf53jVhsiZF0
vhUUIS7tila5KQAOSrARGa5nmkM+0DFQ2sLclBBKmLFhxdoAEW6ol8v97fsiubFnuKuT1hVn
PhhslC2muSKmtFEDgisLZOTm5rChGwBMS1UCZ1Axdj0qMZHHX+2PhU7znGtzRKA9nQss6yGP
KnwKsKnbGPJHZk5INffxJmPqyTpI8x+PNL9EmtCf9cbYOChLdb7dbNq45GSZruBVti/gqSnu
coKVF1Ip9R/iVvY9hiMk9zhRD/HLe1Zx08kqDbQy+Uv8weRfXTaLlYDRyspb4c48Xd5am0US
HHNB1te9TnvX+0fnXh2/6716tbEx59gH+FkNzy4zhnVnTSlV2lw8GEa/IO/ZDcGsIIniJJEV
JWmkmAKlsIzx2XhMYYFVutKA3QzoCnO3tnUdjzAFpX9rbWs/eYgw6C4DJ8s4DOIhe23A2Aci
qBTvXrbKkWAnsZYhoM4Vo9eyCS41n282Sb8mUFjQZ57mJgxCN649yeTHZvF56LJo2SzDZnWe
5uO0gVxmx1g7wV+6nSGPl1Ez0BIuEJFUJSQ/IKFgV5BOofp8mpwlm2XSOoW030Dc784jXYcC
5yH3JHELJveFJRWQkR+GsjZtMgqY5yrg9DTbVvg8Ql72aIFlMpKeYPGTJZ5j9Jwm9qAAUNQR
PHgaHqp8nlNen8Y08nFr9jT4lmxyaaGU9P9OYAeRERcLFvTpUmvRwjdX3WqhKsVtU/JEeHS+
DaecJwKxKuVvbjjQroh9Z7iuRoGMoTIdbujGLAMM2/Zk9w5luuFdTadX6mS0mhDfwLUODlhN
jds/6dCbGqSwG5BKuLiNxgoXbC5XD4q440EJ6zuuB3V6jQEguxNw77Jdh7dMYiqGM6wGHH4d
wftqO4ok8hD/uLmBVFNDcRfI6HA24qPhzHNBwOHMd69q+DakYkIdXC7kcJZ+gD3ufEd1GKHV
4W84i/DcI/+YCDeC8rMOJlwfqlbt7BqO1gnCR231G8JpOAv4BOdk0ZQKpqCPRxFEKC50AQux
BWJU3MYw0oldKa+58Iez/M3BAwEUn0ma9rAI+YIhXD8s3vFHT1pXKwfUcnCFG86uQSxa+YY9
GEM+s4JOUqVg3ymrTdokM/xxpOdGjq4vc4r01MzRpwn5BPNtmASSsjUWHJwSAkM2idzAGbne
JWSvOuEdgS+Hs6krM3zoINa2yd5TsK/kRFaTxCqqGS3ZWsH8G6d+SgOkOCENubgFOwRcObCR
ccYQdqBVelqnIMQnaMHVzEZK31poDK4WdywfIedn1/qP82UTTzQ/t6y3XzZrD6knTUYM6TiJ
a1OnHq9LXTfpoQlxCKjp1NETMZgP+DJ8VSMWA/k4+jZgo0aOzwJgyhEDNF2hnZo28gKoEfDg
OM/0s7wF9rk18v7oFrMXaIbZIM7L0rkPp6RZyQ6ykj9ynREdc0FxCdGf+mx79WAAEQcFoBxh
C6dikgepVLCuQ9wBXdp0jxIhjwZykSZZhgNM3/5y5OjC8Nt3PORLj00SvR4QyyNJpF+pX7RF
5HXr7e/pzQk4IbsInb8Iae/MXTkES64x9P0AcusnEflLXzVVT2j+0kcYLgk4j/E2IQlUp2Xg
JQBJT2JYp73LspOY4vJ0lxXHMMV9FCghtG1Jg70jDXQm1adpjzjRSY9NHjw10b2Lj5AerJBT
faRpb+rR696iutfcJb0/Lvr7Tr931O8NBs7Z+cWgY1qWP9I3OZUq38woL3qDlCR0b6xypZed
HfPuhV56FHZ65hz1TtGCva7zobff7fUHnTb5NOg5H8+6n056Tvegoy8DtD3f7x+fbJC9vfvX
UA0v9ay+O3gJmmlMEXLtMlV6ozJKYsnP5JT6Fb8AJRsTUCsmax5PAp9EXKUnqCOsLtcWY6q4
Is4nQ1l7XTN1UYqhtv4YAb9LveUwMuTPwfjtves8Q9/JNFImxv/6Pyj8fH4+P5+fz8/nWZ7/
Ao53fhsAKAAA

--------------i7xs3ZtaYw1IyiUzW2PAMSH4--