From: David Raymond <David.Raymond@tomtom.com>
To: Daniele Varrazzo <daniele.varrazzo@gmail.com>
CC: "psycopg@postgresql.org" <psycopg@postgresql.org>
Subject: RE: CPU usage for queries, psycopg 2 vs 3
Thread-Topic: CPU usage for queries, psycopg 2 vs 3
Thread-Index: AdngG2Om+7oR5OUYTk6QRCXQ7I8s9wA+0WmAABtShdACbYdCAAAk8LZw
Date: Wed, 20 Sep 2023 14:27:27 +0000
Message-ID: 
 <DB7PR07MB391650DB3D30B796DE2F81F687F9A@DB7PR07MB3916.eurprd07.prod.outlook.com>
References: 
 <DB7PR07MB3916A230837909499A87EF3287E8A@DB7PR07MB3916.eurprd07.prod.outlook.com>
 <CA+mi_8Z98u3Gq-3JrAbWRW7ar9hhPAO5umze==xgpEHdxdcPKQ@mail.gmail.com>
 <DB7PR07MB39169ECBC7E0E02E8315982F87EEA@DB7PR07MB3916.eurprd07.prod.outlook.com>
 <CA+mi_8Z0JkU0pu=dax5o3Hs0JUKf+9QOWbp3LsMJXLUk0Ke4Gw@mail.gmail.com>
In-Reply-To: 
 <CA+mi_8Z0JkU0pu=dax5o3Hs0JUKf+9QOWbp3LsMJXLUk0Ke4Gw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?Windows-1252?Q?FFXfOEGEykqmW3cd2WxH0I+G5peI2FmIMotV4RnauWdIHgSL/Pg92+Gr?=
 =?Windows-1252?Q?8CDtWyLYqxE0qfVE0GoXLa9eyD0Ov8cgbNJcetnEyKiS3oBkEpc9eHvm?=
 =?Windows-1252?Q?jiXaA7wQjrSudBaX+7PMF60zRWLGNBV0sRuF3cQTl/5El1Ykusd+asBr?=
 =?Windows-1252?Q?hczz+AB4aK1AnpUaBpLXRbT4TEwgaPlL2wBlHZPq3kvpAPhwkfOz69HT?=
 =?Windows-1252?Q?hvuBrhK526cU1Qtb5XRlaXSY66Fhvd/+B5p3t4FQoNrDD1rhU6vXbMuo?=
 =?Windows-1252?Q?ISGWyztbsZ13OvuEaTM04mF1XEMDqOcnQMPrdv+j5z6g4Un2MaBZJUcI?=
 =?Windows-1252?Q?h9KXrce9UfdXjVgXy8mMHo60wfQ3So2+rM85LdlFbSWp3DORPaYOs+dn?=
 =?Windows-1252?Q?aU4lK/vU/AjBs/EkdPx1w72c5fl+LLMtRvRE3PB5yF9z3vNR7V/NdQ7J?=
 =?Windows-1252?Q?bA0E38dsqYGXM/Q8tnmHFzTthxFIEdqrjHVpLkjaw6desjWqYM5JBPFF?=
 =?Windows-1252?Q?FBbmxqPbuEik6QFm8b0pbYLvel170NQ9V1FsS1vAwlyFgsGCPNsYpiaP?=
 =?Windows-1252?Q?dhlAG50ebs65H7X7PEQwYjwtmYKRxl6xZpakSqOMD5557y55rXxGpoXo?=
 =?Windows-1252?Q?+CgYssg/FWGFYgiLVUf9n7/sPBY+E+IJsmvxomMD1C5rAL3fxObjqs6s?=
 =?Windows-1252?Q?Ucg/vUIXVqPwIO8+Yg/BwLTgo+d+9aQsQEY+GBaeoBgxAP+N/s/639ev?=
 =?Windows-1252?Q?mVER/f/1ZVjDQIgsoLRIcaN74VXOJ95VFjsJ8rAdz4vqY0mh7zmJpr4k?=
 =?Windows-1252?Q?zE/686BbHkcoHqLcx11Ba7AAhKfoLfZqbmrczDmOvBqOxJrcwzvIrMK0?=
 =?Windows-1252?Q?wdEjilkt8jfVkwy9qpyhWFq/FeTq8Mb7BSYuzhZ7FU7l+ez7hfVCZx67?=
 =?Windows-1252?Q?JPQVt3PV1u/7zkyov+dQtODNI7Gg6Cjl3oV9Nh6h5njabHQDdoqmL6Sy?=
 =?Windows-1252?Q?FivNf5UQ3mEMLDV/OfXVbYhe240Yn+gPyZX3zX8+z/kA8gvpEf0p22wi?=
 =?Windows-1252?Q?OU3NEgFxsRGwE0BjKyy3YsO8U6AgLbiL6TSVJn3NFgs7+i29yN1giMtZ?=
 =?Windows-1252?Q?4HWis9lspjmK9iPG4RUdMguetZDOalb/dB2wy7gr0ptD5M5x8o2kPBK1?=
 =?Windows-1252?Q?BV93zfXO1udtOX7X4h1jd8LX8eZFCc8z/fD4eQVaPxy5B3u5/j1E+46L?=
 =?Windows-1252?Q?0bGpcY420f6HAO0LAiUxSWGPG635s64CODmQsYXxwRtMfF7YiBzCT8jZ?=
 =?Windows-1252?Q?ckd6a/rQaJBvpN+w2COQtywQ4K9XeqzLyOC+P13o+Xdsu+7u9SnFIndo?=
 =?Windows-1252?Q?n+zXScbFZZf5TMLeqThMgZVi+MoY60rNjgIQ5gCevjj1kDdg3sJabGbq?=
 =?Windows-1252?Q?VfLjolgHsnuG0NF78cfxmjjr/fFZPUmwhvjaXPs/1sBFFQBRSMY8Jp+z?=
 =?Windows-1252?Q?tLv1fqYuTael8BIo3wChnDoLxJzGM6UHWd8dW52Bkoh8GGBhQ2d2kDsL?=
 =?Windows-1252?Q?8y6lgzi48b4qml/74zxh5h5/AhSsAVAnQeC2cOY2BGfYGqoyHN5WjIRr?=
 =?Windows-1252?Q?WgbCervm9VFUImB1kfJuqKsQtIraqLRk2x7evDnYzTDbOPjVTgVuWsW0?=
 =?Windows-1252?Q?mFBiN50NQoj/QBBJlT3EzcTDk1Q5lDpg?=
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: tomtom.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DB7PR07MB3916.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 
 484551a3-9ec4-4580-e884-08dbb9e5b71e
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Sep 2023 14:27:27.3344
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 374f8026-7b54-4a3a-b87d-328fa26ec10d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 
 PSAA4qKtplPK2wTrve5LIiEr0ZE6dCltTljWC7U4tcNVishbqqiVM7quDyJ1FjuAem74mQ4MJqRaFIfkcZQEr5MZNeTKRwlcEXNOC/WK6GQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR07MB9361
List-Id: <psycopg.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:psycopg@lists.postgresql.org>
List-Owner: <mailto:psycopg-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/psycopg>
Archived-At: 
 <https://www.postgresql.org/message-id/DB7PR07MB391650DB3D30B796DE2F81F687F9A%40DB7PR07MB3916.eurprd07.prod.outlook.com>
Precedence: bulk

Looks good. With the change it quietly idles while waiting for query result=
s, with no CPU spike.

-David

> Hello David,
>
> On Thu, 7 Sept 2023 at 14:17, David Raymond <David.Raymond@tomtom.com> wr=
ote:
>>
>> > Could you please be so kind as to repeat your tests with psycopg 3.1.4=
?
>> Tried with 3.1.4 and it has idle processor usage while waiting for the 8=
 results, just like psycopg2.
>> Tried with 3.1.5 and it spikes up to 62% reported CPU usage while waitin=
g for the 8 results, just like the current release.

> I am not able to quickly investigate this issue, because I don't have
> a Windows machine handy. However I would like to release 3.1.11 soon,
> so I would like to merge this changeset, which should revert the wait
> function to what was in 3.1.4 on Windows:

> Are you able to test it to verify that CPU usage is regular with this cha=
ngeset?


> Cheers

> -- Daniele