Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nw0PA-00066m-IZ for pgsql-bugs@arkaria.postgresql.org; Tue, 31 May 2022 11:51:20 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nw0P9-0006RU-Ex for pgsql-bugs@arkaria.postgresql.org; Tue, 31 May 2022 11:51:19 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nw0P9-0006Qm-2a for pgsql-bugs@lists.postgresql.org; Tue, 31 May 2022 11:51:19 +0000 Received: from sfr-1.cri.mines-paristech.fr ([77.158.180.225] helo=smtp.cri.ensmp.fr) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nw0P1-0006lT-5y for pgsql-bugs@lists.postgresql.org; Tue, 31 May 2022 11:51:18 +0000 Received: from laroche-adap.cri.ensmp.fr (laroche-adap.cri.ensmp.fr [10.2.14.39]) by smtp.cri.ensmp.fr (Postfix) with ESMTPSA id 90A39C005F; Tue, 31 May 2022 13:54:50 +0200 (CEST) Date: Tue, 31 May 2022 13:51:07 +0200 (CEST) From: Fabien COELHO X-X-Sender: fabien@pseudo Reply-To: Fabien COELHO To: Tom Lane cc: Alastair McKinley , PostgreSQL Bugs , Peter Eisentraut Subject: Re: psql 15beta1 does not print notices on the console until transaction completes In-Reply-To: Message-ID: References: <3662994.1653856025@sss.pgh.pa.us> User-Agent: Alpine 2.22 (DEB 394 2020-01-19) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="2056645051-785713024-1653997868=:1672363" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2056645051-785713024-1653997868=:1672363 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT >>>> create or replace function test_notice() returns void as >>>> $$ >>>> begin >>>> raise notice 'hello'; >>>> perform pg_sleep(10); >>>> end; $$ language plpgsql; >>> >>>> In psql 15beta1, the "hello" message only appears on the console when >>>> the transaction completes. in psql 14.3, it appears immediately as I >>>> would have expected. [...] > I've looked into it. This behavior is triggered by the fact that "psql" is > trying to keep notices and results in some logical order. > > The issue did not arise before because intermediate results where coldly > discarded, so it did not show whether they were discarded before of after > some notice was already shown:-) > > If notices are to be shown "right away", it means that some notice may be > shown *before* the previous result has been shown, just because psql may not > have had time to process it and they are just sent through the protocol and > arriving for the next result, as shown with the attached patch on > non-regression tests: > > SELECT 1 AS one \; SELECT warn('1.5') \; SELECT 2 AS two ; > +NOTICE: warn 1.5 > +CONTEXT: PL/pgSQL function warn(text) line 2 at RAISE > one > ----- > 1 > (1 row) > > -NOTICE: warn 1.5 > -CONTEXT: PL/pgSQL function warn(text) line 2 at RAISE > warn > ------ > t > > Now the second statement warning is shown before the first statement output. > I'm pretty sure that the initial order shoul show up from time to time, just > because of the underlying race condition. > > Would this be okay? It simplifies the code significantly, but introduces > possible race conditions in the output, which could be hidden from the non > regression tests by removing the tests… This would be the simplest solution. > If notices are to be shown "right away" when processing a result, but not > quite "right away" when the previous result is not yet finished, this is > probably achievable but would require some effort and not so clear code. > Maybe psql could not get the next result when we do not need to know which is > last, but the execution would have somehow 2 different paths. I have to try > to see how terrible or not the resulting code would be. I've tried that (see v2 attached), but it does not work as I hoped: the fact that a result has not yet been extracted does not preclude the server from sending the notices relating to a statement, so delaying the getResult does not fix the display order problem in anyway. I was quite naïve. > Another approach could be do drop the "only show the last result" > compatibility, because then "psql" would not need to know that a result is > the last, which requires to trigger the next one before closing the previous > one, hence opening the race condition. That would not work, either, for the reason stated above. In order to have the best of both world, this requires more hopus-pocus: accumulating the notices *but* displaying them as soon as dealing with a statement is done *and* also while waiting for getResult. I see how I can try to reduce a lot the out-of-order window… but not how to fully get rid of it. -- Fabien. --2056645051-785713024-1653997868=:1672363 Content-Type: text/x-diff; name=psql-notices-2.patch Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename=psql-notices-2.patch ZGlmZiAtLWdpdCBhL3NyYy9iaW4vcHNxbC9jb21tb24uYyBiL3NyYy9iaW4v cHNxbC9jb21tb24uYw0KaW5kZXggMDNlNmQ5Y2U4Zi4uNzFiMzNlMmMzNCAx MDA2NDQNCi0tLSBhL3NyYy9iaW4vcHNxbC9jb21tb24uYw0KKysrIGIvc3Jj L2Jpbi9wc3FsL2NvbW1vbi5jDQpAQCAtMTA2MSw0NCArMTA2MSw2IEBAIFBy aW50UXVlcnlSZXN1bHQoUEdyZXN1bHQgKnJlc3VsdCwgYm9vbCBsYXN0LCBi b29sIGlzX3dhdGNoLCBjb25zdCBwcmludFF1ZXJ5T3B0DQogCXJldHVybiBz dWNjZXNzOw0KIH0NCiANCi0vKg0KLSAqIERhdGEgc3RydWN0dXJlIGFuZCBm dW5jdGlvbnMgdG8gcmVjb3JkIG5vdGljZXMgd2hpbGUgdGhleSBhcmUNCi0g KiBlbWl0dGVkLCBzbyB0aGF0IHRoZXkgY2FuIGJlIHNob3duIGxhdGVyLg0K LSAqDQotICogV2UgbmVlZCB0byBrbm93IHdoaWNoIHJlc3VsdCBpcyBsYXN0 LCB3aGljaCByZXF1aXJlcyB0byBleHRyYWN0DQotICogb25lIHJlc3VsdCBp biBhZHZhbmNlLCBoZW5jZSB0d28gYnVmZmVycyBhcmUgbmVlZGVkLg0KLSAq Lw0KLXN0cnVjdCB0X25vdGljZV9tZXNzYWdlcw0KLXsNCi0JUFFFeHBCdWZm ZXJEYXRhIG1lc3NhZ2VzWzJdOw0KLQlpbnQJCQljdXJyZW50Ow0KLX07DQot DQotLyoNCi0gKiBTdG9yZSBub3RpY2VzIGluIGFwcHJvcHJpYXRlIGJ1ZmZl ciwgZm9yIGxhdGVyIGRpc3BsYXkuDQotICovDQotc3RhdGljIHZvaWQNCi1B cHBlbmROb3RpY2VNZXNzYWdlKHZvaWQgKmFyZywgY29uc3QgY2hhciAqbXNn KQ0KLXsNCi0Jc3RydWN0IHRfbm90aWNlX21lc3NhZ2VzICpub3RpY2VzID0g YXJnOw0KLQ0KLQlhcHBlbmRQUUV4cEJ1ZmZlclN0cigmbm90aWNlcy0+bWVz c2FnZXNbbm90aWNlcy0+Y3VycmVudF0sIG1zZyk7DQotfQ0KLQ0KLS8qDQot ICogU2hvdyBub3RpY2VzIHN0b3JlZCBpbiBidWZmZXIsIHdoaWNoIGlzIHRo ZW4gcmVzZXQuDQotICovDQotc3RhdGljIHZvaWQNCi1TaG93Tm90aWNlTWVz c2FnZShzdHJ1Y3QgdF9ub3RpY2VfbWVzc2FnZXMgKm5vdGljZXMpDQotew0K LQlQUUV4cEJ1ZmZlckRhdGEgKmN1cnJlbnQgPSAmbm90aWNlcy0+bWVzc2Fn ZXNbbm90aWNlcy0+Y3VycmVudF07DQotDQotCWlmICgqY3VycmVudC0+ZGF0 YSAhPSAnXDAnKQ0KLQkJcGdfbG9nX2luZm8oIiVzIiwgY3VycmVudC0+ZGF0 YSk7DQotCXJlc2V0UFFFeHBCdWZmZXIoY3VycmVudCk7DQotfQ0KLQ0KLQ0K IC8qDQogICogU2VuZFF1ZXJ5OiBzZW5kIHRoZSBxdWVyeSBzdHJpbmcgdG8g dGhlIGJhY2tlbmQNCiAgKiAoYW5kIHByaW50IG91dCByZXN1bHQpDQpAQCAt MTQ4Myw3ICsxNDQ1LDcgQEAgRXhlY1F1ZXJ5QW5kUHJvY2Vzc1Jlc3VsdHMo Y29uc3QgY2hhciAqcXVlcnksIGRvdWJsZSAqZWxhcHNlZF9tc2VjLCBib29s ICpzdnB0X2cNCiAJaW5zdHJfdGltZQliZWZvcmUsDQogCQkJCWFmdGVyOw0K IAlQR3Jlc3VsdCAgICpyZXN1bHQ7DQotCXN0cnVjdCB0X25vdGljZV9tZXNz YWdlcyBub3RpY2VzOw0KKwlib29sCQluZWVkX2xhc3QgPSBwc2V0LmdzZXRf cHJlZml4IHx8IHBzZXQuZ2V4ZWNfZmxhZyB8fCBwc2V0LmNyb3NzdGFiX2Zs YWcgfHwgIXBzZXQuc2hvd19hbGxfcmVzdWx0czsNCiANCiAJaWYgKHRpbWlu ZykNCiAJCUlOU1RSX1RJTUVfU0VUX0NVUlJFTlQoYmVmb3JlKTsNCkBAIC0x NTEzLDEyICsxNDc1LDYgQEAgRXhlY1F1ZXJ5QW5kUHJvY2Vzc1Jlc3VsdHMo Y29uc3QgY2hhciAqcXVlcnksIGRvdWJsZSAqZWxhcHNlZF9tc2VjLCBib29s ICpzdnB0X2cNCiAJCXJldHVybiAwOw0KIAl9DQogDQotCS8qIGludGVyY2Vw dCBub3RpY2VzICovDQotCW5vdGljZXMuY3VycmVudCA9IDA7DQotCWluaXRQ UUV4cEJ1ZmZlcigmbm90aWNlcy5tZXNzYWdlc1swXSk7DQotCWluaXRQUUV4 cEJ1ZmZlcigmbm90aWNlcy5tZXNzYWdlc1sxXSk7DQotCVBRc2V0Tm90aWNl UHJvY2Vzc29yKHBzZXQuZGIsIEFwcGVuZE5vdGljZU1lc3NhZ2UsICZub3Rp Y2VzKTsNCi0NCiAJLyogZmlyc3QgcmVzdWx0ICovDQogCXJlc3VsdCA9IFBR Z2V0UmVzdWx0KHBzZXQuZGIpOw0KIA0KQEAgLTE1MzYsNyArMTQ5Miw2IEBA IEV4ZWNRdWVyeUFuZFByb2Nlc3NSZXN1bHRzKGNvbnN0IGNoYXIgKnF1ZXJ5 LCBkb3VibGUgKmVsYXBzZWRfbXNlYywgYm9vbCAqc3ZwdF9nDQogCQkJICov DQogCQkJY29uc3QgY2hhciAqZXJyb3IgPSBQUXJlc3VsdEVycm9yTWVzc2Fn ZShyZXN1bHQpOw0KIA0KLQkJCVNob3dOb3RpY2VNZXNzYWdlKCZub3RpY2Vz KTsNCiAJCQlpZiAoc3RybGVuKGVycm9yKSkNCiAJCQkJcGdfbG9nX2luZm8o IiVzIiwgZXJyb3IpOw0KIA0KQEAgLTE2MDEsOCArMTU1Niw2IEBAIEV4ZWNR dWVyeUFuZFByb2Nlc3NSZXN1bHRzKGNvbnN0IGNoYXIgKnF1ZXJ5LCBkb3Vi bGUgKmVsYXBzZWRfbXNlYywgYm9vbCAqc3ZwdF9nDQogCQlpZiAocmVzdWx0 X3N0YXR1cyA9PSBQR1JFU19DT1BZX0lOIHx8DQogCQkJcmVzdWx0X3N0YXR1 cyA9PSBQR1JFU19DT1BZX09VVCkNCiAJCXsNCi0JCQlTaG93Tm90aWNlTWVz c2FnZSgmbm90aWNlcyk7DQotDQogCQkJaWYgKGlzX3dhdGNoKQ0KIAkJCXsN CiAJCQkJQ2xlYXJPclNhdmVBbGxSZXN1bHRzKCk7DQpAQCAtMTYxMCwxMiAr MTU2Myw3IEBAIEV4ZWNRdWVyeUFuZFByb2Nlc3NSZXN1bHRzKGNvbnN0IGNo YXIgKnF1ZXJ5LCBkb3VibGUgKmVsYXBzZWRfbXNlYywgYm9vbCAqc3ZwdF9n DQogCQkJCXJldHVybiAtMTsNCiAJCQl9DQogDQotCQkJLyogdXNlIG5vcm1h bCBub3RpY2UgcHJvY2Vzc29yIGR1cmluZyBDT1BZICovDQotCQkJUFFzZXRO b3RpY2VQcm9jZXNzb3IocHNldC5kYiwgTm90aWNlUHJvY2Vzc29yLCBOVUxM KTsNCi0NCiAJCQlzdWNjZXNzICY9IEhhbmRsZUNvcHlSZXN1bHQoJnJlc3Vs dCk7DQotDQotCQkJUFFzZXROb3RpY2VQcm9jZXNzb3IocHNldC5kYiwgQXBw ZW5kTm90aWNlTWVzc2FnZSwgJm5vdGljZXMpOw0KIAkJfQ0KIA0KIAkJLyoN CkBAIC0xNjIzLDkgKzE1NzEsNyBAQCBFeGVjUXVlcnlBbmRQcm9jZXNzUmVz dWx0cyhjb25zdCBjaGFyICpxdWVyeSwgZG91YmxlICplbGFwc2VkX21zZWMs IGJvb2wgKnN2cHRfZw0KIAkJICogc3RyaW5nLCBpdCB3aWxsIHJldHVybiBO VUxMLiAgT3RoZXJ3aXNlLCB3ZSdsbCBoYXZlIG90aGVyIHJlc3VsdHMNCiAJ CSAqIHRvIHByb2Nlc3MuICBXZSBuZWVkIHRvIGRvIHRoYXQgdG8gY2hlY2sg d2hldGhlciB0aGlzIGlzIHRoZSBsYXN0Lg0KIAkJICovDQotCQlub3RpY2Vz LmN1cnJlbnQgXj0gMTsNCi0JCW5leHRfcmVzdWx0ID0gUFFnZXRSZXN1bHQo cHNldC5kYik7DQotCQlub3RpY2VzLmN1cnJlbnQgXj0gMTsNCisJCW5leHRf cmVzdWx0ID0gbmVlZF9sYXN0ID8gUFFnZXRSZXN1bHQocHNldC5kYikgOiBO VUxMOw0KIAkJbGFzdCA9IChuZXh0X3Jlc3VsdCA9PSBOVUxMKTsNCiANCiAJ CS8qDQpAQCAtMTY0Nyw5ICsxNTkzLDYgQEAgRXhlY1F1ZXJ5QW5kUHJvY2Vz c1Jlc3VsdHMoY29uc3QgY2hhciAqcXVlcnksIGRvdWJsZSAqZWxhcHNlZF9t c2VjLCBib29sICpzdnB0X2cNCiAJCQkqZWxhcHNlZF9tc2VjID0gSU5TVFJf VElNRV9HRVRfTUlMTElTRUMoYWZ0ZXIpOw0KIAkJfQ0KIA0KLQkJLyogbm90 aWNlcyBhbHJlYWR5IHNob3duIGFib3ZlIGZvciBjb3B5ICovDQotCQlTaG93 Tm90aWNlTWVzc2FnZSgmbm90aWNlcyk7DQotDQogCQkvKiB0aGlzIG1heSBv ciBtYXkgbm90IHByaW50IHNvbWV0aGluZyBkZXBlbmRpbmcgb24gc2V0dGlu Z3MgKi8NCiAJCWlmIChyZXN1bHQgIT0gTlVMTCkNCiAJCQlzdWNjZXNzICY9 IFByaW50UXVlcnlSZXN1bHQocmVzdWx0LCBsYXN0LCBmYWxzZSwgb3B0LCBw cmludFF1ZXJ5Rm91dCk7DQpAQCAtMTY1OSw4ICsxNjAyLDcgQEAgRXhlY1F1 ZXJ5QW5kUHJvY2Vzc1Jlc3VsdHMoY29uc3QgY2hhciAqcXVlcnksIGRvdWJs ZSAqZWxhcHNlZF9tc2VjLCBib29sICpzdnB0X2cNCiAJCQlTZXRSZXN1bHRW YXJpYWJsZXMocmVzdWx0LCB0cnVlKTsNCiANCiAJCUNsZWFyT3JTYXZlUmVz dWx0KHJlc3VsdCk7DQotCQlub3RpY2VzLmN1cnJlbnQgXj0gMTsNCi0JCXJl c3VsdCA9IG5leHRfcmVzdWx0Ow0KKwkJcmVzdWx0ID0gbmVlZF9sYXN0ID8g bmV4dF9yZXN1bHQgOiBQUWdldFJlc3VsdChwc2V0LmRiKTsNCiANCiAJCWlm IChjYW5jZWxfcHJlc3NlZCkNCiAJCXsNCkBAIC0xNjY5LDExICsxNjExLDYg QEAgRXhlY1F1ZXJ5QW5kUHJvY2Vzc1Jlc3VsdHMoY29uc3QgY2hhciAqcXVl cnksIGRvdWJsZSAqZWxhcHNlZF9tc2VjLCBib29sICpzdnB0X2cNCiAJCX0N CiAJfQ0KIA0KLQkvKiByZXNldCBub3RpY2UgaG9vayAqLw0KLQlQUXNldE5v dGljZVByb2Nlc3Nvcihwc2V0LmRiLCBOb3RpY2VQcm9jZXNzb3IsIE5VTEwp Ow0KLQl0ZXJtUFFFeHBCdWZmZXIoJm5vdGljZXMubWVzc2FnZXNbMF0pOw0K LQl0ZXJtUFFFeHBCdWZmZXIoJm5vdGljZXMubWVzc2FnZXNbMV0pOw0KLQ0K IAkvKiBtYXkgbmVlZCB0aGlzIHRvIHJlY292ZXIgZnJvbSBjb25uIGxvc3Mg ZHVyaW5nIENPUFkgKi8NCiAJaWYgKCFDaGVja0Nvbm5lY3Rpb24oKSkNCiAJ CXJldHVybiAtMTsNCmRpZmYgLS1naXQgYS9zcmMvdGVzdC9yZWdyZXNzL2V4 cGVjdGVkL3BzcWwub3V0IGIvc3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9w c3FsLm91dA0KaW5kZXggMmEzOGE5M2EzYi4uYmFlZGYwNGJhNSAxMDA2NDQN Ci0tLSBhL3NyYy90ZXN0L3JlZ3Jlc3MvZXhwZWN0ZWQvcHNxbC5vdXQNCisr KyBiL3NyYy90ZXN0L3JlZ3Jlc3MvZXhwZWN0ZWQvcHNxbC5vdXQNCkBAIC01 MzM1LDEzICs1MzM1LDEzIEBAIENPTlRFWFQ6ICBQTC9wZ1NRTCBmdW5jdGlv biB3YXJuKHRleHQpIGxpbmUgMiBhdCBSQUlTRQ0KIA0KIC0tIFxnc2V0IGFw cGxpZXMgdG8gbGFzdCBxdWVyeSBvbmx5DQogU0VMRUNUIDMgQVMgdGhyZWUg XDsgU0VMRUNUIHdhcm4oJzMuNScpIFw7IFNFTEVDVCA0IEFTIGZvdXIgXGdz ZXQNCitOT1RJQ0U6ICB3YXJuIDMuNQ0KK0NPTlRFWFQ6ICBQTC9wZ1NRTCBm dW5jdGlvbiB3YXJuKHRleHQpIGxpbmUgMiBhdCBSQUlTRQ0KICB0aHJlZSAN CiAtLS0tLS0tDQogICAgICAzDQogKDEgcm93KQ0KIA0KLU5PVElDRTogIHdh cm4gMy41DQotQ09OVEVYVDogIFBML3BnU1FMIGZ1bmN0aW9uIHdhcm4odGV4 dCkgbGluZSAyIGF0IFJBSVNFDQogIHdhcm4gDQogLS0tLS0tDQogIHQNCg== --2056645051-785713024-1653997868=:1672363--