MIME-Version: 1.0
References: <CA+E0NR4S+NC6+QHyY_vUuQZMzLhKqczMx-jJVqtjAxF6+=JwAA@mail.gmail.com>
 <94d53a41-d1f5-4fd1-8d1e-ff36ac569848@app.fastmail.com> <4DA5A159-39FD-47DB-95E5-A4F65BC5782A@yandex-team.ru>
 <53e96b5a-cc77-4afa-8103-79c69afef1a6@uni-muenster.de>
In-Reply-To: <53e96b5a-cc77-4afa-8103-79c69afef1a6@uni-muenster.de>
From: Kirill Gavrilov <diphantxm@gmail.com>
Date: Tue, 1 Oct 2024 18:46:32 +0300
Message-ID: <CA+E0NR4=J-pgqEAV2weymygGmk1ePym45m2AcGyi2hpa9xRvBg@mail.gmail.com>
Subject: Re: Truncate logs by max_log_size
To: Jim Jones <jim.jones@uni-muenster.de>
Cc: "Andrey M. Borodin" <x4mmm@yandex-team.ru>, Euler Taveira <euler@eulerto.com>, 
	pgsql-hackers@lists.postgresql.org
Content-Type: multipart/mixed; boundary="00000000000069af8b06236c3a53"
Archived-At: <https://www.postgresql.org/message-id/CA%2BE0NR4%3DJ-pgqEAV2weymygGmk1ePym45m2AcGyi2hpa9xRvBg%40mail.gmail.com>
Precedence: bulk

--00000000000069af8b06236c3a53
Content-Type: multipart/alternative; boundary="00000000000069af8b06236c3a51"

--00000000000069af8b06236c3a51
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Oct 1, 2024 at 2:46=E2=80=AFPM Jim Jones <jim.jones@uni-muenster.de=
> wrote:

>
> On 27.09.24 12:36, Andrey M. Borodin wrote:
> > Consider max_log_size =3D 10Mb. The perspective might look very differe=
nt.
> >  It=E2=80=99s not about WHERE anymore. It's a guard against heavy abuse=
.
> >
> > The feature looks very important for me.
> I have the same opinion. As a fail safe it sounds very useful to me.
>
> Unfortunately, the patch does not apply:
>
> $ git apply
> ~/patches/max_log_query/0001-parameter-max_log_size-to-truncate-logs.patc=
h
> -v
> Checking patch src/backend/utils/error/elog.c...
> error: while searching for:
> char       *Log_destination_string =3D NULL;
> bool            syslog_sequence_numbers =3D true;
> bool            syslog_split_messages =3D true;
>
> /* Processed form of backtrace_symbols GUC */
> static char *backtrace_symbol_list;
>
> error: patch failed: src/backend/utils/error/elog.c:114
> error: src/backend/utils/error/elog.c: patch does not apply
> Checking patch src/backend/utils/misc/guc_tables.c...
> Hunk #1 succeeded at 3714 (offset 247 lines).
> Checking patch src/backend/utils/misc/postgresql.conf.sample...
> Hunk #1 succeeded at 615 (offset 23 lines).
> Checking patch src/bin/pg_ctl/t/004_logrotate.pl...
> error: while searching for:
> check_log_pattern('csvlog',  $new_current_logfiles, 'syntax error', $node=
);
> check_log_pattern('jsonlog', $new_current_logfiles, 'syntax error', $node=
);
>
> $node->stop();
>
> done_testing();
>
> error: patch failed: src/bin/pg_ctl/t/004_logrotate.pl:135
> error: src/bin/pg_ctl/t/004_logrotate.pl: patch does not apply
> Checking patch src/include/utils/elog.h...
> Hunk #1 succeeded at 502 (offset 1 line).
>
> Does it truncate only single queries or also a transaction with many
> statements?
>
> Thanks!
>
> Best, Jim
>

 My apologies, attached patch should work on master branch.
It truncates single queries and a statement in transaction that occurred an
error.

--00000000000069af8b06236c3a51
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Tue, Oct 1, 2024 at 2:46=E2=80=AFPM Ji=
m Jones &lt;<a href=3D"mailto:jim.jones@uni-muenster.de">jim.jones@uni-muen=
ster.de</a>&gt; wrote:<br></div><div class=3D"gmail_quote"><blockquote clas=
s=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid r=
gb(204,204,204);padding-left:1ex"><br>
On 27.09.24 12:36, Andrey M. Borodin wrote:<br>
&gt; Consider max_log_size =3D 10Mb. The perspective might look very differ=
ent.<br>
&gt;=C2=A0 It=E2=80=99s not about WHERE anymore. It&#39;s a guard against h=
eavy abuse.<br>
&gt;<br>
&gt; The feature looks very important for me.<br>
I have the same opinion. As a fail safe it sounds very useful to me.<br>
<br>
Unfortunately, the patch does not apply:<br>
<br>
$ git apply<br>
~/patches/max_log_query/0001-parameter-max_log_size-to-truncate-logs.patch<=
br>
-v<br>
Checking patch src/backend/utils/error/elog.c...<br>
error: while searching for:<br>
char=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 *Log_destination_string =3D NULL;<=
br>
bool=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sysl=
og_sequence_numbers =3D true;<br>
bool=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sysl=
og_split_messages =3D true;<br>
<br>
/* Processed form of backtrace_symbols GUC */<br>
static char *backtrace_symbol_list;<br>
<br>
error: patch failed: src/backend/utils/error/elog.c:114<br>
error: src/backend/utils/error/elog.c: patch does not apply<br>
Checking patch src/backend/utils/misc/guc_tables.c...<br>
Hunk #1 succeeded at 3714 (offset 247 lines).<br>
Checking patch src/backend/utils/misc/postgresql.conf.sample...<br>
Hunk #1 succeeded at 615 (offset 23 lines).<br>
Checking patch src/bin/pg_ctl/t/004_logrotate.pl...<br>
error: while searching for:<br>
check_log_pattern(&#39;csvlog&#39;,=C2=A0 $new_current_logfiles, &#39;synta=
x error&#39;, $node);<br>
check_log_pattern(&#39;jsonlog&#39;, $new_current_logfiles, &#39;syntax err=
or&#39;, $node);<br>
<br>
$node-&gt;stop();<br>
<br>
done_testing();<br>
<br>
error: patch failed: src/bin/pg_ctl/t/<a href=3D"http://004_logrotate.pl:13=
5" rel=3D"noreferrer" target=3D"_blank">004_logrotate.pl:135</a><br>
error: src/bin/pg_ctl/t/<a href=3D"http://004_logrotate.pl" rel=3D"noreferr=
er" target=3D"_blank">004_logrotate.pl</a>: patch does not apply<br>
Checking patch src/include/utils/elog.h...<br>
Hunk #1 succeeded at 502 (offset 1 line).<br>
<br>
Does it truncate only single queries or also a transaction with many<br>
statements?<br>
<br>
Thanks!<br>
<br>
Best, Jim<br></blockquote><div><br></div><div>=C2=A0My apologies, attached =
patch should work on master branch.</div><div>It truncates single queries a=
nd a statement in transaction that occurred an error.</div><div><br></div><=
/div></div>

--00000000000069af8b06236c3a51--
--00000000000069af8b06236c3a53
Content-Type: application/octet-stream; 
	name="0002-parameter-max_log_size-to-truncate-logs.patch"
Content-Disposition: attachment; 
	filename="0002-parameter-max_log_size-to-truncate-logs.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_m1qlxzr20>
X-Attachment-Id: f_m1qlxzr20

RnJvbSBkMmViOWNjZGU5NzBmNzUyYmUxMmNlNGRiY2Y0MGM1NTgwNjU5OWM5IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBkaXBoYW50eG0gPGRpcGhhbnR4bUB5YW5kZXgtdGVhbS5ydT4K
RGF0ZTogVHVlLCAxIE9jdCAyMDI0IDE3OjI4OjE2ICswMzAwClN1YmplY3Q6IFtQQVRDSF0gcGFy
YW1ldGVyIG1heF9sb2dfc2l6ZSB0byB0cnVuY2F0ZSBsb2dzCgpUaGVyZSBpcyBubyBuZWVkIHRv
IGxvZyB0aGUgZW50aXJlIHF1ZXJ5LCBiZWNhdXNlIGl0IG1heSBiZSBsYXJnZSBhbmQgdGFrZSBs
b3RzIG9mIHNwYWNlIG9uIGRpc2suIFBhcmFtZXRlciBtYXhfbG9nX3NpemUgc2V0IHRoZSBtYXhp
bXVtIGxlbmd0aCBmb3IgbG9nZ2VkIHF1ZXJ5LiBFdmVyeXRoaW5nIGJleW9uZCB0aGF0IGxlbmd0
aCBpcyB0cnVuY2F0ZWQuIFZhbHVlIDAgZGlzYWJsZXMgdGhlIHBhcmFtZXRlci4KLS0tCiBzcmMv
YmFja2VuZC91dGlscy9lcnJvci9lbG9nLmMgICAgICAgICAgICAgICAgfCAgOCArKysrKysrKwog
c3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9ndWNfdGFibGVzLmMgICAgICAgICAgIHwgMTEgKysrKysr
KysrKysKIHNyYy9iYWNrZW5kL3V0aWxzL21pc2MvcG9zdGdyZXNxbC5jb25mLnNhbXBsZSB8ICAy
ICsrCiBzcmMvYmluL3BnX2N0bC90LzAwNF9sb2dyb3RhdGUucGwgICAgICAgICAgICAgfCAxNSAr
KysrKysrKysrKysrKysKIHNyYy9pbmNsdWRlL3V0aWxzL2Vsb2cuaCAgICAgICAgICAgICAgICAg
ICAgICB8ICAxICsKIDUgZmlsZXMgY2hhbmdlZCwgMzcgaW5zZXJ0aW9ucygrKQoKZGlmZiAtLWdp
dCBhL3NyYy9iYWNrZW5kL3V0aWxzL2Vycm9yL2Vsb2cuYyBiL3NyYy9iYWNrZW5kL3V0aWxzL2Vy
cm9yL2Vsb2cuYwppbmRleCA1Y2JiNWI1NDE2Li4wNjkwZWM1N2NlIDEwMDY0NAotLS0gYS9zcmMv
YmFja2VuZC91dGlscy9lcnJvci9lbG9nLmMKKysrIGIvc3JjL2JhY2tlbmQvdXRpbHMvZXJyb3Iv
ZWxvZy5jCkBAIC0xMTEsNiArMTExLDcgQEAgaW50CQkJTG9nX2Rlc3RpbmF0aW9uID0gTE9HX0RF
U1RJTkFUSU9OX1NUREVSUjsKIGNoYXIJICAgKkxvZ19kZXN0aW5hdGlvbl9zdHJpbmcgPSBOVUxM
OwogYm9vbAkJc3lzbG9nX3NlcXVlbmNlX251bWJlcnMgPSB0cnVlOwogYm9vbAkJc3lzbG9nX3Nw
bGl0X21lc3NhZ2VzID0gdHJ1ZTsKK2ludAkJCW1heF9sb2dfc2l6ZSA9IDA7CiAKIC8qIFByb2Nl
c3NlZCBmb3JtIG9mIGJhY2t0cmFjZV9mdW5jdGlvbnMgR1VDICovCiBzdGF0aWMgY2hhciAqYmFj
a3RyYWNlX2Z1bmN0aW9uX2xpc3Q7CkBAIC0xNjkzLDYgKzE2OTQsMTMgQEAgRW1pdEVycm9yUmVw
b3J0KHZvaWQpCiAJQ0hFQ0tfU1RBQ0tfREVQVEgoKTsKIAlvbGRjb250ZXh0ID0gTWVtb3J5Q29u
dGV4dFN3aXRjaFRvKGVkYXRhLT5hc3NvY19jb250ZXh0KTsKIAorCWlmIChtYXhfbG9nX3NpemUg
IT0gMCAmJiBkZWJ1Z19xdWVyeV9zdHJpbmcgIT0gTlVMTCkKKwl7CisJCWNoYXIqIHN0ciA9IGRl
YnVnX3F1ZXJ5X3N0cmluZzsKKwkJc3RyW3BnX21iY2xpcGxlbihzdHIsIHN0cmxlbihzdHIpLCBt
YXhfbG9nX3NpemUpXSA9ICdcMCc7CisJCWRlYnVnX3F1ZXJ5X3N0cmluZyA9IHN0cjsKKwl9CisK
IAkvKgogCSAqIFJlc2V0IHRoZSBmb3JtYXR0ZWQgdGltZXN0YW1wIGZpZWxkcyBiZWZvcmUgZW1p
dHRpbmcgYW55IGxvZ3MuICBUaGlzCiAJICogaW5jbHVkZXMgYWxsIHRoZSBsb2cgZGVzdGluYXRp
b25zIGFuZCBlbWl0X2xvZ19ob29rLCBhcyB0aGUgbGF0dGVyCmRpZmYgLS1naXQgYS9zcmMvYmFj
a2VuZC91dGlscy9taXNjL2d1Y190YWJsZXMuYyBiL3NyYy9iYWNrZW5kL3V0aWxzL21pc2MvZ3Vj
X3RhYmxlcy5jCmluZGV4IDY4NjMwOWRiNTguLmNmMTQwNGMwNTkgMTAwNjQ0Ci0tLSBhL3NyYy9i
YWNrZW5kL3V0aWxzL21pc2MvZ3VjX3RhYmxlcy5jCisrKyBiL3NyYy9iYWNrZW5kL3V0aWxzL21p
c2MvZ3VjX3RhYmxlcy5jCkBAIC0zNzE0LDYgKzM3MTQsMTcgQEAgc3RydWN0IGNvbmZpZ19pbnQg
Q29uZmlndXJlTmFtZXNJbnRbXSA9CiAJCU5VTEwsIE5VTEwsIE5VTEwKIAl9LAogCisJeworCQl7
Im1heF9sb2dfc2l6ZSIsIFBHQ19TSUdIVVAsIExPR0dJTkdfV0hBVCwKKwkJCWdldHRleHRfbm9v
cCgiU2V0cyBtYXggc2l6ZSBvZiBsb2dnZWQgc3RhdGVtZW50LiIpLAorCQkJTlVMTAorCQl9LAor
CQkmbWF4X2xvZ19zaXplLAorCQk1ICogKDEwMjQgKiAxMDI0KSwKKwkJMCwgSU5UX01BWCwKKwkJ
TlVMTCwgTlVMTCwgTlVMTAorCX0sCisKIAkvKiBFbmQtb2YtbGlzdCBtYXJrZXIgKi8KIAl7CiAJ
CXtOVUxMLCAwLCAwLCBOVUxMLCBOVUxMfSwgTlVMTCwgMCwgMCwgMCwgTlVMTCwgTlVMTCwgTlVM
TApkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9wb3N0Z3Jlc3FsLmNvbmYuc2Ft
cGxlIGIvc3JjL2JhY2tlbmQvdXRpbHMvbWlzYy9wb3N0Z3Jlc3FsLmNvbmYuc2FtcGxlCmluZGV4
IDY2N2UwZGM0MGEuLjBmZmIzMmQ1ZmIgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL3V0aWxzL21p
c2MvcG9zdGdyZXNxbC5jb25mLnNhbXBsZQorKysgYi9zcmMvYmFja2VuZC91dGlscy9taXNjL3Bv
c3RncmVzcWwuY29uZi5zYW1wbGUKQEAgLTYxNSw2ICs2MTUsOCBAQAogCQkJCQkjIGJpbmQtcGFy
YW1ldGVyIHZhbHVlcyB0byBOIGJ5dGVzOwogCQkJCQkjIC0xIG1lYW5zIHByaW50IGluIGZ1bGws
IDAgZGlzYWJsZXMKICNsb2dfc3RhdGVtZW50ID0gJ25vbmUnCQkJIyBub25lLCBkZGwsIG1vZCwg
YWxsCisjbWF4X2xvZ19zaXplID0gMCAJCQkJIyBtYXggc2l6ZSBvZiBsb2dnZWQgc3RhdGVtZW50
X3RpbWVvdXQKKwkJCQkgICAgIyAwIGRpc2FibGVzIHRoZSBmZWF0dXJlCiAjbG9nX3JlcGxpY2F0
aW9uX2NvbW1hbmRzID0gb2ZmCiAjbG9nX3RlbXBfZmlsZXMgPSAtMQkJCSMgbG9nIHRlbXBvcmFy
eSBmaWxlcyBlcXVhbCBvciBsYXJnZXIKIAkJCQkJIyB0aGFuIHRoZSBzcGVjaWZpZWQgc2l6ZSBp
biBraWxvYnl0ZXM7CmRpZmYgLS1naXQgYS9zcmMvYmluL3BnX2N0bC90LzAwNF9sb2dyb3RhdGUu
cGwgYi9zcmMvYmluL3BnX2N0bC90LzAwNF9sb2dyb3RhdGUucGwKaW5kZXggZWFjY2ExYTY1Mi4u
MWMxZjg5Y2IwYSAxMDA2NDQKLS0tIGEvc3JjL2Jpbi9wZ19jdGwvdC8wMDRfbG9ncm90YXRlLnBs
CisrKyBiL3NyYy9iaW4vcGdfY3RsL3QvMDA0X2xvZ3JvdGF0ZS5wbApAQCAtNjksNiArNjksNyBA
QCBsb2dfZGVzdGluYXRpb24gPSAnc3RkZXJyLCBjc3Zsb2csIGpzb25sb2cnCiAjIHRoZXNlIGVu
c3VyZSBzdGFiaWxpdHkgb2YgdGVzdCByZXN1bHRzOgogbG9nX3JvdGF0aW9uX2FnZSA9IDAKIGxj
X21lc3NhZ2VzID0gJ0MnCittYXhfbG9nX3NpemUgPSAzMgogKSk7CiAKICRub2RlLT5zdGFydCgp
OwpAQCAtMTM1LDYgKzEzNiwyMCBAQCBjaGVja19sb2dfcGF0dGVybignc3RkZXJyJywgJG5ld19j
dXJyZW50X2xvZ2ZpbGVzLCAnc3ludGF4IGVycm9yJywgJG5vZGUpOwogY2hlY2tfbG9nX3BhdHRl
cm4oJ2NzdmxvZycsICRuZXdfY3VycmVudF9sb2dmaWxlcywgJ3N5bnRheCBlcnJvcicsICRub2Rl
KTsKIGNoZWNrX2xvZ19wYXR0ZXJuKCdqc29ubG9nJywgJG5ld19jdXJyZW50X2xvZ2ZpbGVzLCAn
c3ludGF4IGVycm9yJywgJG5vZGUpOwogCiskbm9kZS0+cHNxbCgncG9zdGdyZXMnLCAnSU5TRVJU
IElOVE8gU09NRV9OT05fRVhJU1RBTlRfVEFCTEUgVkFMVUVTIChURVNUKScpOworZm9yIChteSAk
YXR0ZW1wdHMgPSAwOyAkYXR0ZW1wdHMgPCAkbWF4X2F0dGVtcHRzOyAkYXR0ZW1wdHMrKykKK3sK
KwlldmFsIHsKKwkJJGN1cnJlbnRfbG9nZmlsZXMgPSBzbHVycF9maWxlKCRub2RlLT5kYXRhX2Rp
ciAuICcvY3VycmVudF9sb2dmaWxlcycpOworCX07CisJbGFzdCB1bmxlc3MgJEA7CisJdXNsZWVw
KDEwMF8wMDApOworfQorZGllICRAIGlmICRAOworY2hlY2tfbG9nX3BhdHRlcm4oJ3N0ZGVycics
ICAkY3VycmVudF9sb2dmaWxlcywgJ0lOU0VSVCBJTlRPIFNPTUVfTk9OX0VYSVNUQU5UX1RBKD8h
KEJMRSBWQUxVRVMgXChURVNUXCkpKScsICRub2RlKTsKK2NoZWNrX2xvZ19wYXR0ZXJuKCdjc3Zs
b2cnLCAgJGN1cnJlbnRfbG9nZmlsZXMsICdJTlNFUlQgSU5UTyBTT01FX05PTl9FWElTVEFOVF9U
QSg/IShCTEUgVkFMVUVTIFwoVEVTVFwpKSknLCAkbm9kZSk7CitjaGVja19sb2dfcGF0dGVybign
anNvbmxvZycsICRjdXJyZW50X2xvZ2ZpbGVzLCAnSU5TRVJUIElOVE8gU09NRV9OT05fRVhJU1RB
TlRfVEEoPyEoQkxFIFZBTFVFUyBcKFRFU1RcKSkpJywgJG5vZGUpOworCiAkbm9kZS0+c3RvcCgp
OwogCiBkb25lX3Rlc3RpbmcoKTsKZGlmZiAtLWdpdCBhL3NyYy9pbmNsdWRlL3V0aWxzL2Vsb2cu
aCBiL3NyYy9pbmNsdWRlL3V0aWxzL2Vsb2cuaAppbmRleCBlNTRlY2E1YjQ4Li5hOGUzMDAwNmYy
IDEwMDY0NAotLS0gYS9zcmMvaW5jbHVkZS91dGlscy9lbG9nLmgKKysrIGIvc3JjL2luY2x1ZGUv
dXRpbHMvZWxvZy5oCkBAIC01MDIsNiArNTAyLDcgQEAgZXh0ZXJuIFBHRExMSU1QT1JUIGludCBM
b2dfZGVzdGluYXRpb247CiBleHRlcm4gUEdETExJTVBPUlQgY2hhciAqTG9nX2Rlc3RpbmF0aW9u
X3N0cmluZzsKIGV4dGVybiBQR0RMTElNUE9SVCBib29sIHN5c2xvZ19zZXF1ZW5jZV9udW1iZXJz
OwogZXh0ZXJuIFBHRExMSU1QT1JUIGJvb2wgc3lzbG9nX3NwbGl0X21lc3NhZ2VzOworZXh0ZXJu
IFBHRExMSU1QT1JUIGludCBtYXhfbG9nX3NpemU7CiAKIC8qIExvZyBkZXN0aW5hdGlvbiBiaXRt
YXAgKi8KICNkZWZpbmUgTE9HX0RFU1RJTkFUSU9OX1NUREVSUgkgMQotLSAKMi4zOS4yIChBcHBs
ZSBHaXQtMTQzKQoK
--00000000000069af8b06236c3a53--