pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedSuppress 'duplicate key' at errorlog?
8+ messages / 5 participants
[nested] [flat]
* Suppress 'duplicate key' at errorlog?
@ 2015-03-30 21:03 Rossi, Maria <maria.rossi@us.pgds.com>
0 siblings, 4 replies; 8+ messages in thread
From: Rossi, Maria @ 2015-03-30 21:03 UTC (permalink / raw)
To: pgsql-admin
Hi,
Is there a way to suppress a 'duplicate key' msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.
Thanks.
Maria
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Suppress 'duplicate key' at errorlog?
@ 2015-03-30 22:41 David G. Johnston <david.g.johnston@gmail.com>
parent: Rossi, Maria <maria.rossi@us.pgds.com>
3 siblings, 0 replies; 8+ messages in thread
From: David G. Johnston @ 2015-03-30 22:41 UTC (permalink / raw)
To: Rossi, Maria <maria.rossi@us.pgds.com>; +Cc: pgsql-admin
On Mon, Mar 30, 2015 at 2:03 PM, Rossi, Maria <maria.rossi@us.pgds.com>
wrote:
> Hi,
>
>
>
> Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due
> to a bug in a 3rd party application, this keeps on appearing and I am
> concerned that it will fill up the filesystem. I can delete the old
> errorlog, but if there is any other meaningful msgs out there, I would
> lose that too.
>
>
>
> Thanks.
>
>
>
> *Maria *
>
>
>
Not in any way that doesn't also entail the "but if there is any other
meaningful msgs out there, I would lose that too." risk.
Stop thinking about the limited problem of removing the excessive messages
and just learn how to deal with high-volume log situations generally. At
worse you could a simple log rotation and archive with a cleanup "sed"
script that deletes these messages. Suggest setting "application_name" in
order to be more targeted.
David J.
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Suppress 'duplicate key' at errorlog?
@ 2015-03-30 22:54 Scott Whitney <scott@journyx.com>
parent: Rossi, Maria <maria.rossi@us.pgds.com>
3 siblings, 1 reply; 8+ messages in thread
From: Scott Whitney @ 2015-03-30 22:54 UTC (permalink / raw)
To: David G. Johnston <david.g.johnston@gmail.com>; +Cc: Rossi, Maria <maria.rossi@us.pgds.com>; pgsql-admin
<p dir="ltr">I _do_ want to phone in on the whole duplicate key issue.</p>
<p dir="ltr">Sure, this could be a race condition. It could be bad programming. It could be a variety of things.</p>
<p dir="ltr">You ABSOLUTELY do not want to ignore that. Ignoring and moving on will cause consistency problems. Catch it and add it to a log or an email. Continue.</p>
<p dir="ltr">Do NOT silently ignore something trying to add a duplicate key.</p>
<br/><div class="cm_quote" style=" color: #787878">On Mon, Mar 30, 2015 at 5:45 PM, David G. Johnston <<a href="mailto:david.g.johnston@gmail.com">david.g.johnston@gmail.com</a>> wrote:</div><br><div id="oldcontent" style="background-color: rgb(255, 255, 255); background-position: initial initial; background-repeat: initial initial;"><blockquote style=""><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><span style="font-family:arial,sans-serif">On Mon, Mar 30, 2015 at 2:03 PM, Rossi, Maria </span><span dir="ltr" style="font-family:arial,sans-serif"><<a href="mailto:maria.rossi@us.pgds.com">maria.rossi@us.pgds.com</a>></span><span style="font-family:arial,sans-serif"> wrote:</span><br></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US">
<div>
<p class="MsoNormal">Hi,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3<sup>rd</sup> party application, this keeps on appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog,
but if there is any other meaningful msgs out there, I would lose that too.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Thanks. <span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span></p><span class="HOEnZb"><font color="#888888">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Times New Roman",serif;color:teal">Maria
</span></b><span style="color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</font></span></div>
</div>
</blockquote></div><br></div><div class="gmail_extra"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Not in any way that doesn't also entail the "but if there is any other meaningful msgs out there, I would lose that too." risk.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Stop thinking about the limited problem of removing the excessive messages and just learn how to deal with high-volume log situations generally. At worse you could a simple log rotation and archive with a cleanup "sed" script that deletes these messages. Suggest setting "application_name" in order to be more targeted.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">David J.</div><br></div></div>
</blockquote></div>
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Suppress 'duplicate key' at errorlog?
@ 2015-03-30 23:05 David G. Johnston <david.g.johnston@gmail.com>
parent: Scott Whitney <scott@journyx.com>
0 siblings, 0 replies; 8+ messages in thread
From: David G. Johnston @ 2015-03-30 23:05 UTC (permalink / raw)
To: Scott Whitney <scott@journyx.com>; +Cc: Rossi, Maria <maria.rossi@us.pgds.com>; pgsql-admin
>
>
>
> On Mon, Mar 30, 2015 at 5:45 PM, David G. Johnston <
> david.g.johnston@gmail.com> wrote:
>
> On Mon, Mar 30, 2015 at 2:03 PM, Rossi, Maria <maria.rossi@us.pgds.com>
> wrote:
>
>> Hi,
>>
>>
>>
>> Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due
>> to a bug in a 3rd party application, this keeps on appearing and I am
>> concerned that it will fill up the filesystem. I can delete the old
>> errorlog, but if there is any other meaningful msgs out there, I would
>> lose that too.
>>
>>
>>
>> Thanks.
>>
>>
>>
>> *Maria *
>>
>>
>>
>
> Not in any way that doesn't also entail the "but if there is any other
> meaningful msgs out there, I would lose that too." risk.
>
> Stop thinking about the limited problem of removing the excessive messages
> and just learn how to deal with high-volume log situations generally. At
> worse you could a simple log rotation and archive with a cleanup "sed"
> script that deletes these messages. Suggest setting "application_name" in
> order to be more targeted.
>
> David J.
>
>
On Mon, Mar 30, 2015 at 3:54 PM, Scott Whitney <scott@journyx.com> wrote:
> I _do_ want to phone in on the whole duplicate key issue.
>
> Sure, this could be a race condition. It could be bad programming. It
> could be a variety of things.
>
> You ABSOLUTELY do not want to ignore that. Ignoring and moving on will
> cause consistency problems. Catch it and add it to a log or an email.
> Continue.
>
> Do NOT silently ignore something trying to add a duplicate key.
>
The OP is "logging" the error...
A reasonably valid process would raise the error to the application which
would then retry the insert using a different key. Application works fine
but the error logs have a lot of expected errors in them. There is a
performance concern but not a data integrity one.
David J.
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Suppress 'duplicate key' at errorlog?
@ 2015-03-30 23:15 Scott Ribe <scott_ribe@elevated-dev.com>
parent: Rossi, Maria <maria.rossi@us.pgds.com>
3 siblings, 0 replies; 8+ messages in thread
From: Scott Ribe @ 2015-03-30 23:15 UTC (permalink / raw)
To: Rossi, Maria <maria.rossi@us.pgds.com>; +Cc: pgsql-admin
On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:
>
> Hi,
>
> Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.
No way to turn off that specific error message, but you could filter them out of the log, being careful of course not to filter any messages which might be more meaningful. You could also prevent the errors from being raised by using a trigger, but I don't like that idea very much at all.
--
Scott Ribe
scott_ribe@elevated-dev.com
http://www.elevated-dev.com/
https://www.linkedin.com/in/scottribe/
(303) 722-0567 voice
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Suppress 'duplicate key' at errorlog?
@ 2015-03-31 00:12 Scott Whitney <scott@journyx.com>
parent: Rossi, Maria <maria.rossi@us.pgds.com>
3 siblings, 1 reply; 8+ messages in thread
From: Scott Whitney @ 2015-03-31 00:12 UTC (permalink / raw)
To: Scott Ribe <scott_ribe@elevated-dev.com>; +Cc: Rossi, Maria <maria.rossi@us.pgds.com>; pgsql-admin
<p dir="ltr">Am I the only one concerned about a product trying to duplicate a pkey? This is not usually the LAST step. It's early on in the logical process. You have to find and solve the underlying cause.</p>
<br/><div class="cm_quote" style=" color: #787878">On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <<a href="mailto:scott_ribe@elevated-dev.com">scott_ribe@elevated-dev.com</a>> wrote:</div><br><div id="oldcontent" style="background-color: rgb(255, 255, 255); background-position: initial initial; background-repeat: initial initial;"><blockquote style=""><p dir="ltr">On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:
<br>
>
<br>
> Hi,
<br>
>
<br>
> Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.
<br>
<br>
No way to turn off that specific error message, but you could filter them out of the log, being careful of course not to filter any messages which might be more meaningful. You could also prevent the errors from being raised by using a trigger, but I don't like that idea very much at all.
<br>
<br>
<br>
--
<br>
Scott Ribe
<br>
scott_ribe@elevated-dev.com
<br>
http://www.elevated-dev.com/
<br>
https://www.linkedin.com/in/scottribe/
<br>
(303) 722-0567 voice
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
--
<br>
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
<br>
To make changes to your subscription:
<br>
http://www.postgresql.org/mailpref/pgsql-admin
<br>
</p>
</blockquote></div>
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Suppress 'duplicate key' at errorlog?
@ 2015-03-31 00:54 Jerry Sievers <gsievers19@comcast.net>
parent: Scott Whitney <scott@journyx.com>
0 siblings, 1 reply; 8+ messages in thread
From: Jerry Sievers @ 2015-03-31 00:54 UTC (permalink / raw)
To: Scott Whitney <scott@journyx.com>; +Cc: Scott Ribe <scott_ribe@elevated-dev.com>; Rossi\, Maria <maria.rossi@us.pgds.com>; pgsql-admin
Scott Whitney <scott@journyx.com> writes:
> Am I the only one concerned about a product trying to duplicate a pkey? This is not usually the LAST step. It's early on in the logical process. You have to find and
> solve the underlying cause.
I've seen lazy developers use a try insert/update if fail to implement a
poor man's upsert... a feature much sought after and only now being
seriously developed for Postgres.
So... whether or not it's a bug is application dependent.
Yes I'd question a 3rd party about this if the product wasn't developed
in house and/or fully open to code review to have them tell me "we did
it that way on purpose" though rather than just declaring it a nuisance.
FWIW
> On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
>
> On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:
> >
> > Hi,
> >
> > Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it
> will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.
>
> No way to turn off that specific error message, but you could filter them out of the log, being careful of course not to filter any messages which might be more
> meaningful. You could also prevent the errors from being raised by using a trigger, but I don't like that idea very much at all.
>
> --
> Scott Ribe
> scott_ribe@elevated-dev.com
> http://www.elevated-dev.com/
> https://www.linkedin.com/in/scottribe/
> (303) 722-0567 voice
>
> --
> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-admin
>
--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Suppress 'duplicate key' at errorlog?
@ 2015-03-31 13:02 Rossi, Maria <maria.rossi@us.pgds.com>
parent: Jerry Sievers <gsievers19@comcast.net>
0 siblings, 0 replies; 8+ messages in thread
From: Rossi, Maria @ 2015-03-31 13:02 UTC (permalink / raw)
To: Jerry Sievers <gsievers19@comcast.net>; Scott Whitney <scott@journyx.com>; +Cc: Scott Ribe <scott_ribe@elevated-dev.com>; pgsql-admin
Thanks for all the comments/suggestions.
As I mentioned, this is a 3rd-party software so what I can do to it is none or limited at best. I will try to filter the errorlog. Another option is to set the log_min_messages and log_min_error_statement to log.
Maria
-----Original Message-----
From: Jerry Sievers [mailto:gsievers19@comcast.net]
Sent: Monday, March 30, 2015 8:54 PM
To: Scott Whitney
Cc: Scott Ribe; Rossi, Maria; pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Suppress 'duplicate key' at errorlog?
Scott Whitney <scott@journyx.com> writes:
> Am I the only one concerned about a product trying to duplicate a
> pkey? This is not usually the LAST step. It's early on in the logical process. You have to find and solve the underlying cause.
I've seen lazy developers use a try insert/update if fail to implement a poor man's upsert... a feature much sought after and only now being seriously developed for Postgres.
So... whether or not it's a bug is application dependent.
Yes I'd question a 3rd party about this if the product wasn't developed in house and/or fully open to code review to have them tell me "we did it that way on purpose" though rather than just declaring it a nuisance.
FWIW
> On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
>
> On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:
> >
> > Hi,
> >
> > Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it
> will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.
>
> No way to turn off that specific error message, but you could filter them out of the log, being careful of course not to filter any messages which might be more
> meaningful. You could also prevent the errors from being raised by using a trigger, but I don't like that idea very much at all.
>
> --
> Scott Ribe
> scott_ribe@elevated-dev.com
> http://www.elevated-dev.com/
> https://www.linkedin.com/in/scottribe/
> (303) 722-0567 voice
>
> --
> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-admin
>
--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2015-03-31 13:02 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2015-03-30 21:03 Suppress 'duplicate key' at errorlog? Rossi, Maria <maria.rossi@us.pgds.com>
2015-03-30 22:41 ` David G. Johnston <david.g.johnston@gmail.com>
2015-03-30 22:54 ` Scott Whitney <scott@journyx.com>
2015-03-30 23:05 ` David G. Johnston <david.g.johnston@gmail.com>
2015-03-30 23:15 ` Scott Ribe <scott_ribe@elevated-dev.com>
2015-03-31 00:12 ` Scott Whitney <scott@journyx.com>
2015-03-31 00:54 ` Jerry Sievers <gsievers19@comcast.net>
2015-03-31 13:02 ` Rossi, Maria <maria.rossi@us.pgds.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox