Subject: Log Stacktrace of current Python Interpreter via PostgreSQL trigger

From: Thomas Güttler <[email-masked]>
Date: Mon, 20 May 2019 09:43:48 +0200

I am hunting a non reproducible in a production environment.

I can detect the buggy change in a postgres trigger.

Since it is production code I must no raise an exception. I can
only use logging.

If I could see the stacktrace of the python interpreter, I could
see which codes the change which I am hunting.

But how to get this interpreter stacktrace, if the condition is
detect in the db trigger?

https://stackoverflow.com/questions/51873708/log-stacktrace-of-python-in-postgresql-trigger

Maybe there is a psycopg2 feature which I don't know up to now.

I guess LISTEN+NOTIFY could get used.
Or setting a connection variable which I check after each SQL statement.

Ideas welcome,

   Thomas Güttler

-- 
Thomas Guettler http://www.thomas-guettler.de/
I am looking for feedback: https://github.com/guettli/programming-guidelines

---

From: Daniele Varrazzo <[email-masked]>
Date: Mon, 20 May 2019 19:19:24 +0900

If you use postgres logging in stored procedures you can retrieve the logs
in 'connection.notices'.

http://initd.org/psycopg/docs/connection.html#connection.notices

On Mon, 20 May 2019, 16:40 Thomas Güttler, <[email-masked]>
wrote:

> I am looking for feedback:
> https://github.com/guettli/programming-guidelines
>
>
>

---

From: Thomas Güttler <[email-masked]>
Date: Mon, 20 May 2019 14:43:19 +0200

Am 20.05.19 um 12:19 schrieb Daniele Varrazzo:
> If you use postgres logging in stored procedures you can retrieve the logs in 'connection.notices'.
> 

This sound great. Unfortunately I can't extract the whole stacktrace.
I only get the lines below psycopg, not the above (lines of the callers).

Here is my code:

class MyAppConfig(AppConfig):

     def ready(self):
         connection_created.connect(connection_created_check_for_notice_in_connection)

class ConnectionNoticeList(object):
     def append(self, message):
         if not 'some_magic_of_db_trigger' in message:
             return
         logger.warn('%s %s' % (message, ''.join(traceback.format_stack())))

def connection_created_check_for_notice_in_connection(sender, connection, **kwargs):
     connection.connection.notices=ConnectionNoticeList()

I see this in the logs:

'NOTICE:  some_magic_of_db_trigger: 17909
      File "/snap/pycharm-community/128/helpers/pycharm/_jb_pytest_runner....ork/foo/apps.py", line 47, in append
       logger.warn(\'%s %s\' % (message, \'\'.join(traceback.format_stack())))
   '

traceback.format_stack() inside ConnectionNoticeList.append() extracts not the callers.

Is there a way to get the callers lines?

---

From: Thomas Güttler <[email-masked]>
Date: Tue, 4 Jun 2019 10:56:16 +0200

> Is there a way to get the callers lines?

Above code works. I see the whole traceback.

I don't know why the traceback was cut in PyCharm. In production I could see the whole traceback and I could find the 
broken code which modified the data in way which should not happen.

Many thanks to Daniele Varrazzo who provided the hint to overwrite connection.notices.

Regards,
   Thomas Güttler