pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
15+ messages / 5 participants
[nested] [flat]

* BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-09-26 18:29  marko@joh.to
  0 siblings, 2 replies; 15+ messages in thread

From: marko@joh.to @ 2017-09-26 18:29 UTC (permalink / raw)
  To: pgsql-bugs; +Cc: marko@joh.to

The following bug has been logged on the website:

Bug reference:      14830
Logged by:          Marko Tiikkaja
Email address:      marko@joh.to
PostgreSQL version: Unsupported/Unknown
Operating system:   Ubuntu 14.04
Description:        

Hey,

I understand this is not much information to go on (but the problem is
extremely difficult to reproduce), and that 9.1 is technically out of
support (but I don't think the relevant code has changed significantly,
either), so I fully expect that nobody will be able to figure out what's
wrong based on that.  But I thought I'd post anyway.

For the past two days I've been tracking down a bug where it would appear
that some NOTIFications are simply lost.  Then a minute later when the
notification is resent by a different transaction, it comes through just
fine.  We have a single program connected to the database all the time,
which LISTENs on around 800 channels and delivers the notifications to its
own clients.  The problem seems to only start happening, or perhaps gets
worse the longer this application is connected to the database.

I'm attaching two excerpts from the strace which, if I'm reading this
correctly, would suggest that there's a bug in postgres here.  Here's how I
read this:

  1) In strace2.txt, the send on line #1 corresponds to 28:3486 in
strace1.txt.  I know this because notification payloads on that channel are
unique.
  2) In strace2.txt, on line #5 something slightly out of the ordinary
happens.  We have around 75 semop calls compared to 5400 semop calls in the
full strace, so no biggie, but perhaps noteworthy.  Contention with another
backend, perhaps.
  3) The send on line #6 seems to correspond to 28:3600 in strace1.txt.
  4) Then here's where the problem seems to occur: the next send, on line
25, corresponds to 28:4458 in strace1.txt.

Within that ~850 bytes that the sending backend seemingly jumped over, we
have multiple notifications on channels we know the backend was listening
on.  That's including a notification on channel "workerid48101842", which is
the one our application was desperately missing in this case.  PostgreSQL's
logs and the state of the database indicate that at least the transaction
which wrote the "workerid48101842" notification committed, and I have no
reason to believe that any of the other ones near it did not commit.

So.. any ideas?  Unfortunately I can't reproduce this in an isolated
environment, and in production this seems to be taking some time before it
builds up into a proper issue.


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-09-26 18:32  Marko Tiikkaja <marko@joh.to>
  parent: marko@joh.to
  1 sibling, 0 replies; 15+ messages in thread

From: Marko Tiikkaja @ 2017-09-26 18:32 UTC (permalink / raw)
  To: pgsql-bugs

Here are said attachments.


.m
39287 12:53:50.158234 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=10663, si_uid=105} ---
39287 12:53:50.158375 open("pg_notify/3AA1", O_RDWR|O_CREAT, 0600) = 7
39287 12:53:50.158486 lseek(7, 204800, SEEK_SET) = 204800
39287 12:53:50.158576 write(7, "8\0\0\0\t=\6\0Vf\326\30\271\236\0\0api_cache_invalidation\0GSR1089556715\0\177\0\0$\0\0\0\t=\6\0Vf\326\30\271\236\0\0workerid39844881\0\0ti8\0\0\0\t=\6\0Uf\326\30X`\0\0api_cache_invalidation\0GSR1382568364\0=\6\0(\0\0\0\t=\6\0Uf\326\30X`\0\0RPC Worker\000423645699\0n\0G$\0\0\0\t=\6\0Uf\326\30X`\0\0orderid1382568364\0\0009$\0\0\0\t=\6\0Uf\326\30X`\0\0workerid40122970\0\0\0009$\0\0\0\t=\6\0^f\326\30\304\220\0\0orderid1332203663\0\0\0008\0\0\0\t=\6\0^f\326\30\304\220\0\0api_cache_invalidation\0GSR1332203663\0\177\0\0$\0\0\0\t=\6\0^f\326\30\304\220\0\0workerid48011112\0\0ti$\0\0\0\t=\6\0`f\326\30\304\220\0\0workerid48011112\0\0\0\0$\0\0\0\t=\6\0Wf\326\30/\234\0\0orderid1089556715\0\0\0008\0\0\0\t=\6\0Wf\326\30/\234\0\0api_cache_invalidation\0GSR1089556715\0=\6\0$\0\0\0\t=\6\0Wf\326\30/\234\0\0workerid42803398\0\0ti8\0\0\0\t=\6\0cf\326\30D\5\0\0api_cache_invalidation\0GSR2234430237\0\177\0\0$\0\0\0\t=\6\0cf\326\30D\5\0\0workerid40122970\0\0ti$\0\0\0\t=\6\0if\326\30\304\220\0\0orderid3388370685\0\0\0008\0\0\0\t=\6\0if\326\30\304\220\0\0api_cache_invalidation\0GSR3388370685\0\177\0\0$\0\0\0\t=\6\0if\326\30\304\220\0\0workerid48011112\0\0ti$\0\0\0\t=\6\0ef\326\30X`\0\0orderid2234430237\0\0\0008\0\0\0\t=\6\0ef\326\30X`\0\0api_cache_invalidation\0GSR2234430237\0=\6\0$\0\0\0\t=\6\0ef\326\30X`\0\0workerid40122970\0\0ti$\0\0\0\t=\6\0nf\326\30\304\220\0\0workerid48011112\0\0\0\0(\0\0\0\t=\6\0bf\326\30\247)\0\0RPC Worker\000423645700\0\0\0\0008\0\0\0\t=\6\0bf\326\30\247)\0\0api_cache_invalidation\0GSR1089556715\0=\6\0$\0\0\0\t=\6\0bf\326\30\247)\0\0orderid1089556715\0\0i$\0\0\0\t=\6\0bf\326\30\247)\0\0workerid42803398\0\0\0i$\0\0\0\t=\6\0qf\326\30\304\220\0\0orderid1332203663\0\0\0008\0\0\0\t=\6\0qf\326\30\304\220\0\0api_cache_invalidation\0GSR1332203663\0\177\0\0$\0\0\0\t=\6\0qf\326\30\304\220\0\0workerid48011112\0\0ti$\0\0\0\t=\6\0sf\326\30\304\220\0\0workerid48011112\0\0\0\0008\0\0\0\t=\6\0of\326\30X`\0\0api_cache_invalidation\0GSR2234430237\0=\6\0(\0\0\0\t=\6\0of\326\30X`\0\0RPC Worker\000423645701\0n\0G$\0\0\0\t=\6\0of\326\30X`\0\0orderid2234430237\0\0001$\0\0\0\t=\6\0of\326\30X`\0\0workerid40122970\0\0\00018\0\0\0\t=\6\0xf\326\30m\217\0\0api_cache_invalidation\0GSR3090422524\0\177\0\0$\0\0\0\t=\6\0xf\326\30m\217\0\0workerid48011128\0\0ti$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102005\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101905\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102007\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48097441\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102010\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48096056\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102037\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102034\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101204\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48089722\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102192\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102181\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid47993000\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101533\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101029\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101509\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102020\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48100315\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48100706\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101535\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48099049\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101186\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101357\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48100319\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48100913\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102090\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102264\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102014\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48101581\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48102322\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48092682\0\0\346:$\0\0\0\t=\6\0rf\326\30\24S\0\0workerid48092738\0\0\346:8\0\0\0\t=\6\0wf\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0(\0\0\0\t=\6\0\227f\326\30?A\0\0RPC Worker\000423645702\0\177\0\0$\0\0\0\t=\6\0\227f\326\30?A\0\0orderid3721676666\0\0002$\0\0\0\t=\6\0\241f\326\30?A\0\0orderid1153754738\0\0\0008\0\0\0\t=\6\0\241f\326\30?A\0\0api_cache_invalidation\0GSR1153754738\0\177\0\0$\0\0\0\t=\6\0\241f\326\30?A\0\0workerid48011188\0\0ti8\0\0\0\t=\6\0\237f\326\0309\223\0\0api_cache_invalidation\0GSR1368137862\0\177\0\0$\0\0\0\t=\6\0\237f\326\0309\223\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0\247f\326\30\327d\0\0workerid48011188\0\0\0\0$\0\0\0\t=\6\0\256f\326\30uB\0\0orderid3969700415\0\0\0008\0\0\0\t=\6\0\256f\326\30uB\0\0api_cache_invalidation\0GSR3969700415\0\177\0\0$\0\0\0\t=\6\0\256f\326\30uB\0\0workerid48102029\0\0ti$\0\0\0\t=\6\0\261f\326\30?A\0\0orderid1368137862\0\0\0008\0\0\0\t=\6\0\261f\326\30?A\0\0api_cache_invalidation\0GSR1368137862\0\177\0\0$\0\0\0\t=\6\0\261f\326\30?A\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0\262f\326\30\304\220\0\0workerid48102029\0\0\0\0$\0\0\0\t=\6\0\264f\326\30uB\0\0workerid48011410\0\0\0\0$\0\0\0\t=\6\0\263f\326\30\304\220\0\0orderid1153754738\0\0\0008\0\0\0\t=\6\0\263f\326\30\304\220\0\0api_cache_invalidation\0GSR1153754738\0\177\0\0$\0\0\0\t=\6\0\263f\326\30\304\220\0\0workerid48011188\0\0ti8\0\0\0\t=\6\0\265f\326\30\5\237\0\0api_cache_invalidation\0GSR2709881967\0\177\0\0$\0\0\0\t=\6\0\265f\326\30\5\237\0\0workerid39844907\0\0ti$\0\0\0\t=\6\0\270f\326\30\304\220\0\0workerid48011188\0\0\0\0008\0\0\0\t=\6\0\272f\326\0300]\0\0api_cache_invalidation\0GSR4124471034\0\177\0\0$\0\0\0\t=\6\0\272f\326\0300]\0\0workerid40122970\0\0ti$\0\0\0\t=\6\0\307f\326\30uB\0\0orderid1574931444\0\0\0008\0\0\0\t=\6\0\307f\326\30uB\0\0api_cache_invalidation\0GSR1574931444\0\177\0\0$\0\0\0\t=\6\0\307f\326\30uB\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\304f\326\30N\217\0\0orderid3318876617\0\0\0008\0\0\0\t=\6\0\304f\326\30N\217\0\0api_cache_invalidation\0GSR3318876617\0\177\0\0$\0\0\0\t=\6\0\304f\326\30N\217\0\0workerid48094158\0\0ti$\0\0\0\t=\6\0\312f\326\30uB\0\0workerid48011104\0\0\0\0008\0\0\0\t=\6\0\311f\326\30\25\222\0\0api_cache_invalidation\0GSR1280623521\0\177\0\0$\0\0\0\t=\6\0\311f\326\30\25\222\0\0workerid48011114\0\0ti$\0\0\0\t=\6\0\310f\326\30\304\220\0\0orderid1686571787\0\0\0008\0\0\0\t=\6\0\310f\326\30\304\220\0\0api_cache_invalidation\0GSR1686571787\0\177\0\0$\0\0\0\t=\6\0\310f\326\30\304\220\0\0workerid48101220\0\0ti$\0\0\0\t=\6\0\303f\326\30X`\0\0orderid4124471034\0\0\0008\0\0\0\t=\6\0\303f\326\30X`\0\0api_cache_invalidation\0GSR4124471034\0=\6\0$\0\0\0\t=\6\0\303f\326\30X`\0\0workerid40122970\0\0ti$\0\0\0\t=\6\0\320f\326\30uB\0\0workerid48101220\0\0\0\0$\0\0\0\t=\6\0\321f\326\30\304\220\0\0orderid1574931444\0\0\0008\0\0\0\t=\6\0\321f\326\30\304\220\0\0api_cache_invalidation\0GSR1574931444\0\177\0\0$\0\0\0\t=\6\0\321f\326\30\304\220\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\323f\326\30uB\0\0workerid48011104\0\0\0\0$\0\0\0\t=\6\0\271f\326\30\343\234\0\0orderid2709881967\0\0\0008\0\0\0\t=\6\0\271f\326\30\343\234\0\0api_cache_invalidation\0GSR2709881967\0=\6\0$\0\0\0\t=\6\0\271f\326\30\343\234\0\0workerid48101535\0\0ti8\0\0\0\t=\6\0\317f\326\30X`\0\0api_cache_invalidation\0GSR4124471034\0=\6\0(\0\0\0\t=\6\0\317f\326\30X`\0\0RPC Worker\000423645703\0n\0G$\0\0\0\t=\6\0\317f\326\30X`\0\0orderid4124471034\0\0003$\0\0\0\t=\6\0\317f\326\30X`\0\0workerid40122970\0\0\00038\0\0\0\t=\6\0\352f\326\30&\235\0\0api_cache_invalidation\0GSR2804509056\0\177\0\0$\0\0\0\t=\6\0\352f\326\30&\235\0\0workerid42803405\0\0ti$\0\0\0\t=\6\0\346f\326\30R\3\0\0orderid3711661531\0\0<$\0\0\0\t=\6\0\346f\326\30R\3\0\0workerid39844907\0\0\0<8\0\0\0\t=\6\0\344f\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0\355f\326\30\256)\0\0orderid2804509056\0\0\0008\0\0\0\t=\6\0\355f\326\30\256)\0\0api_cache_invalidation\0GSR2804509056\0=\6\0$\0\0\0\t=\6\0\355f\326\30\256)\0\0workerid39844881\0\0ti$\0\0\0\t=\6\0\371f\326\30\304\220\0\0orderid1368137862\0\0\0008\0\0\0\t=\6\0\371f\326\30\304\220\0\0api_cache_invalidation\0GSR1368137862\0\177\0\0$\0\0\0\t=\6\0\371f\326\30\304\220\0\0workerid48011410\0\0ti(\0\0\0\t=\6\0\367f\326\30uB\0\0RPC Worker\000423645704\0\177\0\0$\0\0\0\t=\6\0\367f\326\30uB\0\0orderid2783550638\0\0004$\0\0\0\t=\6\0\0g\326\30?A\0\0workerid48011410\0\0\0\0008\0\0\0\t=\6\0\2g\326\0305\232\0\0api_cache_invalidation\0GSR1724534419\0\0\0\0$\0\0\0\t=\6\0\2g\326\0305\232\0\0workerid39844894\0\0ti$\0\0\0\t=\6\0\5g\326\30?A\0\0orderid3505409988\0\0\0008\0\0\0\t=\6\0\5g\326\30?A\0\0api_cache_invalidation\0GSR3505409988\0\177\0\0$\0\0\0\t=\6\0\5g\326\30?A\0\0workerid48101561\0\0ti$\0\0\0\t=\6\0\vg\326\30\327d\0\0workerid48101561\0\0\0\0(\0\0\0\t=\6\0\1g\326\30{\206\0\0RPC Worker\000423645705\0\177\0\0$\0\0\0\t=\6\0\370f\326\30/\234\0\0orderid2804509056\0\0\0008\0\0\0\t=\6\0\370f\326\30/\234\0\0api_cache_invalidation\0GSR2804509056\0=\6\0$\0\0\0\t=\6\0\370f\326\30/\234\0\0workerid39844881\0\0ti$\0\0\0\t=\6\0\6g\326\30\\\234\0\0orderid1724534419\0\0\0008\0\0\0\t=\6\0\6g\326\30\\\234\0\0api_cache_invalidation\0GSR1724534419\0=\6\0$\0\0\0\t=\6\0\6g\326\30\\\234\0\0workerid39844894\0\0ti8\0\0\0\t=\6\0\22g\326\30\\\234\0\0api_cache_invalidation\0GSR1724534419\0=\6\0$\0\0\0\t=\6\0\22g\326\30\\\234\0\0orderid1724534419\0\0i$\0\0\0\t=\6\0\22g\326\30\\\234\0\0workerid39844894\0\0\0i(\0\0\0\t=\6\0\22g\326\30\\\234\0\0RPC Worker\000423645706\0n\0G8\0\0\0\t=\6\0\35g\326\30n\232\0\0api_cache_invalidation\0GSR1361312496\0\177\0\0$\0\0\0\t=\6\0\35g\326\30n\232\0\0workerid42803401\0\0ti$\0\0\0\t=\6\0\36g\326\30N\217\0\0orderid2261969345\0\0\0008\0\0\0\t=\6\0\36g\326\30N\217\0\0api_cache_invalidation\0GSR2261969345\0\177\0\0$\0\0\0\t=\6\0\36g\326\30N\217\0\0workerid48094158\0\0ti(\0\0\0\t=\6\0%g\326\30\33f\0\0RPC Worker\000423645707\0\0\0\0$\0\0\0\t=\6\0(g\326\30\327d\0\0orderid1889579687\0\0\0008\0\0\0\t=\6\0(g\326\30\327d\0\0api_cache_invalidation\0GSR1889579687\0\177\0\0$\0\0\0\t=\6\0(g\326\30\327d\0\0workerid48101935\0\0ti$\0\0\0\t=\6\0+g\326\30\327d\0\0workerid48101935\0\0\0\0$\0\0\0\t=\6\0-g\326\30\327d\0\0orderid1737859930\0\0\0008\0\0\0\t=\6\0-g\326\30\327d\0\0api_cache_invalidation\0GSR1737859930\0\177\0\0$\0\0\0\t=\6\0-g\326\30\327d\0\0workerid48102014\0\0ti$\0\0\0\t=\6\0!g\326\30\253)\0\0orderid1361312496\0\0\0008\0\0\0\t=\6\0!g\326\30\253)\0\0api_cache_invalidation\0GSR1361312496\0=\6\0$\0\0\0\t=\6\0!g\326\30\253)\0\0workerid48100522\0\0ti$\0\0\0\t=\6\0000g\326\30\327d\0\0workerid48102014\0\0\0\0$\0\0\0\t=\6\0?g\326\30\227\r\0\0orderid4268124171\0\0;$\0\0\0\t=\6\0?g\326\30\227\r\0\0workerid39844876\0\0\0;8\0\0\0\t=\6\0Ig\326\30\313\1\0\0api_cache_invalidation\0GSR3160677022\0\177\0\0$\0\0\0\t=\6\0Ig\326\30\313\1\0\0workerid48101863\0\0ti$\0\0\0\t=\6\0Og\326\30\327d\0\0orderid2785837120\0\0\0008\0\0\0\t=\6\0Og\326\30\327d\0\0api_cache_invalidation\0GSR2785837120\0\177\0\0$\0\0\0\t=\6\0Og\326\30\327d\0\0workerid48102100\0\0ti<\0\0\0\t=\6\0Ng\326\30\353\4\0\0CLIENT_FUNDS_SPAIN_BBVA_2\0bankwithdrawal\0\0\0\0$\0\0\0\t=\6\0Tg\326\30\327d\0\0workerid48102100\0\0\0\0008\0\0\0\t=\6\0>g\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0Jg\326\30'\234\0\0orderid4268124171\0\0\0008\0\0\0\t=\6\0Jg\326\30'\234\0\0api_cache_invalidation\0GSR4268124171\0=\6\0$\0\0\0\t=\6\0Jg\326\30'\234\0\0workerid39844876\0\0ti$\0\0\0\t=\6\0`g\326\30\327d\0\0orderid2593971518\0\0\0008\0\0\0\t=\6\0`g\326\30\327d\0\0api_cache_invalidation\0GSR2593971518\0\177\0\0$\0\0\0\t=\6\0`g\326\30\327d\0\0workerid48011114\0\0ti$\0\0\0\t=\6\0cg\326\30\327d\0\0workerid48011114\0\0\0\0$\0\0\0\t=\6\0fg\326\30\327d\0\0orderid1280623521\0\0\0008\0\0\0\t=\6\0fg\326\30\327d\0\0api_cache_invalidation\0GSR1280623521\0\177\0\0$\0\0\0\t=\6\0fg\326\30\327d\0\0workerid48011114\0\0ti$\0\0\0\t=\6\0ig\326\30\327d\0\0workerid48011114\0\0\0\0008\0\0\0\t=\6\0\326f\326\30\347\20\0\0api_cache_invalidation\0GSR3740233186\0\177\0\0$\0\0\0\t=\6\0\326f\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0\326f\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\0\0\0008\0\0\0\t=\6\0qg\326\30h\222\0\0api_cache_invalidation\0GSR1640929266\0\177\0\0$\0\0\0\t=\6\0qg\326\30h\222\0\0workerid48011096\0\0ti$\0\0\0\t=\6\0sg\326\30\327d\0\0orderid2396627220\0\0\0008\0\0\0\t=\6\0sg\326\30\327d\0\0api_cache_invalidation\0GSR2396627220\0\177\0\0$\0\0\0\t=\6\0sg\326\30\327d\0\0workerid48011224\0\0ti\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 8192) = 8192
39287 12:53:50.158792 close(7)          = 0
39287 12:53:50.158848 open("pg_notify/3AA1", O_RDWR|O_CREAT, 0600) = 7
39287 12:53:50.158900 lseek(7, 212992, SEEK_SET) = 212992
39287 12:53:50.158939 write(7, "$\0\0\0\t=\6\0ug\326\30\327d\0\0workerid48011224\0\0\0\0$\0\0\0\t=\6\0og\326\30I\234\0\0orderid3740233186\0\0\0008\0\0\0\t=\6\0og\326\30I\234\0\0api_cache_invalidation\0GSR3740233186\0=\6\0$\0\0\0\t=\6\0og\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0vg\326\30I\234\0\0orderid3740233186\0\0\0$\0\0\0\t=\6\0|g\326\30\327d\0\0orderid1640929266\0\0\0008\0\0\0\t=\6\0|g\326\30\327d\0\0api_cache_invalidation\0GSR1640929266\0\177\0\0$\0\0\0\t=\6\0|g\326\30\327d\0\0workerid48011096\0\0ti$\0\0\0\t=\6\0\177g\326\30?A\0\0workerid48011096\0\0\0\0$\0\0\0\t=\6\0\203g\326\30?A\0\0orderid2396627220\0\0\0008\0\0\0\t=\6\0\203g\326\30?A\0\0api_cache_invalidation\0GSR2396627220\0\177\0\0$\0\0\0\t=\6\0\203g\326\30?A\0\0workerid48011224\0\0ti$\0\0\0\t=\6\0\202g\326\30\327d\0\0orderid3196802946\0\0\0008\0\0\0\t=\6\0\202g\326\30\327d\0\0api_cache_invalidation\0GSR3196802946\0\177\0\0$\0\0\0\t=\6\0\202g\326\30\327d\0\0workerid48102282\0\0ti$\0\0\0\t=\6\0\206g\326\30?A\0\0workerid48102282\0\0\0\0$\0\0\0\t=\6\0\211g\326\30\327d\0\0workerid48011224\0\0\0\0008\0\0\0\t=\6\0\215g\326\30+\3\0\0api_cache_invalidation\0GSR3039800402\0\177\0\0$\0\0\0\t=\6\0\215g\326\30+\3\0\0workerid48011188\0\0ti8\0\0\0\t=\6\0zg\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0008\0\0\0\t=\6\0\221g\326\30E\217\0\0api_cache_invalidation\0GSR3706058774\0\177\0\0$\0\0\0\t=\6\0\221g\326\30E\217\0\0workerid48011204\0\0ti$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102005\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48101905\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102007\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48097441\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102010\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48096056\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102037\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102034\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48101204\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48089722\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102192\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102181\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid47993000\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48101949\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102323\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102324\0\0\346:$\0\0\0\t=\6\0\204g\326\30\24S\0\0workerid48102325\0\0\346:$\0\0\0\t=\6\0\226g\326\30\327d\0\0orderid3706058774\0\0\0008\0\0\0\t=\6\0\226g\326\30\327d\0\0api_cache_invalidation\0GSR3706058774\0\177\0\0$\0\0\0\t=\6\0\226g\326\30\327d\0\0workerid48011204\0\0ti$\0\0\0\t=\6\0\230g\326\30?A\0\0workerid48011204\0\0\0\0008\0\0\0\t=\6\0pg\326\30\347\20\0\0api_cache_invalidation\0GSR1360486349\0\177\0\0$\0\0\0\t=\6\0pg\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0pg\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\224g\326\30\230\r\0\0orderid2150497078\0\0\0$\0\0\0\t=\6\0\224g\326\30\230\r\0\0workerid42803426\0\0\0\0$\0\0\0\t=\6\0\246g\326\30?A\0\0orderid4201123660\0\0\0008\0\0\0\t=\6\0\246g\326\30?A\0\0api_cache_invalidation\0GSR4201123660\0\177\0\0$\0\0\0\t=\6\0\246g\326\30?A\0\0workerid48101242\0\0ti$\0\0\0\t=\6\0\235g\326\30I\234\0\0orderid1360486349\0\0\0008\0\0\0\t=\6\0\235g\326\30I\234\0\0api_cache_invalidation\0GSR1360486349\0=\6\0$\0\0\0\t=\6\0\235g\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0\255g\326\30uB\0\0workerid48101242\0\0\0\0$\0\0\0\t=\6\0\256g\326\30\304\220\0\0orderid3039800402\0\0\0008\0\0\0\t=\6\0\256g\326\30\304\220\0\0api_cache_invalidation\0GSR3039800402\0\177\0\0$\0\0\0\t=\6\0\256g\326\30\304\220\0\0workerid48011188\0\0ti$\0\0\0\t=\6\0\262g\326\30uB\0\0workerid48011188\0\0\0\0$\0\0\0\t=\6\0\251g\326\30I\234\0\0orderid1360486349\0\0\0$\0\0\0\t=\6\0\265g\326\30uB\0\0orderid1280623521\0\0\0008\0\0\0\t=\6\0\265g\326\30uB\0\0api_cache_invalidation\0GSR1280623521\0\177\0\0$\0\0\0\t=\6\0\265g\326\30uB\0\0workerid48011114\0\0ti$\0\0\0\t=\6\0\266g\326\30?A\0\0orderid2709881967\0\0\0008\0\0\0\t=\6\0\266g\326\30?A\0\0api_cache_invalidation\0GSR2709881967\0\177\0\0$\0\0\0\t=\6\0\266g\326\30?A\0\0workerid48101535\0\0ti$\0\0\0\t=\6\0\267g\326\30uB\0\0workerid48011114\0\0\0\0$\0\0\0\t=\6\0\271g\326\30uB\0\0workerid48101535\0\0\0\0$\0\0\0\t=\6\0\241g\326\30\377)\0\0orderid2150497078\0\0\0008\0\0\0\t=\6\0\241g\326\30\377)\0\0api_cache_invalidation\0GSR2150497078\0=\6\0$\0\0\0\t=\6\0\241g\326\30\377)\0\0workerid42803426\0\0ti$\0\0\0\t=\6\0\304g\326\30?A\0\0orderid1069829288\0\0\0008\0\0\0\t=\6\0\304g\326\30?A\0\0api_cache_invalidation\0GSR1069829288\0\177\0\0$\0\0\0\t=\6\0\304g\326\30?A\0\0workerid48101949\0\0ti8\0\0\0\t=\6\0\303g\326\30@\217\0\0api_cache_invalidation\0GSR1332203663\0\177\0\0$\0\0\0\t=\6\0\303g\326\30@\217\0\0workerid48011112\0\0ti$\0\0\0\t=\6\0\305g\326\30?A\0\0workerid48101949\0\0\0\0008\0\0\0\t=\6\0\240g\326\30\347\20\0\0api_cache_invalidation\0GSR3452600087\0\177\0\0$\0\0\0\t=\6\0\240g\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\240g\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\316g\326\30?A\0\0orderid2709881967\0\0\0008\0\0\0\t=\6\0\316g\326\30?A\0\0api_cache_invalidation\0GSR2709881967\0\177\0\0$\0\0\0\t=\6\0\316g\326\30?A\0\0workerid48101535\0\0ti<\0\0\0\t=\6\0\311g\326\30jG\0\0CLIENT_FUNDS_FINLAND_OKOY\0bankwithdrawal\0\225>;$\0\0\0\t=\6\0\321g\326\30\327d\0\0orderid1332203663\0\0\0008\0\0\0\t=\6\0\321g\326\30\327d\0\0api_cache_invalidation\0GSR1332203663\0\177\0\0$\0\0\0\t=\6\0\321g\326\30\327d\0\0workerid48011112\0\0ti$\0\0\0\t=\6\0\324g\326\30\327d\0\0workerid48011112\0\0\0\0$\0\0\0\t=\6\0\312g\326\30\260)\0\0orderid3452600087\0\0\0008\0\0\0\t=\6\0\312g\326\30\260)\0\0api_cache_invalidation\0GSR3452600087\0=\6\0$\0\0\0\t=\6\0\312g\326\30\260)\0\0workerid42803406\0\0ti$\0\0\0\t=\6\0\335g\326\30uB\0\0orderid1361312496\0\0\0008\0\0\0\t=\6\0\335g\326\30uB\0\0api_cache_invalidation\0GSR1361312496\0\177\0\0$\0\0\0\t=\6\0\335g\326\30uB\0\0workerid48100522\0\0ti$\0\0\0\t=\6\0\334g\326\30\327d\0\0orderid1640929266\0\0\0008\0\0\0\t=\6\0\334g\326\30\327d\0\0api_cache_invalidation\0GSR1640929266\0\177\0\0$\0\0\0\t=\6\0\334g\326\30\327d\0\0workerid48011096\0\0ti$\0\0\0\t=\6\0\336g\326\30\327d\0\0workerid48100522\0\0\0\0$\0\0\0\t=\6\0\337g\326\30\304\220\0\0workerid48011096\0\0\0\0$\0\0\0\t=\6\0\340g\326\30?A\0\0orderid3090422524\0\0\0008\0\0\0\t=\6\0\340g\326\30?A\0\0api_cache_invalidation\0GSR3090422524\0\177\0\0$\0\0\0\t=\6\0\340g\326\30?A\0\0workerid48011128\0\0ti$\0\0\0\t=\6\0\332g\326\30\260)\0\0orderid3452600087\0\0\0$\0\0\0\t=\6\0\350g\326\30uB\0\0workerid48011128\0\0\0\0008\0\0\0\t=\6\0\352g\326\30\303\220\0\0api_cache_invalidation\0GSR2097988329\0\177\0\0$\0\0\0\t=\6\0\352g\326\30\303\220\0\0workerid48102145\0\0ti8\0\0\0\t=\6\0\333g\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0008\0\0\0\t=\6\0\362g\326\30\273\223\0\0api_cache_invalidation\0GSR2564376963\0\177\0\0$\0\0\0\t=\6\0\362g\326\30\273\223\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\364g\326\30?A\0\0orderid2824179994\0\0\0008\0\0\0\t=\6\0\364g\326\30?A\0\0api_cache_invalidation\0GSR2824179994\0\177\0\0$\0\0\0\t=\6\0\364g\326\30?A\0\0workerid48102111\0\0ti$\0\0\0\t=\6\0\367g\326\30?A\0\0workerid48102111\0\0\0\0008\0\0\0\t=\6\0\366g\326\30\27\217\0\0api_cache_invalidation\0GSR2261969345\0\177\0\0$\0\0\0\t=\6\0\366g\326\30\27\217\0\0workerid48094158\0\0ti$\0\0\0\t=\6\0\371g\326\30\327d\0\0orderid1361312496\0\0\0008\0\0\0\t=\6\0\371g\326\30\327d\0\0api_cache_invalidation\0GSR1361312496\0\177\0\0$\0\0\0\t=\6\0\371g\326\30\327d\0\0workerid48100522\0\0ti$\0\0\0\t=\6\0\373g\326\30?A\0\0orderid3706058774\0\0\0008\0\0\0\t=\6\0\373g\326\30?A\0\0api_cache_invalidation\0GSR3706058774\0\177\0\0$\0\0\0\t=\6\0\373g\326\30?A\0\0workerid48011204\0\0ti$\0\0\0\t=\6\0\377g\326\30?A\0\0orderid2097988329\0\0\0008\0\0\0\t=\6\0\377g\326\30?A\0\0api_cache_invalidation\0GSR2097988329\0\177\0\0$\0\0\0\t=\6\0\377g\326\30?A\0\0workerid48102145\0\0ti$\0\0\0\t=\6\0\0h\326\30\327d\0\0workerid48011204\0\0\0\0$\0\0\0\t=\6\0\2h\326\30\327d\0\0workerid48102145\0\0\0\0008\0\0\0\t=\6\0\315g\326\30\347\20\0\0api_cache_invalidation\0GSR2128252967\0\177\0\0$\0\0\0\t=\6\0\315g\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\315g\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\2248\0\0\0\t=\6\0\3h\326\30\227\206\0\0api_cache_invalidation\0GSR3375949113\0\177\0\0$\0\0\0\t=\6\0\3h\326\30\227\206\0\0workerid39844909\0\0ti$\0\0\0\t=\6\0\7h\326\30uB\0\0orderid2564376963\0\0\0008\0\0\0\t=\6\0\7h\326\30uB\0\0api_cache_invalidation\0GSR2564376963\0\177\0\0$\0\0\0\t=\6\0\7h\326\30uB\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\5h\326\30\260)\0\0orderid2128252967\0\0\0008\0\0\0\t=\6\0\5h\326\30\260)\0\0api_cache_invalidation\0GSR2128252967\0=\6\0$\0\0\0\t=\6\0\5h\326\30\260)\0\0workerid42803406\0\0ti$\0\0\0\t=\6\0\16h\326\30\260)\0\0orderid2128252967\0\0\0$\0\0\0\t=\6\0\24h\326\30uB\0\0orderid3160677022\0\0\0008\0\0\0\t=\6\0\24h\326\30uB\0\0api_cache_invalidation\0GSR3160677022\0\177\0\0$\0\0\0\t=\6\0\24h\326\30uB\0\0workerid48101863\0\0ti$\0\0\0\t=\6\0\27h\326\30uB\0\0workerid48101863\0\0\0\0$\0\0\0\t=\6\0\31h\326\30\327d\0\0orderid3039800402\0\0\0008\0\0\0\t=\6\0\31h\326\30\327d\0\0api_cache_invalidation\0GSR3039800402\0\177\0\0$\0\0\0\t=\6\0\31h\326\30\327d\0\0workerid48011188\0\0ti$\0\0\0\t=\6\0\33h\326\30\327d\0\0workerid48011188\0\0\0\0$\0\0\0\t=\6\0\6h\326\30d\235\0\0orderid3375949113\0\0\0008\0\0\0\t=\6\0\6h\326\30d\235\0\0api_cache_invalidation\0GSR3375949113\0=\6\0$\0\0\0\t=\6\0\6h\326\30d\235\0\0workerid48101842\0\0ti8\0\0\0\t=\6\0\37h\326\30f\217\0\0api_cache_invalidation\0GSR2335984064\0\177\0\0$\0\0\0\t=\6\0\37h\326\30f\217\0\0workerid42803401\0\0ti8\0\0\0\t=\6\0%h\326\30\351\234\0\0api_cache_invalidation\0GSR2563038584\0\177\0\0$\0\0\0\t=\6\0%h\326\30\351\234\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0&h\326\30uB\0\0orderid2097988329\0\0\0008\0\0\0\t=\6\0&h\326\30uB\0\0api_cache_invalidation\0GSR2097988329\0\177\0\0$\0\0\0\t=\6\0&h\326\30uB\0\0workerid48102145\0\0ti$\0\0\0\t=\6\0)h\326\30\327d\0\0workerid48102145\0\0\0\0008\0\0\0\t=\6\0\10h\326\30\347\20\0\0api_cache_invalidation\0GSR2974232622\0\177\0\0$\0\0\0\t=\6\0\10h\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\10h\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0000h\326\30uB\0\0orderid2563038584\0\0\0008\0\0\0\t=\6\0000h\326\30uB\0\0api_cache_invalidation\0GSR2563038584\0\177\0\0$\0\0\0\t=\6\0000h\326\30uB\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0'h\326\30N\217\0\0orderid2261969345\0\0\0008\0\0\0\t=\6\0'h\326\30N\217\0\0api_cache_invalidation\0GSR2261969345\0\0\0\0$\0\0\0\t=\6\0'h\326\30N\217\0\0workerid48094158\0\0ti$\0\0\0\t=\6\0007h\326\30uB\0\0workerid48011410\0\0\0\0$\0\0\0\t=\6\0 h\326\30\253)\0\0orderid2335984064\0\0\0008\0\0\0\t=\6\0 h\326\30\253)\0\0api_cache_invalidation\0GSR2335984064\0=\6\0$\0\0\0\t=\6\0 h\326\30\253)\0\0workerid48011114\0\0ti$\0\0\0\t=\6\0,h\326\30\260)\0\0orderid2974232622\0\0\0008\0\0\0\t=\6\0,h\326\30\260)\0\0api_cache_invalidation\0GSR2974232622\0=\6\0$\0\0\0\t=\6\0,h\326\30\260)\0\0workerid42803406\0\0ti$\0\0\0\t=\6\0?h\326\30\327d\0\0orderid1332203663\0\0\0008\0\0\0\t=\6\0?h\326\30\327d\0\0api_cache_invalidation\0GSR1332203663\0\177\0\0$\0\0\0\t=\6\0?h\326\30\327d\0\0workerid48011112\0\0ti$\0\0\0\t=\6\0.h\326\30k\235\0\0orderid3103146891\0\0<$\0\0\0\t=\6\0.h\326\30k\235\0\0workerid42803423\0\0\0<$\0\0\0\t=\6\0Bh\326\30\327d\0\0workerid48011112\0\0\0\0$\0\0\0\t=\6\0Gh\326\30?A\0\0orderid2785837120\0\0\0008\0\0\0\t=\6\0Gh\326\30?A\0\0api_cache_invalidation\0GSR2785837120\0\177\0\0$\0\0\0\t=\6\0Gh\326\30?A\0\0workerid48102100\0\0ti$\0\0\0\t=\6\0Ih\326\30\327d\0\0workerid48102100\0\0\0\0$\0\0\0\t=\6\0Ah\326\30\260)\0\0orderid2974232622\0\0\0008\0\0\0\t=\6\0>h\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0Eh\326\30\361)\0\0orderid3103146891\0\0\0008\0\0\0\t=\6\0Eh\326\30\361)\0\0api_cache_invalidation\0GSR3103146891\0=\6\0$\0\0\0\t=\6\0Eh\326\30\361)\0\0workerid42803423\0\0ti$\0\0\0\t=\6\0\\h\326\30\304\220\0\0orderid2819124331\0\0\0008\0\0\0\t=\6\0\\h\326\30\304\220\0\0api_cache_invalidation\0GSR2819124331\0\177\0\0$\0\0\0\t=\6\0\\h\326\30\304\220\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0_h\326\30\304\220\0\0workerid48011111\0\0\0\0008\0\0\0\t=\6\0003h\326\30\347\20\0\0api_cache_invalidation\0GSR2181232124\0\177\0\0$\0\0\0\t=\6\0003h\326\30\347\20\0\0workerid42803406\0\0ti\34\0\0\0\0\0\0\0003h\326\30\347\20\0\0\0\0IENT_FUNDS", 8192) = 8192
39287 12:53:50.159109 close(7)          = 0
39287 12:53:50.159151 open("pg_notify/3AA1", O_RDWR|O_CREAT, 0600) = 7
39287 12:53:50.159196 lseek(7, 221184, SEEK_SET) = 221184
39287 12:53:50.159234 write(7, "<\0\0\0\t=\6\0003h\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224<\0\0\0\t=\6\0Mh\326\30wg\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\0\0\0$\0\0\0\t=\6\0ah\326\30\260)\0\0orderid2181232124\0\0\0008\0\0\0\t=\6\0ah\326\30\260)\0\0api_cache_invalidation\0GSR2181232124\0=\6\0$\0\0\0\t=\6\0ah\326\30\260)\0\0workerid42803406\0\0ti8\0\0\0\t=\6\0nh\326\30\215\r\0\0api_cache_invalidation\0GSR3622500741\0\177\0\0$\0\0\0\t=\6\0nh\326\30\215\r\0\0workerid48101635\0\0ti8\0\0\0\t=\6\0qh\326\30+\3\0\0api_cache_invalidation\0GSR3429101202\0\177\0\0$\0\0\0\t=\6\0qh\326\30+\3\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0mh\326\30\260)\0\0orderid2181232124\0\0\0$\0\0\0\t=\6\0yh\326\30\304\220\0\0orderid4030375186\0\0\0008\0\0\0\t=\6\0yh\326\30\304\220\0\0api_cache_invalidation\0GSR4030375186\0\177\0\0$\0\0\0\t=\6\0yh\326\30\304\220\0\0workerid48102187\0\0ti(\0\0\0\t=\6\0zh\326\30<\232\0\0RPC Worker\000423645708\0\0\0\0$\0\0\0\t=\6\0\200h\326\30\304\220\0\0workerid48102187\0\0\0\0008\0\0\0\t=\6\0\215h\326\30\300\236\0\0api_cache_invalidation\0GSR3748780999\0\177\0\0$\0\0\0\t=\6\0\215h\326\30\300\236\0\0workerid48101751\0\0ti$\0\0\0\t=\6\0\220h\326\30\327d\0\0orderid3429101202\0\0\0008\0\0\0\t=\6\0\220h\326\30\327d\0\0api_cache_invalidation\0GSR3429101202\0\177\0\0$\0\0\0\t=\6\0\220h\326\30\327d\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\210h\326\0305\5\0\0orderid3186457059\0\0;$\0\0\0\t=\6\0\210h\326\0305\5\0\0workerid42803423\0\0\0;$\0\0\0\t=\6\0\223h\326\30\327d\0\0workerid48011104\0\0\0\0008\0\0\0\t=\6\0\222h\326\30R\3\0\0api_cache_invalidation\0GSR3685528914\0\177\0\0$\0\0\0\t=\6\0\222h\326\30R\3\0\0workerid39844907\0\0ti8\0\0\0\t=\6\0ch\326\30\347\20\0\0api_cache_invalidation\0GSR3883578049\0\177\0\0$\0\0\0\t=\6\0ch\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0ch\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\230h\326\30\304\220\0\0orderid2824179994\0\0\0008\0\0\0\t=\6\0\230h\326\30\304\220\0\0api_cache_invalidation\0GSR2824179994\0\177\0\0$\0\0\0\t=\6\0\230h\326\30\304\220\0\0workerid48102111\0\0ti$\0\0\0\t=\6\0\234h\326\30\304\220\0\0workerid48102111\0\0\0\0$\0\0\0\t=\6\0\235h\326\30?A\0\0orderid2563038584\0\0\0008\0\0\0\t=\6\0\235h\326\30?A\0\0api_cache_invalidation\0GSR2563038584\0\177\0\0$\0\0\0\t=\6\0\235h\326\30?A\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0\242h\326\30?A\0\0workerid48011410\0\0\0\0$\0\0\0\t=\6\0\232h\326\30\260)\0\0orderid3883578049\0\0\0008\0\0\0\t=\6\0\232h\326\30\260)\0\0api_cache_invalidation\0GSR3883578049\0=\6\0$\0\0\0\t=\6\0\232h\326\30\260)\0\0workerid42803406\0\0ti8\0\0\0\t=\6\0\247h\326\30\320\20\0\0api_cache_invalidation\0GSR3817171252\0\0\0\0$\0\0\0\t=\6\0\247h\326\30\320\20\0\0workerid42803403\0\0ti$\0\0\0\t=\6\0\252h\326\30\260)\0\0orderid3883578049\0\0\0$\0\0\0\t=\6\0\257h\326\30\254)\0\0orderid2986350075\0\0\0008\0\0\0\t=\6\0\257h\326\30\254)\0\0api_cache_invalidation\0GSR2986350075\0=\6\0$\0\0\0\t=\6\0\257h\326\30\254)\0\0workerid42803403\0\0ti$\0\0\0\t=\6\0\224h\326\30\361)\0\0orderid3186457059\0\0\0008\0\0\0\t=\6\0\224h\326\30\361)\0\0api_cache_invalidation\0GSR3186457059\0=\6\0$\0\0\0\t=\6\0\224h\326\30\361)\0\0workerid42803423\0\0ti8\0\0\0\t=\6\0\236h\326\30\347\20\0\0api_cache_invalidation\0GSR3392699031\0\177\0\0$\0\0\0\t=\6\0\236h\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\236h\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\2248\0\0\0\t=\6\0\274h\326\0309\223\0\0api_cache_invalidation\0GSR3434046459\0\177\0\0$\0\0\0\t=\6\0\274h\326\0309\223\0\0workerid40122977\0\0ti8\0\0\0\t=\6\0\275h\326\30Z\6\0\0api_cache_invalidation\0GSR3257519304\0\177\0\0$\0\0\0\t=\6\0\275h\326\30Z\6\0\0workerid39844881\0\0ti$\0\0\0\t=\6\0\271h\326\30\360\220\0\0orderid3983674014\0\0<$\0\0\0\t=\6\0\271h\326\30\360\220\0\0workerid42803420\0\0\0<$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102005\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101905\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102007\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48097441\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102010\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48096056\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102037\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102034\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101204\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48089722\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102192\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102181\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102326\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102327\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid42803400\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101411\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102213\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102268\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102266\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101137\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102156\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102267\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102021\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102092\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102093\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101607\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101412\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101537\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101716\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48101475\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102155\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102265\0\0\346:$\0\0\0\t=\6\0\266h\326\30\24S\0\0workerid48102328\0\0\346:$\0\0\0\t=\6\0\265h\326\30\254)\0\0orderid3817171252\0\0\0008\0\0\0\t=\6\0\265h\326\30\254)\0\0api_cache_invalidation\0GSR3817171252\0=\6\0$\0\0\0\t=\6\0\265h\326\30\254)\0\0workerid42803398\0\0ti$\0\0\0\t=\6\0\277h\326\30\260)\0\0orderid3392699031\0\0\0008\0\0\0\t=\6\0\277h\326\30\260)\0\0api_cache_invalidation\0GSR3392699031\0=\6\0$\0\0\0\t=\6\0\277h\326\30\260)\0\0workerid42803406\0\0ti$\0\0\0\t=\6\0\332h\326\30?A\0\0orderid3505409988\0\0\0008\0\0\0\t=\6\0\332h\326\30?A\0\0api_cache_invalidation\0GSR3505409988\0\177\0\0$\0\0\0\t=\6\0\332h\326\30?A\0\0workerid48101561\0\0ti$\0\0\0\t=\6\0\317h\326\30\260)\0\0orderid3392699031\0\0\0$\0\0\0\t=\6\0\340h\326\30?A\0\0workerid48101561\0\0\0\0008\0\0\0\t=\6\0\276h\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0(\0\0\0\t=\6\0\315h\326\30\247)\0\0RPC Worker\000423645709\0\0\0\0008\0\0\0\t=\6\0\315h\326\30\247)\0\0api_cache_invalidation\0GSR3817171252\0=\6\0$\0\0\0\t=\6\0\315h\326\30\247)\0\0orderid3817171252\0\0i$\0\0\0\t=\6\0\315h\326\30\247)\0\0workerid42803398\0\0\0i$\0\0\0\t=\6\0\300h\326\30/\234\0\0orderid3257519304\0\0\0008\0\0\0\t=\6\0\300h\326\30/\234\0\0api_cache_invalidation\0GSR3257519304\0=\6\0$\0\0\0\t=\6\0\300h\326\30/\234\0\0workerid48102267\0\0ti$\0\0\0\t=\6\0\301h\326\30c`\0\0orderid3434046459\0\0\0008\0\0\0\t=\6\0\301h\326\30c`\0\0api_cache_invalidation\0GSR3434046459\0=\6\0$\0\0\0\t=\6\0\301h\326\30c`\0\0workerid48011095\0\0ti$\0\0\0\t=\6\0\364h\326\30\327d\0\0orderid3375949113\0\0\0008\0\0\0\t=\6\0\364h\326\30\327d\0\0api_cache_invalidation\0GSR3375949113\0\177\0\0$\0\0\0\t=\6\0\364h\326\30\327d\0\0workerid48101842\0\0ti$\0\0\0\t=\6\0\302h\326\30\346)\0\0orderid3983674014\0\0\0008\0\0\0\t=\6\0\302h\326\30\346)\0\0api_cache_invalidation\0GSR3983674014\0=\6\0$\0\0\0\t=\6\0\302h\326\30\346)\0\0workerid42803420\0\0ti$\0\0\0\t=\6\0\365h\326\30?A\0\0orderid3039800402\0\0\0008\0\0\0\t=\6\0\365h\326\30?A\0\0api_cache_invalidation\0GSR3039800402\0\177\0\0$\0\0\0\t=\6\0\365h\326\30?A\0\0workerid48011188\0\0ti$\0\0\0\t=\6\0\345h\326\30n\232\0\0orderid1615799612\0\0\0$\0\0\0\t=\6\0\345h\326\30n\232\0\0workerid42803405\0\0\0\0$\0\0\0\t=\6\0\373h\326\30;5\0\0workerid48101842\0\0\0\0$\0\0\0\t=\6\0\375h\326\30\334'\0\0workerid48011188\0\0\0\0$\0\0\0\t=\6\0\1i\326\30\304\220\0\0orderid3683401193\0\0\0008\0\0\0\t=\6\0\1i\326\30\304\220\0\0api_cache_invalidation\0GSR3683401193\0\177\0\0$\0\0\0\t=\6\0\1i\326\30\304\220\0\0workerid48101606\0\0ti$\0\0\0\t=\6\0\5i\326\30\334'\0\0workerid48101606\0\0\0\0$\0\0\0\t=\6\0\vi\326\30uB\0\0orderid3429101202\0\0\0008\0\0\0\t=\6\0\vi\326\30uB\0\0api_cache_invalidation\0GSR3429101202\0\177\0\0$\0\0\0\t=\6\0\vi\326\30uB\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\16i\326\30?A\0\0workerid48011104\0\0\0\0$\0\0\0\t=\6\0\ri\326\30uB\0\0orderid3316971179\0\0\0008\0\0\0\t=\6\0\ri\326\30uB\0\0api_cache_invalidation\0GSR3316971179\0\177\0\0$\0\0\0\t=\6\0\ri\326\30uB\0\0workerid48102204\0\0ti$\0\0\0\t=\6\0\20i\326\30\334'\0\0workerid48102204\0\0\0\0008\0\0\0\t=\6\0\31i\326\30\"\233\0\0api_cache_invalidation\0GSR1982810203\0\177\0\0$\0\0\0\t=\6\0\31i\326\30\"\233\0\0workerid48101586\0\0ti$\0\0\0\t=\6\0\37i\326\30\334'\0\0orderid3375949113\0\0\0008\0\0\0\t=\6\0\37i\326\30\334'\0\0api_cache_invalidation\0GSR3375949113\0\177\0\0$\0\0\0\t=\6\0\37i\326\30\334'\0\0workerid48101842\0\0ti8\0\0\0\t=\6\0 i\326\30\272\4\0\0api_cache_invalidation\0GSR1262529471\0\177\0\0$\0\0\0\t=\6\0 i\326\30\272\4\0\0workerid42803401\0\0ti8\0\0\0\t=\6\0$i\326\30\360\224\0\0api_cache_invalidation\0GSR2429638001\0\177\0\0$\0\0\0\t=\6\0$i\326\30\360\224\0\0workerid48011122\0\0ti$\0\0\0\t=\6\0&i\326\30\334'\0\0orderid3683401193\0\0\0008\0\0\0\t=\6\0&i\326\30\334'\0\0api_cache_invalidation\0GSR3683401193\0\177\0\0$\0\0\0\t=\6\0&i\326\30\334'\0\0workerid48101606\0\0ti$\0\0\0\t=\6\0'i\326\30?A\0\0orderid3530358144\0\0\0008\0\0\0\t=\6\0'i\326\30?A\0\0api_cache_invalidation\0GSR3530358144\0\177\0\0$\0\0\0\t=\6\0'i\326\30?A\0\0workerid48011096\0\0ti$\0\0\0\t=\6\0*i\326\30?A\0\0workerid48101606\0\0\0\0008\0\0\0\t=\6\0\352h\326\30\347\20\0\0api_cache_invalidation\0GSR2169687121\0\177\0\0$\0\0\0\t=\6\0\352h\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0\352h\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0,i\326\30\334'\0\0workerid48011096\0\0\0\0$\0\0\0\t=\6\0+i\326\30uB\0\0orderid2819124331\0\0\0008\0\0\0\t=\6\0+i\326\30uB\0\0api_cache_invalidation\0GSR2819124331\0\177\0\0$\0\0\0\t=\6\0+i\326\30uB\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0/i\326\30?A\0\0workerid48011111\0\0\0\0$\0\0\0\t=\6\0:i\326\30?A\0\0orderid2429638001\0\0\0008\0\0\0\t=\6\0:i\326\30?A\0\0api_cache_invalidation\0GSR2429638001\0\177\0\0$\0\0\0\t=\6\0:i\326\30?A\0\0workerid48011122\0\0ti$\0\0\0\t=\6\0>i\326\30uB\0\0workerid48011122\0\0\0\0$\0\0\0\t=\6\0?i\326\30?A\0\0orderid1982810203\0\0\0008\0\0\0\t=\6\0?i\326\30?A\0\0api_cache_invalidation\0GSR1982810203\0\177\0\0$\0\0\0\t=\6\0?i\326\30?A\0\0workerid48101586\0\0ti$\0\0\0\t=\6\0@i\326\30?A\0\0workerid48101586\0\0\0\0$\0\0\0\t=\6\0Bi\326\30\334'\0\0orderid3505409988\0\0\0008\0\0\0\t=\6\0Bi\326\30\334'\0\0api_cache_invalidation\0GSR3505409988\0\177\0\0$\0\0\0\t=\6\0Bi\326\30\334'\0\0workerid48101561\0\0ti$\0\0\0\t=\6\0(i\326\30\253)\0\0orderid1262529471\0\0\0008\0\0\0\t=\6\0(i\326\30\253)\0\0api_cache_invalidation\0GSR1262529471\0=\6\0$\0\0\0\t=\6\0(i\326\30\253)\0\0workerid42803401\0\0ti$\0\0\0\t=\6\0Ji\326\30uB\0\0workerid48101561\0\0\0\0$\0\0\0\t=\6\0Ni\326\30uB\0\0orderid2819124331\0\0\0008\0\0\0\t=\6\0Ni\326\30uB\0\0api_cache_invalidation\0GSR2819124331\0\177\0\0$\0\0\0\t=\6\0Ni\326\30uB\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0Vi\326\30uB\0\0workerid48011111\0\0\0\0$\0\0\0\t=\6\0001i\326\30\320\20\0\0orderid3992931044\0\0;$\0\0\0\t=\6\0001i\326\30\320\20\0\0workerid42803426\0\0\0;$\0\0\0\t=\6\0Xi\326\30k\235\0\0orderid1419632139\0\0<$\0\0\0\t=\6\0Xi\326\30k\235\0\0workerid42803423\0\0\0<8\0\0\0\t=\6\0_i\326\30Z\6\0\0api_cache_invalidation\0GSR2944316298\0\177\0\0$\0\0\0\t=\6\0_i\326\30Z\6\0\0workerid48101932\0\0ti$\0\0\0\t=\6\0Mi\326\30\253)\0\0orderid1262529471\0\0\0008\0\0\0\t=\6\0Mi\326\30\253)\0\0api_cache_invalidation\0GSR1262529471\0=\6\0$\0\0\0\t=\6\0Mi\326\30\253)\0\0workerid48101670\0\0ti$\0\0\0\t=\6\0hi\326\30?A\0\0orderid3039800402\0\0\0008\0\0\0\t=\6\0hi\326\30?A\0\0api_cache_invalidation\0GSR3039800402\0\177\0\0$\0\0\0\t=\6\0hi\326\30?A\0\0workerid48011188\0\0ti8\0\0\0\t=\6\0ji\326\30R\222\0\0api_cache_invalidation\0GSR2830686796\0\177\0\0$\0\0\0\t=\6\0ji\326\30R\222\0\0workerid48102366\0\0ti8\0\0\0\t=\6\0.i\326\30\347\20\0\0api_cache_invalidation\0GSR1201226125\0\177\0\0$\0\0\0\t=\6\0.i\326\30\347\20\0\0workerid39844922\0\0ti0\0\0\0\0\0\0\0.i\326\30\347\20\0\0\0\0IENT_FUNDS_ESTONIA_SBMB\0bankwi", 8192) = 8192
39287 12:53:50.159387 close(7)          = 0
39287 12:53:50.159429 open("pg_notify/3AA1", O_RDWR|O_CREAT, 0600) = 7
39287 12:53:50.159479 lseek(7, 229376, SEEK_SET) = 229376
39287 12:53:50.160019 write(7, "<\0\0\0\t=\6\0.i\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0ki\326\30\334'\0\0orderid2534083970\0\0\0008\0\0\0\t=\6\0ki\326\30\334'\0\0api_cache_invalidation\0GSR2534083970\0\177\0\0$\0\0\0\t=\6\0ki\326\30\334'\0\0workerid48102295\0\0ti$\0\0\0\t=\6\0ni\326\30;5\0\0workerid48011188\0\0\0\0$\0\0\0\t=\6\0qi\326\30;5\0\0workerid48102295\0\0\0\0008\0\0\0\t=\6\0Wi\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0vi\326\30;5\0\0orderid4261414632\0\0\0008\0\0\0\t=\6\0vi\326\30;5\0\0api_cache_invalidation\0GSR4261414632\0\177\0\0$\0\0\0\t=\6\0vi\326\30;5\0\0workerid48011128\0\0ti$\0\0\0\t=\6\0yi\326\30\334'\0\0workerid48011128\0\0\0\0$\0\0\0\t=\6\0xi\326\30;5\0\0orderid2335984064\0\0\0008\0\0\0\t=\6\0xi\326\30;5\0\0api_cache_invalidation\0GSR2335984064\0\177\0\0$\0\0\0\t=\6\0xi\326\30;5\0\0workerid48011114\0\0ti$\0\0\0\t=\6\0}i\326\30;5\0\0workerid48011114\0\0\0\0$\0\0\0\t=\6\0Zi\326\30\377)\0\0orderid3992931044\0\0\0008\0\0\0\t=\6\0Zi\326\30\377)\0\0api_cache_invalidation\0GSR3992931044\0=\6\0$\0\0\0\t=\6\0Zi\326\30\377)\0\0workerid42803426\0\0ti$\0\0\0\t=\6\0\204i\326\30uB\0\0orderid2944316298\0\0\0008\0\0\0\t=\6\0\204i\326\30uB\0\0api_cache_invalidation\0GSR2944316298\0\177\0\0$\0\0\0\t=\6\0\204i\326\30uB\0\0workerid48101932\0\0ti$\0\0\0\t=\6\0\211i\326\30uB\0\0workerid48101932\0\0\0\0<\0\0\0\t=\6\0\214i\326\30jG\0\0CLIENT_FUNDS_FINLAND_OKOY\0bankwithdrawal\0\0\0\0008\0\0\0\t=\6\0\226i\326\30n\232\0\0api_cache_invalidation\0GSR3953521531\0\177\0\0$\0\0\0\t=\6\0\226i\326\30n\232\0\0workerid48011128\0\0ti$\0\0\0\t=\6\0\232i\326\30\334'\0\0orderid2099358214\0\0\0008\0\0\0\t=\6\0\232i\326\30\334'\0\0api_cache_invalidation\0GSR2099358214\0\177\0\0$\0\0\0\t=\6\0\232i\326\30\334'\0\0workerid48011224\0\0ti$\0\0\0\t=\6\0ci\326\30\361)\0\0orderid1419632139\0\0\0008\0\0\0\t=\6\0ci\326\30\361)\0\0api_cache_invalidation\0GSR1419632139\0=\6\0$\0\0\0\t=\6\0ci\326\30\361)\0\0workerid42803423\0\0ti$\0\0\0\t=\6\0\231i\326\30uB\0\0orderid2335984064\0\0\0008\0\0\0\t=\6\0\231i\326\30uB\0\0api_cache_invalidation\0GSR2335984064\0\177\0\0$\0\0\0\t=\6\0\231i\326\30uB\0\0workerid48011114\0\0ti$\0\0\0\t=\6\0\234i\326\30;5\0\0workerid48011224\0\0\0\0$\0\0\0\t=\6\0\235i\326\30\334'\0\0workerid48011114\0\0\0\0$\0\0\0\t=\6\0\236i\326\30;5\0\0orderid1606374576\0\0\0008\0\0\0\t=\6\0\236i\326\30;5\0\0api_cache_invalidation\0GSR1606374576\0\177\0\0$\0\0\0\t=\6\0\236i\326\30;5\0\0workerid48011122\0\0ti$\0\0\0\t=\6\0\240i\326\30\334'\0\0workerid48011122\0\0\0\0$\0\0\0\t=\6\0\244i\326\30;5\0\0orderid3953521531\0\0\0008\0\0\0\t=\6\0\244i\326\30;5\0\0api_cache_invalidation\0GSR3953521531\0\177\0\0$\0\0\0\t=\6\0\244i\326\30;5\0\0workerid48011128\0\0ti8\0\0\0\t=\6\0\243i\326\30\21\2\0\0api_cache_invalidation\0GSR1153754738\0\177\0\0$\0\0\0\t=\6\0\243i\326\30\21\2\0\0workerid48011188\0\0ti$\0\0\0\t=\6\0\246i\326\30;5\0\0workerid48011128\0\0\0\0008\0\0\0\t=\6\0ti\326\30\347\20\0\0api_cache_invalidation\0GSR1348405512\0\177\0\0$\0\0\0\t=\6\0ti\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0ti\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\250i\326\30;5\0\0orderid3434046459\0\0\0008\0\0\0\t=\6\0\250i\326\30;5\0\0api_cache_invalidation\0GSR3434046459\0\177\0\0$\0\0\0\t=\6\0\250i\326\30;5\0\0workerid48011095\0\0ti$\0\0\0\t=\6\0\253i\326\30\334'\0\0workerid48011095\0\0\0\0$\0\0\0\t=\6\0\252i\326\30;5\0\0orderid2429638001\0\0\0008\0\0\0\t=\6\0\252i\326\30;5\0\0api_cache_invalidation\0GSR2429638001\0\177\0\0$\0\0\0\t=\6\0\252i\326\30;5\0\0workerid48011122\0\0ti$\0\0\0\t=\6\0\260i\326\30\304\220\0\0workerid48011122\0\0\0\0008\0\0\0\t=\6\0oi\326\30\6\2\0\0CLIENT_FUNDS_SWEDEN_HAND\0bankwithdrawal\0008\0\0\0\t=\6\0oi\326\30\6\2\0\0api_cache_invalidation\0GSR3892633253\0al\0$\0\0\0\t=\6\0oi\326\30\6\2\0\0orderid3892633253\0\0i$\0\0\0\t=\6\0oi\326\30\6\2\0\0workerid39844922\0\0\0i$\0\0\0\t=\6\0\254i\326\30I\234\0\0orderid1348405512\0\0\0008\0\0\0\t=\6\0\254i\326\30I\234\0\0api_cache_invalidation\0GSR1348405512\0=\6\0$\0\0\0\t=\6\0\254i\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0\272i\326\30\304\220\0\0orderid3434046459\0\0\0008\0\0\0\t=\6\0\272i\326\30\304\220\0\0api_cache_invalidation\0GSR3434046459\0\177\0\0$\0\0\0\t=\6\0\272i\326\30\304\220\0\0workerid48011095\0\0ti$\0\0\0\t=\6\0\277i\326\30\304\220\0\0workerid48011095\0\0\0\0$\0\0\0\t=\6\0\300i\326\30;5\0\0orderid1153754738\0\0\0008\0\0\0\t=\6\0\300i\326\30;5\0\0api_cache_invalidation\0GSR1153754738\0\177\0\0$\0\0\0\t=\6\0\300i\326\30;5\0\0workerid48011188\0\0ti$\0\0\0\t=\6\0\301i\326\30;5\0\0workerid48011188\0\0\0\0$\0\0\0\t=\6\0\271i\326\30\6\2\0\0orderid1201226125\0\0\0008\0\0\0\t=\6\0\271i\326\30\6\2\0\0api_cache_invalidation\0GSR1201226125\0=\6\0$\0\0\0\t=\6\0\271i\326\30\6\2\0\0workerid39844922\0\0ti$\0\0\0\t=\6\0\276i\326\30I\234\0\0orderid1348405512\0\0\0$\0\0\0\t=\6\0\305i\326\30\6\2\0\0orderid1201226125\0\0\0$\0\0\0\t=\6\0\306i\326\30I\234\0\0orderid2169687121\0\0\0008\0\0\0\t=\6\0\306i\326\30I\234\0\0api_cache_invalidation\0GSR2169687121\0=\6\0$\0\0\0\t=\6\0\306i\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0\316i\326\30uB\0\0orderid2739104041\0\0\0008\0\0\0\t=\6\0\316i\326\30uB\0\0api_cache_invalidation\0GSR2739104041\0\177\0\0$\0\0\0\t=\6\0\316i\326\30uB\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0\321i\326\30uB\0\0workerid48011410\0\0\0\0$\0\0\0\t=\6\0\320i\326\30\334'\0\0orderid3530358144\0\0\0008\0\0\0\t=\6\0\320i\326\30\334'\0\0api_cache_invalidation\0GSR3530358144\0\177\0\0$\0\0\0\t=\6\0\320i\326\30\334'\0\0workerid48011096\0\0ti$\0\0\0\t=\6\0\313i\326\30I\234\0\0orderid2169687121\0\0\0$\0\0\0\t=\6\0\323i\326\30\334'\0\0workerid48011096\0\0\0\0008\0\0\0\t=\6\0\257i\326\30\347\20\0\0api_cache_invalidation\0GSR2335114143\0\177\0\0$\0\0\0\t=\6\0\257i\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\257i\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\2248\0\0\0\t=\6\0\340i\326\30R\217\0\0api_cache_invalidation\0GSR1797080752\0\177\0\0$\0\0\0\t=\6\0\340i\326\30R\217\0\0workerid48101982\0\0ti$\0\0\0\t=\6\0\347i\326\30\334'\0\0orderid3530358144\0\0\0008\0\0\0\t=\6\0\347i\326\30\334'\0\0api_cache_invalidation\0GSR3530358144\0\177\0\0$\0\0\0\t=\6\0\347i\326\30\334'\0\0workerid48011096\0\0ti$\0\0\0\t=\6\0\335i\326\30\260)\0\0orderid2335114143\0\0\0008\0\0\0\t=\6\0\335i\326\30\260)\0\0api_cache_invalidation\0GSR2335114143\0=\6\0$\0\0\0\t=\6\0\335i\326\30\260)\0\0workerid42803406\0\0ti8\0\0\0\t=\6\0\317i\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0\355i\326\30\334'\0\0workerid48011096\0\0\0\0$\0\0\0\t=\6\0\345i\326\0309\235\0\0orderid3952522971\0\0;$\0\0\0\t=\6\0\345i\326\0309\235\0\0workerid42803416\0\0\0;$\0\0\0\t=\6\0\360i\326\30\334'\0\0orderid2298879395\0\0\0008\0\0\0\t=\6\0\360i\326\30\334'\0\0api_cache_invalidation\0GSR2298879395\0\177\0\0$\0\0\0\t=\6\0\360i\326\30\334'\0\0workerid48101850\0\0ti$\0\0\0\t=\6\0\363i\326\30\304\220\0\0workerid48100522\0\0\0\0$\0\0\0\t=\6\0\356i\326\30\260)\0\0orderid2335114143\0\0\0$\0\0\0\t=\6\0\366i\326\30\334'\0\0workerid48101850\0\0\0\0$\0\0\0\t=\6\0\367i\326\30\321\234\0\0orderid4269927946\0\0<$\0\0\0\t=\6\0\367i\326\30\321\234\0\0workerid42803420\0\0\0<$\0\0\0\t=\6\0\376i\326\30uB\0\0orderid3953521531\0\0\0008\0\0\0\t=\6\0\376i\326\30uB\0\0api_cache_invalidation\0GSR3953521531\0\177\0\0$\0\0\0\t=\6\0\376i\326\30uB\0\0workerid48011128\0\0ti8\0\0\0\t=\6\0\337i\326\30\347\20\0\0api_cache_invalidation\0GSR3875739102\0\177\0\0$\0\0\0\t=\6\0\337i\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\337i\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224<\0\0\0\t=\6\0\373i\326\30\33f\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\0\0\0$\0\0\0\t=\6\0\5j\326\30uB\0\0orderid4261414632\0\0\0008\0\0\0\t=\6\0\5j\326\30uB\0\0api_cache_invalidation\0GSR4261414632\0\177\0\0$\0\0\0\t=\6\0\5j\326\30uB\0\0workerid48011128\0\0ti$\0\0\0\t=\6\0\nj\326\30uB\0\0workerid48011128\0\0\0\0$\0\0\0\t=\6\0\fj\326\30uB\0\0workerid48011128\0\0\0\0$\0\0\0\t=\6\0\rj\326\30uB\0\0orderid1153754738\0\0\0008\0\0\0\t=\6\0\rj\326\30uB\0\0api_cache_invalidation\0GSR1153754738\0\177\0\0$\0\0\0\t=\6\0\rj\326\30uB\0\0workerid48011188\0\0ti$\0\0\0\t=\6\0\16j\326\30\334'\0\0orderid3709554593\0\0\0008\0\0\0\t=\6\0\16j\326\30\334'\0\0api_cache_invalidation\0GSR3709554593\0\177\0\0$\0\0\0\t=\6\0\16j\326\30\334'\0\0workerid48098775\0\0ti$\0\0\0\t=\6\0\17j\326\30\304\220\0\0orderid3429101202\0\0\0008\0\0\0\t=\6\0\17j\326\30\304\220\0\0api_cache_invalidation\0GSR3429101202\0\177\0\0$\0\0\0\t=\6\0\17j\326\30\304\220\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\20j\326\30\304\220\0\0workerid48098775\0\0\0\0$\0\0\0\t=\6\0\21j\326\30\334'\0\0workerid48011188\0\0\0\0008\0\0\0\t=\6\0\0j\326\30;5\0\0api_cache_invalidation\0GSR2785837120\0\177\0\0$\0\0\0\t=\6\0\0j\326\30;5\0\0workerid48102100\0\0ti(\0\0\0\t=\6\0\0j\326\30;5\0\0RPC Worker\000423645710\0n\0G$\0\0\0\t=\6\0\26j\326\30uB\0\0workerid48011104\0\0\0\0$\0\0\0\t=\6\0\365i\326\30\326)\0\0orderid3952522971\0\0\0008\0\0\0\t=\6\0\365i\326\30\326)\0\0api_cache_invalidation\0GSR3952522971\0=\6\0$\0\0\0\t=\6\0\365i\326\30\326)\0\0workerid42803416\0\0ti$\0\0\0\t=\6\0\27j\326\30uB\0\0orderid1068615604\0\0\0008\0\0\0\t=\6\0\27j\326\30uB\0\0api_cache_invalidation\0GSR1068615604\0\177\0\0$\0\0\0\t=\6\0\27j\326\30uB\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\30j\326\30;5\0\0workerid48011104\0\0\0\0$\0\0\0\t=\6\0\33j\326\30;5\0\0orderid2785837120\0\0\0008\0\0\0\t=\6\0\33j\326\30;5\0\0api_cache_invalidation\0GSR2785837120\0\177\0\0$\0\0\0\t=\6\0\33j\326\30;5\0\0workerid48102100\0\0ti$\0\0\0\t=\6\0$j\326\30;5\0\0orderid3622500741\0\0\0008\0\0\0\t=\6\0$j\326\30;5\0\0api_cache_invalidation\0GSR3622500741\0\177\0\0$\0\0\0\t=\6\0$j\326\30;5\0\0workerid48101635\0\0ti$\0\0\0\t=\6\0&j\326\30;5\0\0workerid48101635\0\0\0\0$\0\0\0\t=\6\0%j\326\30\334'\0\0orderid4261414632\0\0\0008\0\0\0\t=\6\0%j\326\30\334'\0\0api_cache_invalidation\0GSR4261414632\0\177\0\0$\0\0\0\t=\6\0%j\326\30\334'\0\0workerid48011128\0\0ti$\0\0\0\t=\6\0)j\326\30;5\0\0workerid48011128\0\0\0\0$\0\0\0\t=\6\0,j\326\30;5\0\0orderid1262529471\0\0\0008\0\0\0\t=\6\0,j\326\30;5\0\0api_cache_invalidation\0GSR1262529471\0\177\0\0$\0\0\0\t=\6\0,j\326\30;5\0\0workerid48101670\0\0ti$\0\0\0\t=\6\0-j\326\30;5\0\0workerid48101670\0\0\0\0$\0\0\0\t=\6\0000j\326\30\334'\0\0orderid2099358214\0\0\0008\0\0\0\t=\6\0000j\326\30\334'\0\0api_cache_invalidation\0GSR2099358214\0\177\0\0$\0\0\0\t=\6\0000j\326\30\334'\0\0workerid48011224\0\0ti$\0\0\0\t=\6\0002j\326\30\334'\0\0workerid48011224\0\0\0\0008\0\0\0\t=\6\0\3j\326\30\347\20\0\0api_cache_invalidation\0GSR2654644181\0\177\0\0$\0\0\0\t=\6\0\3j\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\3j\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\2248\0\0\0\t=\6\0004j\326\30\273\223\0\0api_cache_invalidation\0GSR2625672040\0\177\0\0$\0\0\0\t=\6\0004j\326\30\273\223\0\0workerid48101182\0\0ti$\0\0\0\t=\6\0006j\326\30;5\0\0orderid3622500741\0\0\0008\0\0\0\t=\6\0006j\326\30;5\0\0api_cache_invalidation\0GSR3622500741\0\177\0\0$\0\0\0\t=\6\0006j\326\30;5\0\0workerid48101635\0\0ti$\0\0\0\t=\6\0008j\326\30\334'\0\0orderid1606374576\0\0\0008\0\0\0\t=\6\0008j\326\30\334'\0\0api_cache_invalidation\0GSR1606374576\0\177\0\0$\0\0\0\t=\6\0008j\326\30\334'\0\0workerid48011122\0\0ti$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102005\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101905\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102007\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48097441\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102010\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48096056\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102037\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102034\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101204\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48089722\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102192\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102181\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101868\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101960\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102157\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101848\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101180\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102270\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102158\0\0\346:\24\0\0\0\0\0\0\0+j\326\30\24S\0\0\0\0rk", 8192) = 8192
39287 12:53:50.160219 close(7)          = 0
39287 12:53:50.160269 open("pg_notify/3AA1", O_RDWR|O_CREAT, 0600) = 7
39287 12:53:50.160320 lseek(7, 237568, SEEK_SET) = 237568
39287 12:53:50.160365 write(7, "$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101605\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101897\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101913\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102024\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48101908\0\0\346:$\0\0\0\t=\6\0+j\326\30\24S\0\0workerid48102159\0\0\346:$\0\0\0\t=\6\0:j\326\30;5\0\0workerid48101635\0\0\0\0$\0\0\0\t=\6\0=j\326\30uB\0\0workerid48011122\0\0\0\0$\0\0\0\t=\6\0;j\326\30\304\220\0\0orderid3586160082\0\0\0008\0\0\0\t=\6\0;j\326\30\304\220\0\0api_cache_invalidation\0GSR3586160082\0\177\0\0$\0\0\0\t=\6\0;j\326\30\304\220\0\0workerid48102117\0\0ti$\0\0\0\t=\6\0Bj\326\30\334'\0\0workerid48102117\0\0\0\0$\0\0\0\t=\6\0Aj\326\30uB\0\0orderid2099358214\0\0\0008\0\0\0\t=\6\0Aj\326\30uB\0\0api_cache_invalidation\0GSR2099358214\0\177\0\0$\0\0\0\t=\6\0Aj\326\30uB\0\0workerid48011224\0\0ti$\0\0\0\t=\6\0Dj\326\30\334'\0\0workerid48011224\0\0\0\0008\0\0\0\t=\6\0.j\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0008\0\0\0\t=\6\0Gj\326\0305\232\0\0api_cache_invalidation\0GSR1089556715\0\177\0\0$\0\0\0\t=\6\0Gj\326\0305\232\0\0workerid42803398\0\0ti$\0\0\0\t=\6\0Sj\326\30\334'\0\0orderid1606374576\0\0\0008\0\0\0\t=\6\0Sj\326\30\334'\0\0api_cache_invalidation\0GSR1606374576\0\177\0\0$\0\0\0\t=\6\0Sj\326\30\334'\0\0workerid48011122\0\0ti$\0\0\0\t=\6\0Vj\326\30\304\220\0\0workerid48011122\0\0\0\0$\0\0\0\t=\6\0Mj\326\30\247)\0\0orderid1089556715\0\0\0008\0\0\0\t=\6\0Mj\326\30\247)\0\0api_cache_invalidation\0GSR1089556715\0=\6\0$\0\0\0\t=\6\0Mj\326\30\247)\0\0workerid42803398\0\0ti$\0\0\0\t=\6\0Zj\326\30\304\220\0\0orderid2739104041\0\0\0008\0\0\0\t=\6\0Zj\326\30\304\220\0\0api_cache_invalidation\0GSR2739104041\0\177\0\0$\0\0\0\t=\6\0Zj\326\30\304\220\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0\\j\326\30\334'\0\0workerid48011410\0\0\0\0008\0\0\0\t=\6\0aj\326\30f\217\0\0api_cache_invalidation\0GSR1177335885\0\177\0\0$\0\0\0\t=\6\0aj\326\30f\217\0\0workerid42803405\0\0ti8\0\0\0\t=\6\0Yj\326\30\247)\0\0api_cache_invalidation\0GSR1089556715\0=\6\0(\0\0\0\t=\6\0Yj\326\30\247)\0\0RPC Worker\000423645711\0n\0G$\0\0\0\t=\6\0Yj\326\30\247)\0\0orderid1089556715\0\0001$\0\0\0\t=\6\0Yj\326\30\247)\0\0workerid42803398\0\0\00018\0\0\0\t=\6\0dj\326\30&\235\0\0api_cache_invalidation\0GSR1010487738\0\177\0\0$\0\0\0\t=\6\0dj\326\30&\235\0\0workerid40122977\0\0ti$\0\0\0\t=\6\0ej\326\30\304\220\0\0workerid48101535\0\0\0\0$\0\0\0\t=\6\0hj\326\30\304\220\0\0orderid2739104041\0\0\0008\0\0\0\t=\6\0hj\326\30\304\220\0\0api_cache_invalidation\0GSR2739104041\0\177\0\0$\0\0\0\t=\6\0hj\326\30\304\220\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0mj\326\30uB\0\0workerid48011410\0\0\0\0$\0\0\0\t=\6\0lj\326\30\334'\0\0orderid4201123660\0\0\0008\0\0\0\t=\6\0lj\326\30\334'\0\0api_cache_invalidation\0GSR4201123660\0\177\0\0$\0\0\0\t=\6\0lj\326\30\334'\0\0workerid48101242\0\0ti$\0\0\0\t=\6\0nj\326\30\334'\0\0orderid3257519304\0\0\0008\0\0\0\t=\6\0nj\326\30\334'\0\0api_cache_invalidation\0GSR3257519304\0\177\0\0$\0\0\0\t=\6\0nj\326\30\334'\0\0workerid48102267\0\0ti$\0\0\0\t=\6\0oj\326\30uB\0\0workerid48101242\0\0\0\0$\0\0\0\t=\6\0qj\326\30\334'\0\0workerid48102267\0\0\0\0008\0\0\0\t=\6\0007j\326\30\347\20\0\0api_cache_invalidation\0GSR2071081489\0\177\0\0$\0\0\0\t=\6\0007j\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0007j\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0xj\326\30\334'\0\0orderid2833015779\0\0\0008\0\0\0\t=\6\0xj\326\30\334'\0\0api_cache_invalidation\0GSR2833015779\0\177\0\0$\0\0\0\t=\6\0xj\326\30\334'\0\0workerid48101858\0\0ti$\0\0\0\t=\6\0bj\326\30\256)\0\0orderid1615799612\0\0\0008\0\0\0\t=\6\0bj\326\30\256)\0\0api_cache_invalidation\0GSR1615799612\0=\6\0$\0\0\0\t=\6\0bj\326\30\256)\0\0workerid42803405\0\0ti$\0\0\0\t=\6\0}j\326\30\304\220\0\0workerid48101858\0\0\0\0$\0\0\0\t=\6\0wj\326\30I\234\0\0orderid2071081489\0\0\0008\0\0\0\t=\6\0wj\326\30I\234\0\0api_cache_invalidation\0GSR2071081489\0=\6\0$\0\0\0\t=\6\0wj\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0\204j\326\30\304\220\0\0orderid4239608871\0\0\0008\0\0\0\t=\6\0\204j\326\30\304\220\0\0api_cache_invalidation\0GSR4239608871\0\177\0\0$\0\0\0\t=\6\0\204j\326\30\304\220\0\0workerid48102156\0\0ti$\0\0\0\t=\6\0\207j\326\30uB\0\0orderid3429101202\0\0\0008\0\0\0\t=\6\0\207j\326\30uB\0\0api_cache_invalidation\0GSR3429101202\0\177\0\0$\0\0\0\t=\6\0\207j\326\30uB\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\210j\326\30\304\220\0\0workerid48102156\0\0\0\0$\0\0\0\t=\6\0\211j\326\30uB\0\0orderid3951971611\0\0\0008\0\0\0\t=\6\0\211j\326\30uB\0\0api_cache_invalidation\0GSR3951971611\0\177\0\0$\0\0\0\t=\6\0\211j\326\30uB\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0\215j\326\30uB\0\0workerid48011104\0\0\0\0(\0\0\0\t=\6\0gj\326\30c`\0\0RPC Worker\000423645712\0\0\0\0$\0\0\0\t=\6\0gj\326\30c`\0\0workerid48101697\0\00012,\0\0\0\t=\6\0gj\326\30c`\0\0workerid48101697\0STOPPED\0\0\0\0(\0\0\0\t=\6\0gj\326\30c`\0\0RPC Worker\000423645713\0PED$\0\0\0\t=\6\0gj\326\30c`\0\0orderid2320557413\0\0003$\0\0\0\t=\6\0gj\326\30c`\0\0orderid1010487738\0\00038\0\0\0\t=\6\0gj\326\30c`\0\0api_cache_invalidation\0GSR1010487738\0=\6\0$\0\0\0\t=\6\0gj\326\30c`\0\0workerid48102197\0\0ti$\0\0\0\t=\6\0\217j\326\30\304\220\0\0workerid48011104\0\0\0\0$\0\0\0\t=\6\0\205j\326\30I\234\0\0orderid2071081489\0\0\0$\0\0\0\t=\6\0|j\326\30\256)\0\0orderid1177335885\0\0\0008\0\0\0\t=\6\0|j\326\30\256)\0\0api_cache_invalidation\0GSR1177335885\0=\6\0$\0\0\0\t=\6\0|j\326\30\256)\0\0workerid39844881\0\0ti$\0\0\0\t=\6\0\226j\326\30/\234\0\0orderid1177335885\0\0\0008\0\0\0\t=\6\0\226j\326\30/\234\0\0api_cache_invalidation\0GSR1177335885\0=\6\0$\0\0\0\t=\6\0\226j\326\30/\234\0\0workerid39844881\0\0ti8\0\0\0\t=\6\0\260j\326\30\360\220\0\0api_cache_invalidation\0GSR2688884640\0\177\0\0$\0\0\0\t=\6\0\260j\326\30\360\220\0\0workerid42803401\0\0ti$\0\0\0\t=\6\0\256j\326\30\273\223\0\0orderid1181643809\0\0:$\0\0\0\t=\6\0\256j\326\30\273\223\0\0workerid40122977\0\0\0:$\0\0\0\t=\6\0\271j\326\30\304\220\0\0orderid1258300118\0\0\0008\0\0\0\t=\6\0\271j\326\30\304\220\0\0api_cache_invalidation\0GSR1258300118\0\177\0\0$\0\0\0\t=\6\0\271j\326\30\304\220\0\0workerid48102269\0\0ti8\0\0\0\t=\6\0\272j\326\30k\235\0\0api_cache_invalidation\0GSR2396627220\0\177\0\0$\0\0\0\t=\6\0\272j\326\30k\235\0\0workerid48011224\0\0ti$\0\0\0\t=\6\0\274j\326\30\304\220\0\0workerid48102269\0\0\0\0008\0\0\0\t=\6\0yj\326\30\347\20\0\0api_cache_invalidation\0GSR1636943971\0\177\0\0$\0\0\0\t=\6\0yj\326\30\347\20\0\0workerid39844917\0\0ti<\0\0\0\t=\6\0yj\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\264j\326\30c`\0\0orderid1181643809\0\0\0008\0\0\0\t=\6\0\264j\326\30c`\0\0api_cache_invalidation\0GSR1181643809\0=\6\0$\0\0\0\t=\6\0\264j\326\30c`\0\0workerid40122977\0\0ti$\0\0\0\t=\6\0\262j\326\30\253)\0\0orderid2688884640\0\0\0008\0\0\0\t=\6\0\262j\326\30\253)\0\0api_cache_invalidation\0GSR2688884640\0=\6\0$\0\0\0\t=\6\0\262j\326\30\253)\0\0workerid48011145\0\0ti$\0\0\0\t=\6\0\277j\326\30\232\237\0\0orderid1636943971\0\0\0008\0\0\0\t=\6\0\277j\326\30\232\237\0\0api_cache_invalidation\0GSR1636943971\0=\6\0$\0\0\0\t=\6\0\277j\326\30\232\237\0\0workerid39844917\0\0ti,\0\0\0\t=\6\0\300j\326\30\304\220\0\0warning_no_idle_worker51\0\0x=$\0\0\0\t=\6\0\300j\326\30\304\220\0\0orderid2564376963\0\0k8\0\0\0\t=\6\0\300j\326\30\304\220\0\0api_cache_invalidation\0GSR2564376963\0\177\0\0008\0\0\0\t=\6\0\250j\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0\313j\326\30\232\237\0\0orderid1636943971\0\0\0008\0\0\0\t=\6\0\337j\326\30U\237\0\0CLIENT_FUNDS_POLAND_POLU\0bankwithdrawal\0$\0\0\0\t=\6\0\341j\326\30\220`\0\0workerid39844876\0\0\0\0008\0\0\0\t=\6\0\302j\326\30\347\20\0\0api_cache_invalidation\0GSR2842767732\0\177\0\0$\0\0\0\t=\6\0\302j\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0\302j\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\347j\326\30\334'\0\0orderid4030375186\0\0\0008\0\0\0\t=\6\0\347j\326\30\334'\0\0api_cache_invalidation\0GSR4030375186\0\177\0\0$\0\0\0\t=\6\0\347j\326\30\334'\0\0workerid48102187\0\0ti8\0\0\0\t=\6\0\351j\326\30@\225\0\0api_cache_invalidation\0GSR2819124331\0\177\0\0$\0\0\0\t=\6\0\351j\326\30@\225\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0\352j\326\30\334'\0\0workerid48102187\0\0\0\0$\0\0\0\t=\6\0\350j\326\30I\234\0\0orderid2842767732\0\0\0008\0\0\0\t=\6\0\350j\326\30I\234\0\0api_cache_invalidation\0GSR2842767732\0=\6\0$\0\0\0\t=\6\0\350j\326\30I\234\0\0workerid39844886\0\0ti8\0\0\0\t=\6\0\334j\326\30uB\0\0api_cache_invalidation\0GSR1068615604\0\177\0\0$\0\0\0\t=\6\0\334j\326\30uB\0\0workerid48011104\0\0ti(\0\0\0\t=\6\0\334j\326\30uB\0\0RPC Worker\000423645715\0n\0G(\0\0\0\t=\6\0\346j\326\30'\234\0\0RPC Worker\000423645716\0\0\0\0$\0\0\0\t=\6\0\346j\326\30'\234\0\0orderid2564376963\0\00068\0\0\0\t=\6\0\346j\326\30'\234\0\0api_cache_invalidation\0GSR2564376963\0=\6\0$\0\0\0\t=\6\0\346j\326\30'\234\0\0workerid39844876\0\0ti$\0\0\0\t=\6\0\373j\326\30uB\0\0orderid1068615604\0\0\0008\0\0\0\t=\6\0\373j\326\30uB\0\0api_cache_invalidation\0GSR1068615604\0\177\0\0$\0\0\0\t=\6\0\373j\326\30uB\0\0workerid48011104\0\0ti8\0\0\0\t=\6\0\343j\326\30\304\220\0\0api_cache_invalidation\0GSR2402812488\0\0\0\0$\0\0\0\t=\6\0\343j\326\30\304\220\0\0workerid48102271\0\0ti(\0\0\0\t=\6\0\343j\326\30\304\220\0\0RPC Worker\000423645714\0n\0G$\0\0\0\t=\6\0\366j\326\30I\234\0\0orderid2842767732\0\0\0000\0\0\0\t=\6\0\tk\326\30\245_\0\0RPC Worker Scheduler\000423645717\0\0$\0\0\0\t=\6\0\4k\326\30uB\0\0orderid4294397464\0\0008$\0\0\0\t=\6\0\fk\326\30uB\0\0orderid2402812488\0\0\0008\0\0\0\t=\6\0\fk\326\30uB\0\0api_cache_invalidation\0GSR2402812488\0\177\0\0$\0\0\0\t=\6\0\fk\326\30uB\0\0workerid48102271\0\0ti$\0\0\0\t=\6\0\20k\326\30uB\0\0orderid2598451143\0\0\0008\0\0\0\t=\6\0\20k\326\30uB\0\0api_cache_invalidation\0GSR2598451143\0\177\0\0$\0\0\0\t=\6\0\20k\326\30uB\0\0workerid48101821\0\0ti$\0\0\0\t=\6\0\23k\326\30uB\0\0workerid48101821\0\0\0\0$\0\0\0\t=\6\0\21k\326\30\304\220\0\0orderid2819124331\0\0\0008\0\0\0\t=\6\0\21k\326\30\304\220\0\0api_cache_invalidation\0GSR2819124331\0\177\0\0$\0\0\0\t=\6\0\21k\326\30\304\220\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0\26k\326\30uB\0\0workerid48011111\0\0\0\0$\0\0\0\t=\6\0\32k\326\30uB\0\0orderid3748780999\0\0\0008\0\0\0\t=\6\0\32k\326\30uB\0\0api_cache_invalidation\0GSR3748780999\0\177\0\0$\0\0\0\t=\6\0\32k\326\30uB\0\0workerid48101751\0\0ti$\0\0\0\t=\6\0 k\326\30\304\220\0\0workerid48101751\0\0\0\0$\0\0\0\t=\6\0\37k\326\30uB\0\0orderid3951971611\0\0\0008\0\0\0\t=\6\0\37k\326\30uB\0\0api_cache_invalidation\0GSR3951971611\0\177\0\0$\0\0\0\t=\6\0\37k\326\30uB\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0!k\326\30uB\0\0workerid48011104\0\0\0\0$\0\0\0\t=\6\0$k\326\30uB\0\0orderid2396627220\0\0\0008\0\0\0\t=\6\0$k\326\30uB\0\0api_cache_invalidation\0GSR2396627220\0\177\0\0$\0\0\0\t=\6\0$k\326\30uB\0\0workerid48011224\0\0ti$\0\0\0\t=\6\0(k\326\30\334'\0\0workerid48011224\0\0\0\0$\0\0\0\t=\6\0&k\326\30\304\220\0\0orderid3951971611\0\0\0008\0\0\0\t=\6\0&k\326\30\304\220\0\0api_cache_invalidation\0GSR3951971611\0\177\0\0$\0\0\0\t=\6\0&k\326\30\304\220\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0*k\326\30uB\0\0workerid48011104\0\0\0\0008\0\0\0\t=\6\0\353j\326\30\347\20\0\0api_cache_invalidation\0GSR3965533227\0\177\0\0$\0\0\0\t=\6\0\353j\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0\353j\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224,\0\0\0\t=\6\0004k\326\30uB\0\0workerid48101836\0STOPPED\0\0\0\0$\0\0\0\t=\6\0/k\326\30I\234\0\0orderid3965533227\0\0\0008\0\0\0\t=\6\0/k\326\30I\234\0\0api_cache_invalidation\0GSR3965533227\0=\6\0$\0\0\0\t=\6\0/k\326\30I\234\0\0workerid39844886\0\0ti8\0\0\0\t=\6\0008k\326\30\277\223\0\0api_cache_invalidation\0GSR2417487412\0\177\0\0$\0\0\0\t=\6\0008k\326\30\277\223\0\0workerid42803401\0\0ti$\0\0\0\t=\6\0<k\326\30uB\0\0orderid1010487738\0\0\0008\0\0\0\t=\6\0<k\326\30uB\0\0api_cache_invalidation\0GSR1010487738\0\177\0\0$\0\0\0\t=\6\0<k\326\30uB\0\0workerid48102197\0\0ti$\0\0\0\t=\6\0005k\326\30I\234\0\0orderid3965533227\0\0\0$\0\0\0\t=\6\0Ak\326\30uB\0\0workerid48102197\0\0\0\0008\0\0\0\t=\6\0\25k\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0008\0\0\0\t=\6\0?k\326\30\227\r\0\0api_cache_invalidation\0GSR1769677481\0\177\0\0$\0\0\0\t=\6\0?k\326\30\227\r\0\0workerid48101983\0\0ti,\0\0\0\0\0\0\0Hk\326\30@\217\0\0\0\0i_cache_invalidation\0GSR25", 8192) = 8192
39287 12:53:50.160512 close(7)          = 0
39287 12:53:50.160600 open("pg_notify/3AA1", O_RDWR|O_CREAT, 0600) = 7
39287 12:53:50.160658 lseek(7, 245760, SEEK_SET) = 245760
39287 12:53:50.160702 write(7, "8\0\0\0\t=\6\0Hk\326\30@\217\0\0api_cache_invalidation\0GSR2535599693\0\177\0\0$\0\0\0\t=\6\0Hk\326\30@\217\0\0workerid48011096\0\0ti0\0\0\0\t=\6\0Fk\326\0300]\0\0RPC Worker Scheduler\000423645718\0\0$\0\0\0\t=\6\0>k\326\30\253)\0\0orderid2417487412\0\0\0008\0\0\0\t=\6\0>k\326\30\253)\0\0api_cache_invalidation\0GSR2417487412\0=\6\0$\0\0\0\t=\6\0>k\326\30\253)\0\0workerid48102348\0\0ti$\0\0\0\t=\6\0Kk\326\30\21\2\0\0orderid2759261382\0\0;$\0\0\0\t=\6\0Kk\326\30\21\2\0\0workerid42803423\0\0\0;$\0\0\0\t=\6\0Xk\326\30\304\220\0\0orderid2535599693\0\0\0008\0\0\0\t=\6\0Xk\326\30\304\220\0\0api_cache_invalidation\0GSR2535599693\0\177\0\0$\0\0\0\t=\6\0Xk\326\30\304\220\0\0workerid48011096\0\0ti$\0\0\0\t=\6\0`k\326\30\304\220\0\0workerid48011096\0\0\0\0008\0\0\0\t=\6\0Yk\326\30D\5\0\0api_cache_invalidation\0GSR2785837120\0\177\0\0$\0\0\0\t=\6\0Yk\326\30D\5\0\0workerid48102100\0\0ti$\0\0\0\t=\6\0ik\326\30\304\220\0\0orderid2799255352\0\0\0008\0\0\0\t=\6\0ik\326\30\304\220\0\0api_cache_invalidation\0GSR2799255352\0\177\0\0$\0\0\0\t=\6\0ik\326\30\304\220\0\0workerid48011348\0\0ti$\0\0\0\t=\6\0lk\326\30\304\220\0\0workerid48011348\0\0\0\0$\0\0\0\t=\6\0jk\326\30uB\0\0orderid1889579687\0\0\0008\0\0\0\t=\6\0jk\326\30uB\0\0api_cache_invalidation\0GSR1889579687\0\177\0\0$\0\0\0\t=\6\0jk\326\30uB\0\0workerid48101935\0\0ti$\0\0\0\t=\6\0pk\326\30uB\0\0workerid48101935\0\0\0\0008\0\0\0\t=\6\0nk\326\30\353\4\0\0CLIENT_FUNDS_SWEDEN_HAND\0bankwithdrawal\0008\0\0\0\t=\6\0sk\326\30\7\235\0\0api_cache_invalidation\0GSR4268124171\0\177\0\0$\0\0\0\t=\6\0sk\326\30\7\235\0\0workerid39844876\0\0ti8\0\0\0\t=\6\0xk\326\30\315\4\0\0api_cache_invalidation\0GSR3318876617\0\177\0\0$\0\0\0\t=\6\0xk\326\30\315\4\0\0workerid48094158\0\0ti8\0\0\0\t=\6\0000k\326\30\347\20\0\0api_cache_invalidation\0GSR2731375633\0\177\0\0$\0\0\0\t=\6\0000k\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0000k\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\201k\326\30uB\0\0orderid1501597258\0\0\0008\0\0\0\t=\6\0\201k\326\30uB\0\0api_cache_invalidation\0GSR1501597258\0\177\0\0$\0\0\0\t=\6\0\201k\326\30uB\0\0workerid48099038\0\0ti$\0\0\0\t=\6\0\204k\326\30uB\0\0workerid48099038\0\0\0\0$\0\0\0\t=\6\0}k\326\30I\234\0\0orderid2731375633\0\0\0008\0\0\0\t=\6\0}k\326\30I\234\0\0api_cache_invalidation\0GSR2731375633\0=\6\0$\0\0\0\t=\6\0}k\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0ak\326\30\361)\0\0orderid2759261382\0\0\0008\0\0\0\t=\6\0ak\326\30\361)\0\0api_cache_invalidation\0GSR2759261382\0=\6\0$\0\0\0\t=\6\0ak\326\30\361)\0\0workerid42803423\0\0ti$\0\0\0\t=\6\0\227k\326\30\334'\0\0orderid2625672040\0\0\0008\0\0\0\t=\6\0\227k\326\30\334'\0\0api_cache_invalidation\0GSR2625672040\0\177\0\0$\0\0\0\t=\6\0\227k\326\30\334'\0\0workerid48101182\0\0ti$\0\0\0\t=\6\0\225k\326\30uB\0\0orderid2799255352\0\0\0008\0\0\0\t=\6\0\225k\326\30uB\0\0api_cache_invalidation\0GSR2799255352\0\177\0\0$\0\0\0\t=\6\0\225k\326\30uB\0\0workerid48011348\0\0ti$\0\0\0\t=\6\0\232k\326\30uB\0\0workerid48101182\0\0\0\0$\0\0\0\t=\6\0\234k\326\30\304\220\0\0workerid48011348\0\0\0\0$\0\0\0\t=\6\0\216k\326\30I\234\0\0orderid2731375633\0\0\0$\0\0\0\t=\6\0yk\326\30'\234\0\0orderid4268124171\0\0\0008\0\0\0\t=\6\0yk\326\30'\234\0\0api_cache_invalidation\0GSR4268124171\0=\6\0$\0\0\0\t=\6\0yk\326\30'\234\0\0workerid48102031\0\0ti$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102005\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101905\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102007\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48097441\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102010\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48096056\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102037\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102034\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101204\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48089722\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102192\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102181\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101868\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101897\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102329\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48090397\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid42803407\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid42803409\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101644\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48011788\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102214\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101476\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48011782\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102023\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102162\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101849\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48099981\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101478\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48099738\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102161\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48100045\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102160\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101358\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48101599\0\0\346:$\0\0\0\t=\6\0\210k\326\30\24S\0\0workerid48102330\0\0\346:$\0\0\0\t=\6\0\254k\326\30\304\220\0\0orderid2819124331\0\0\0008\0\0\0\t=\6\0\254k\326\30\304\220\0\0api_cache_invalidation\0GSR2819124331\0\177\0\0$\0\0\0\t=\6\0\254k\326\30\304\220\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0\261k\326\30;5\0\0workerid48011111\0\0\0\0008\0\0\0\t=\6\0\233k\326\30\334'\0\0api_cache_invalidation\0GSR3160677022\0\177\0\0$\0\0\0\t=\6\0\233k\326\30\334'\0\0workerid48101863\0\0ti(\0\0\0\t=\6\0\233k\326\30\334'\0\0RPC Worker\000423645719\0n\0G$\0\0\0\t=\6\0\302k\326\30uB\0\0orderid3160677022\0\0\0008\0\0\0\t=\6\0\302k\326\30uB\0\0api_cache_invalidation\0GSR3160677022\0\177\0\0$\0\0\0\t=\6\0\302k\326\30uB\0\0workerid48101863\0\0ti8\0\0\0\t=\6\0\177k\326\30\347\20\0\0api_cache_invalidation\0GSR2476390894\0\177\0\0$\0\0\0\t=\6\0\177k\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0\177k\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\2248\0\0\0\t=\6\0\246k\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0\306k\326\30I\234\0\0orderid2476390894\0\0\0008\0\0\0\t=\6\0\306k\326\30I\234\0\0api_cache_invalidation\0GSR2476390894\0=\6\0$\0\0\0\t=\6\0\306k\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0\322k\326\30I\234\0\0orderid2476390894\0\0\0008\0\0\0\t=\6\0\273k\326\30\304\220\0\0api_cache_invalidation\0GSR1707042247\0\177\0\0$\0\0\0\t=\6\0\273k\326\30\304\220\0\0workerid48100986\0\0ti(\0\0\0\t=\6\0\273k\326\30\304\220\0\0RPC Worker\000423645720\0n\0G$\0\0\0\t=\6\0\337k\326\30\304\220\0\0orderid1707042247\0\0\0008\0\0\0\t=\6\0\337k\326\30\304\220\0\0api_cache_invalidation\0GSR1707042247\0\177\0\0$\0\0\0\t=\6\0\337k\326\30\304\220\0\0workerid48100986\0\0ti$\0\0\0\t=\6\0\341k\326\30\304\220\0\0orderid2535599693\0\0\0008\0\0\0\t=\6\0\341k\326\30\304\220\0\0api_cache_invalidation\0GSR2535599693\0\177\0\0$\0\0\0\t=\6\0\341k\326\30\304\220\0\0workerid48011096\0\0ti$\0\0\0\t=\6\0\344k\326\30\304\220\0\0workerid48011096\0\0\0\0008\0\0\0\t=\6\0\342k\326\30\236P\0\0api_cache_invalidation\0GSR1724534419\0\0\0\0$\0\0\0\t=\6\0\342k\326\30\236P\0\0workerid39844894\0\0ti8\0\0\0\t=\6\0\346k\326\30N\217\0\0api_cache_invalidation\0GSR3318876617\0\0\0\0$\0\0\0\t=\6\0\346k\326\30N\217\0\0orderid3318876617\0\0i$\0\0\0\t=\6\0\346k\326\30N\217\0\0workerid48094158\0\0\0i(\0\0\0\t=\6\0\346k\326\30N\217\0\0RPC Worker\000423645721\0n\0G8\0\0\0\t=\6\0\313k\326\30\347\20\0\0api_cache_invalidation\0GSR3384311075\0\177\0\0$\0\0\0\t=\6\0\313k\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0\313k\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\353k\326\30e\3\0\0orderid3415747590\0\0;$\0\0\0\t=\6\0\353k\326\30e\3\0\0workerid42803405\0\0\0;$\0\0\0\t=\6\0\367k\326\30I\234\0\0orderid3384311075\0\0\0008\0\0\0\t=\6\0\367k\326\30I\234\0\0api_cache_invalidation\0GSR3384311075\0=\6\0$\0\0\0\t=\6\0\367k\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0\375k\326\30\304\220\0\0workerid48101842\0\0\0\0$\0\0\0\t=\6\0\376k\326\30I\234\0\0orderid3384311075\0\0\0$\0\0\0\t=\6\0\2l\326\30\304\220\0\0orderid2830686796\0\0\0008\0\0\0\t=\6\0\2l\326\30\304\220\0\0api_cache_invalidation\0GSR2830686796\0\177\0\0$\0\0\0\t=\6\0\2l\326\30\304\220\0\0workerid48102366\0\0ti$\0\0\0\t=\6\0\7l\326\30\304\220\0\0workerid48102366\0\0\0\0$\0\0\0\t=\6\0\373k\326\30\256)\0\0orderid3415747590\0\0\0008\0\0\0\t=\6\0\373k\326\30\256)\0\0api_cache_invalidation\0GSR3415747590\0=\6\0$\0\0\0\t=\6\0\373k\326\30\256)\0\0workerid42803405\0\0ti$\0\0\0\t=\6\0\rl\326\30;5\0\0orderid1797080752\0\0\0008\0\0\0\t=\6\0\rl\326\30;5\0\0api_cache_invalidation\0GSR1797080752\0\177\0\0$\0\0\0\t=\6\0\rl\326\30;5\0\0workerid48101982\0\0ti$\0\0\0\t=\6\0\17l\326\30\304\220\0\0workerid48101982\0\0\0\0008\0\0\0\t=\6\0\371k\326\30\347\20\0\0api_cache_invalidation\0GSR2270475736\0\177\0\0$\0\0\0\t=\6\0\371k\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\371k\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0\16l\326\30;5\0\0orderid3816003109\0\0\0008\0\0\0\t=\6\0\16l\326\30;5\0\0api_cache_invalidation\0GSR3816003109\0\177\0\0$\0\0\0\t=\6\0\16l\326\30;5\0\0workerid48011145\0\0ti8\0\0\0\t=\6\0\21l\326\30\26\222\0\0api_cache_invalidation\0GSR2523393846\0\177\0\0$\0\0\0\t=\6\0\21l\326\30\26\222\0\0workerid42803398\0\0ti$\0\0\0\t=\6\0\24l\326\30;5\0\0workerid48011145\0\0\0\0$\0\0\0\t=\6\0\26l\326\30;5\0\0orderid2688884640\0\0\0008\0\0\0\t=\6\0\26l\326\30;5\0\0api_cache_invalidation\0GSR2688884640\0\177\0\0$\0\0\0\t=\6\0\26l\326\30;5\0\0workerid48011145\0\0ti$\0\0\0\t=\6\0\25l\326\30\260)\0\0orderid2270475736\0\0\0008\0\0\0\t=\6\0\25l\326\30\260)\0\0api_cache_invalidation\0GSR2270475736\0=\6\0$\0\0\0\t=\6\0\25l\326\30\260)\0\0workerid42803406\0\0ti$\0\0\0\t=\6\0#l\326\30;5\0\0workerid48011145\0\0\0\0$\0\0\0\t=\6\0%l\326\30\304\220\0\0orderid3258393514\0\0\0008\0\0\0\t=\6\0%l\326\30\304\220\0\0api_cache_invalidation\0GSR3258393514\0\177\0\0$\0\0\0\t=\6\0%l\326\30\304\220\0\0workerid48101529\0\0ti8\0\0\0\t=\6\0\23l\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0008\0\0\0\t=\6\0\37l\326\30,]\0\0api_cache_invalidation\0GSR1707042247\0\177\0\0$\0\0\0\t=\6\0\37l\326\30,]\0\0workerid48100986\0\0ti(\0\0\0\t=\6\0\37l\326\30,]\0\0RPC Worker\000423645722\0n\0G$\0\0\0\t=\6\0!l\326\30\260)\0\0orderid2270475736\0\0\0008\0\0\0\t=\6\0,l\326\30m\220\0\0api_cache_invalidation\0GSR3338462872\0\177\0\0$\0\0\0\t=\6\0,l\326\30m\220\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0000l\326\30;5\0\0workerid48101529\0\0\0\0$\0\0\0\t=\6\0003l\326\30;5\0\0orderid2688884640\0\0\0008\0\0\0\t=\6\0003l\326\30;5\0\0api_cache_invalidation\0GSR2688884640\0\177\0\0$\0\0\0\t=\6\0003l\326\30;5\0\0workerid48011145\0\0ti$\0\0\0\t=\6\0-l\326\30\260)\0\0orderid3875739102\0\0\0008\0\0\0\t=\6\0-l\326\30\260)\0\0api_cache_invalidation\0GSR3875739102\0=\6\0$\0\0\0\t=\6\0-l\326\30\260)\0\0workerid42803406\0\0ti$\0\0\0\t=\6\0.l\326\30\341\234\0\0orderid2081449127\0\0<$\0\0\0\t=\6\0.l\326\30\341\234\0\0workerid42803405\0\0\0<$\0\0\0\t=\6\0008l\326\30;5\0\0workerid48011145\0\0\0\0$\0\0\0\t=\6\0009l\326\30\260)\0\0orderid3875739102\0\0\0$\0\0\0\t=\6\0=l\326\30\334'\0\0orderid3316971179\0\0\0008\0\0\0\t=\6\0=l\326\30\334'\0\0api_cache_invalidation\0GSR3316971179\0\177\0\0$\0\0\0\t=\6\0=l\326\30\334'\0\0workerid48102204\0\0ti$\0\0\0\t=\6\0Al\326\30;5\0\0workerid48102204\0\0\0\0$\0\0\0\t=\6\0Fl\326\30;5\0\0orderid3338462872\0\0\0008\0\0\0\t=\6\0Fl\326\30;5\0\0api_cache_invalidation\0GSR3338462872\0\177\0\0$\0\0\0\t=\6\0Fl\326\30;5\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0Gl\326\30;5\0\0workerid48011111\0\0\0\0008\0\0\0\t=\6\0\"l\326\30\347\20\0\0api_cache_invalidation\0GSR1162505132\0\177\0\0$\0\0\0\t=\6\0\"l\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0\"l\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0?l\326\30\260)\0\0orderid2654644181\0\0\0008\0\0\0\t=\6\0?l\326\30\260)\0\0api_cache_invalidation\0GSR2654644181\0=\6\0$\0\0\0\t=\6\0?l\326\30\260)\0\0workerid42803406\0\0ti8\0\0\0\t=\6\0Il\326\0305\232\0\0api_cache_invalidation\0GSR3160677022\0\0\0\1$\0\0\0\t=\6\0Il\326\0305\232\0\0workerid48101863\0\0ti(\0\0\0\t=\6\0Il\326\0305\232\0\0RPC Worker\000423645723\0n\0G\30\0\0\0\0\0\0\0Ll\326\30\260)\0\0\0\0derid2", 8192) = 8192
39287 12:53:50.160863 close(7)          = 0
39287 12:53:50.160906 open("pg_notify/3AA1", O_RDWR|O_CREAT, 0600) = 7
39287 12:53:50.160949 lseek(7, 253952, SEEK_SET) = 253952
39287 12:53:50.160985 write(7, "$\0\0\0\t=\6\0Ll\326\30\260)\0\0orderid2654644181\0\0\0$\0\0\0\t=\6\0Nl\326\30I\234\0\0orderid1162505132\0\0\0008\0\0\0\t=\6\0Nl\326\30I\234\0\0api_cache_invalidation\0GSR1162505132\0=\6\0$\0\0\0\t=\6\0Nl\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0]l\326\30I\234\0\0orderid1162505132\0\0\0008\0\0\0\t=\6\0`l\326\0300]\0\0api_cache_invalidation\0GSR3318876617\0\0\0\0$\0\0\0\t=\6\0`l\326\0300]\0\0workerid48094158\0\0ti8\0\0\0\t=\6\0nl\326\30\227\r\0\0api_cache_invalidation\0GSR4261414632\0\177\0\0$\0\0\0\t=\6\0nl\326\30\227\r\0\0workerid48011128\0\0ti8\0\0\0\t=\6\0Ol\326\30\347\20\0\0api_cache_invalidation\0GSR4210915558\0\177\0\0$\0\0\0\t=\6\0Ol\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0Ol\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\2248\0\0\0\t=\6\0hl\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0ll\326\30\17\225\0\0orderid2346696624\0\0;$\0\0\0\t=\6\0ll\326\30\17\225\0\0workerid42803416\0\0\0;8\0\0\0\t=\6\0\177l\326\30N\3\0\0api_cache_invalidation\0GSR4030375186\0\177\0\0$\0\0\0\t=\6\0\177l\326\30N\3\0\0workerid48102187\0\0ti$\0\0\0\t=\6\0\210l\326\30\334'\0\0orderid3990290077\0\0\0008\0\0\0\t=\6\0\210l\326\30\334'\0\0api_cache_invalidation\0GSR3990290077\0\177\0\0$\0\0\0\t=\6\0\210l\326\30\334'\0\0workerid48011410\0\0ti$\0\0\0\t=\6\0\205l\326\30I\234\0\0orderid4210915558\0\0\0008\0\0\0\t=\6\0\205l\326\30I\234\0\0api_cache_invalidation\0GSR4210915558\0=\6\0$\0\0\0\t=\6\0\205l\326\30I\234\0\0workerid39844886\0\0ti$\0\0\0\t=\6\0\230l\326\30\334'\0\0workerid48011410\0\0\0\0008\0\0\0\t=\6\0\232l\326\30+\3\0\0api_cache_invalidation\0GSR3103146891\0\177\0\0$\0\0\0\t=\6\0\232l\326\30+\3\0\0workerid42803423\0\0ti0\0\0\0\t=\6\0\241l\326\0300]\0\0RPC Worker Scheduler\000423645724\0\0$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102005\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101905\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102007\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48097441\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102010\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48096056\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102037\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102034\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101204\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48089722\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102192\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102181\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101868\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101897\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48090397\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101599\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102331\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48011787\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101851\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102096\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101966\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101904\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101655\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102097\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102098\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48101724\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48102332\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48092738\0\0\346:$\0\0\0\t=\6\0\233l\326\30\24S\0\0workerid48092682\0\0\346:$\0\0\0\t=\6\0\252l\326\30;5\0\0orderid3338462872\0\0\0008\0\0\0\t=\6\0\252l\326\30;5\0\0api_cache_invalidation\0GSR3338462872\0\177\0\0$\0\0\0\t=\6\0\252l\326\30;5\0\0workerid48011111\0\0ti8\0\0\0\t=\6\0\253l\326\30\240\215\0\0api_cache_invalidation\0GSR3375949113\0\177\0\0$\0\0\0\t=\6\0\253l\326\30\240\215\0\0workerid48101842\0\0ti8\0\0\0\t=\6\0\254l\326\30\202\236\0\0api_cache_invalidation\0GSR1615799612\0\177\0\0$\0\0\0\t=\6\0\254l\326\30\202\236\0\0workerid42803405\0\0ti$\0\0\0\t=\6\0\251l\326\30I\234\0\0orderid4210915558\0\0\0008\0\0\0\t=\6\0\261l\326\30P\235\0\0api_cache_invalidation\0GSR3869658511\0\177\0\0$\0\0\0\t=\6\0\261l\326\30P\235\0\0workerid42803420\0\0ti8\0\0\0\t=\6\0\206l\326\30\347\20\0\0api_cache_invalidation\0GSR1383332683\0\177\0\0$\0\0\0\t=\6\0\206l\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\206l\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\2240\0\0\0\t=\6\0\262l\326\0305\232\0\0RPC Worker Scheduler\000423645725\0\0$\0\0\0\t=\6\0\266l\326\30\327d\0\0orderid1262529471\0\0\0008\0\0\0\t=\6\0\266l\326\30\327d\0\0api_cache_invalidation\0GSR1262529471\0\177\0\0$\0\0\0\t=\6\0\266l\326\30\327d\0\0workerid48101670\0\0ti$\0\0\0\t=\6\0\300l\326\30\304\220\0\0orderid2097988329\0\0\0008\0\0\0\t=\6\0\300l\326\30\304\220\0\0api_cache_invalidation\0GSR2097988329\0\177\0\0$\0\0\0\t=\6\0\300l\326\30\304\220\0\0workerid48102145\0\0ti8\0\0\0\t=\6\0\304l\326\30\303\220\0\0api_cache_invalidation\0GSR2136479381\0\177\0\0$\0\0\0\t=\6\0\304l\326\30\303\220\0\0workerid42803423\0\0ti$\0\0\0\t=\6\0\320l\326\30\334'\0\0workerid48102145\0\0\0\0$\0\0\0\t=\6\0\322l\326\30\362\n\0\0workerid48101670\0\0\0\0$\0\0\0\t=\6\0\313l\326\30?A\0\0orderid2819124331\0\0\0008\0\0\0\t=\6\0\313l\326\30?A\0\0api_cache_invalidation\0GSR2819124331\0\177\0\0$\0\0\0\t=\6\0\313l\326\30?A\0\0workerid48011111\0\0ti8\0\0\0\t=\6\0\315l\326\30\333\220\0\0api_cache_invalidation\0GSR3566178926\0\177\0\0$\0\0\0\t=\6\0\315l\326\30\333\220\0\0workerid48100500\0\0ti$\0\0\0\t=\6\0\332l\326\30\327d\0\0workerid48011111\0\0\0\0$\0\0\0\t=\6\0\335l\326\30\334'\0\0workerid48011111\0\0\0\0008\0\0\0\t=\6\0\247l\326\30uB\0\0api_cache_invalidation\0GSR3816003109\0\177\0\0$\0\0\0\t=\6\0\247l\326\30uB\0\0workerid48011145\0\0ti(\0\0\0\t=\6\0\247l\326\30uB\0\0RPC Worker\000423645726\0n\0G$\0\0\0\t=\6\0\345l\326\30\327d\0\0orderid3816003109\0\0\0008\0\0\0\t=\6\0\345l\326\30\327d\0\0api_cache_invalidation\0GSR3816003109\0\177\0\0$\0\0\0\t=\6\0\345l\326\30\327d\0\0workerid48011145\0\0ti$\0\0\0\t=\6\0\306l\326\30\361)\0\0orderid3103146891\0\0\0008\0\0\0\t=\6\0\306l\326\30\361)\0\0api_cache_invalidation\0GSR3103146891\0=\6\0$\0\0\0\t=\6\0\306l\326\30\361)\0\0workerid48011133\0\0ti$\0\0\0\t=\6\0\371l\326\30uB\0\0orderid2097988329\0\0\0008\0\0\0\t=\6\0\371l\326\30uB\0\0api_cache_invalidation\0GSR2097988329\0\177\0\0$\0\0\0\t=\6\0\371l\326\30uB\0\0workerid48102145\0\0ti$\0\0\0\t=\6\0\375l\326\30?A\0\0workerid48102145\0\0\0\0$\0\0\0\t=\6\0\370l\326\30N\217\0\0orderid3318876617\0\0\0008\0\0\0\t=\6\0\370l\326\30N\217\0\0api_cache_invalidation\0GSR3318876617\0=\6\0$\0\0\0\t=\6\0\370l\326\30N\217\0\0workerid48094158\0\0ti$\0\0\0\t=\6\0\366l\326\30\361)\0\0orderid2136479381\0\0\0008\0\0\0\t=\6\0\366l\326\30\361)\0\0api_cache_invalidation\0GSR2136479381\0=\6\0$\0\0\0\t=\6\0\366l\326\30\361)\0\0workerid48102239\0\0ti$\0\0\0\t=\6\0\nm\326\30\327d\0\0orderid1982810203\0\0\0008\0\0\0\t=\6\0\nm\326\30\327d\0\0api_cache_invalidation\0GSR1982810203\0\177\0\0$\0\0\0\t=\6\0\nm\326\30\327d\0\0workerid48101586\0\0ti$\0\0\0\t=\6\0\fm\326\30?A\0\0orderid4268124171\0\0\0008\0\0\0\t=\6\0\fm\326\30?A\0\0api_cache_invalidation\0GSR4268124171\0\177\0\0$\0\0\0\t=\6\0\fm\326\30?A\0\0workerid48102031\0\0ti8\0\0\0\t=\6\0\rm\326\30\227\206\0\0api_cache_invalidation\0GSR3951971611\0\177\0\0$\0\0\0\t=\6\0\rm\326\30\227\206\0\0workerid48011104\0\0ti8\0\0\0\t=\6\0\305l\326\30\347\20\0\0api_cache_invalidation\0GSR2610683032\0\177\0\0$\0\0\0\t=\6\0\305l\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\305l\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\2248\0\0\0\t=\6\0\16m\326\30\271\236\0\0api_cache_invalidation\0GSR2219289536\0\177\0\0$\0\0\0\t=\6\0\16m\326\30\271\236\0\0workerid48102307\0\0ti$\0\0\0\t=\6\0\22m\326\30?A\0\0workerid48101586\0\0\0\0$\0\0\0\t=\6\0\25m\326\30\327d\0\0workerid48102031\0\0\0\0$\0\0\0\t=\6\0\30m\326\30\327d\0\0orderid2097988329\0\0\0008\0\0\0\t=\6\0\30m\326\30\327d\0\0api_cache_invalidation\0GSR2097988329\0\177\0\0$\0\0\0\t=\6\0\30m\326\30\327d\0\0workerid48102145\0\0ti$\0\0\0\t=\6\0\34m\326\30?A\0\0workerid48102145\0\0\0\0$\0\0\0\t=\6\0\"m\326\30\334'\0\0orderid3951971611\0\0\0008\0\0\0\t=\6\0\"m\326\30\334'\0\0api_cache_invalidation\0GSR3951971611\0\177\0\0$\0\0\0\t=\6\0\"m\326\30\334'\0\0workerid48011104\0\0ti$\0\0\0\t=\6\0(m\326\30?A\0\0workerid48011104\0\0\0\0$\0\0\0\t=\6\0)m\326\30\327d\0\0orderid2417487412\0\0\0008\0\0\0\t=\6\0)m\326\30\327d\0\0api_cache_invalidation\0GSR2417487412\0\177\0\0$\0\0\0\t=\6\0)m\326\30\327d\0\0workerid48102348\0\0ti$\0\0\0\t=\6\0-m\326\30\327d\0\0workerid48102348\0\0\0\0008\0\0\0\t=\6\0\24m\326\30uB\0\0api_cache_invalidation\0GSR3137416136\0\177\0\0$\0\0\0\t=\6\0\24m\326\30uB\0\0workerid48101436\0\0ti(\0\0\0\t=\6\0\24m\326\30uB\0\0RPC Worker\000423645727\0n\0G(\0\0\0\t=\6\0\24m\326\30uB\0\0RPC Worker\000423645728\0n\0G8\0\0\0\t=\6\0001m\326\30\374\1\0\0api_cache_invalidation\0GSR3434046459\0\177\0\0$\0\0\0\t=\6\0001m\326\30\374\1\0\0workerid48011095\0\0ti8\0\0\0\t=\6\0!m\326\30\364u\0\0CLIENT_FUNDS_PAYPAL_PPLX\0bankwithdrawal\0$\0\0\0\t=\6\0006m\326\30uB\0\0orderid3137416136\0\0\0008\0\0\0\t=\6\0006m\326\30uB\0\0api_cache_invalidation\0GSR3137416136\0\177\0\0$\0\0\0\t=\6\0006m\326\30uB\0\0workerid48101436\0\0ti$\0\0\0\t=\6\0007m\326\30?A\0\0orderid3257519304\0\0\0008\0\0\0\t=\6\0007m\326\30?A\0\0api_cache_invalidation\0GSR3257519304\0\177\0\0$\0\0\0\t=\6\0007m\326\30?A\0\0workerid48102267\0\0ti$\0\0\0\t=\6\0:m\326\30?A\0\0workerid48102267\0\0\0\0$\0\0\0\t=\6\0>m\326\30?A\0\0orderid2829542767\0\0\0008\0\0\0\t=\6\0>m\326\30?A\0\0api_cache_invalidation\0GSR2829542767\0\177\0\0$\0\0\0\t=\6\0>m\326\30?A\0\0workerid48101762\0\0ti$\0\0\0\t=\6\0Cm\326\30\334'\0\0workerid48101762\0\0\0\0008\0\0\0\t=\6\0\32m\326\30\347\20\0\0api_cache_invalidation\0GSR3661306146\0\177\0\0$\0\0\0\t=\6\0\32m\326\30\347\20\0\0workerid42803406\0\0ti<\0\0\0\t=\6\0\32m\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224$\0\0\0\t=\6\0Fm\326\30\334'\0\0orderid3103146891\0\0\0008\0\0\0\t=\6\0Fm\326\30\334'\0\0api_cache_invalidation\0GSR3103146891\0\177\0\0$\0\0\0\t=\6\0Fm\326\30\334'\0\0workerid48011133\0\0ti$\0\0\0\t=\6\0Jm\326\30\334'\0\0workerid48011133\0\0\0\0$\0\0\0\t=\6\0Hm\326\30\327d\0\0orderid3434046459\0\0\0008\0\0\0\t=\6\0Hm\326\30\327d\0\0api_cache_invalidation\0GSR3434046459\0\177\0\0$\0\0\0\t=\6\0Hm\326\30\327d\0\0workerid48011095\0\0ti$\0\0\0\t=\6\0Lm\326\30\327d\0\0workerid48011095\0\0\0\0$\0\0\0\t=\6\0Em\326\30\260)\0\0orderid3661306146\0\0\0008\0\0\0\t=\6\0Em\326\30\260)\0\0api_cache_invalidation\0GSR3661306146\0=\6\0$\0\0\0\t=\6\0Em\326\30\260)\0\0workerid42803406\0\0ti$\0\0\0\t=\6\0Pm\326\30\334'\0\0orderid2819124331\0\0\0008\0\0\0\t=\6\0Pm\326\30\334'\0\0api_cache_invalidation\0GSR2819124331\0\177\0\0$\0\0\0\t=\6\0Pm\326\30\334'\0\0workerid48011111\0\0ti$\0\0\0\t=\6\0Sm\326\30\327d\0\0workerid48011111\0\0\0\0008\0\0\0\t=\6\0Tm\326\30\5\237\0\0api_cache_invalidation\0GSR1177335885\0\177\0\0$\0\0\0\t=\6\0Tm\326\30\5\237\0\0workerid39844881\0\0ti$\0\0\0\t=\6\0Vm\326\30\327d\0\0orderid3103146891\0\0\0008\0\0\0\t=\6\0Vm\326\30\327d\0\0api_cache_invalidation\0GSR3103146891\0\177\0\0$\0\0\0\t=\6\0Vm\326\30\327d\0\0workerid48011133\0\0ti$\0\0\0\t=\6\0Zm\326\30\327d\0\0workerid48011133\0\0\0\0$\0\0\0\t=\6\0Qm\326\30\260)\0\0orderid3661306146\0\0\0(\0\0\0\t=\6\0Um\326\30\334'\0\0RPC Worker\000423645729\0\177\0\0$\0\0\0\t=\6\0Um\326\30\334'\0\0orderid1068615604\0\0009$\0\0\0\t=\6\0]m\326\30\260)\0\0orderid2610683032\0\0\0008\0\0\0\t=\6\0]m\326\30\260)\0\0api_cache_invalidation\0GSR2610683032\0=\6\0$\0\0\0\t=\6\0]m\326\30\260)\0\0workerid42803406\0\0ti$\0\0\0\t=\6\0Xm\326\30/\234\0\0orderid1177335885\0\0\0008\0\0\0\t=\6\0Xm\326\30/\234\0\0api_cache_invalidation\0GSR1177335885\0=\6\0$\0\0\0\t=\6\0Xm\326\30/\234\0\0workerid42803398\0\0ti8\0\0\0\t=\6\0^m\326\30e\3\0\0api_cache_invalidation\0GSR1222166337\0\177\0\0$\0\0\0\t=\6\0^m\326\30e\3\0\0workerid39844881\0\0ti$\0\0\0\t=\6\0em\326\30\260)\0\0orderid2610683032\0\0\0$\0\0\0\t=\6\0jm\326\30\260)\0\0orderid1383332683\0\0\0008\0\0\0\t=\6\0jm\326\30\260)\0\0api_cache_invalidation\0GSR1383332683\0=\6\0$\0\0\0\t=\6\0jm\326\30\260)\0\0workerid42803406\0\0ti8\0\0\0\t=\6\0Im\326\30\347\20\0\0api_cache_invalidation\0GSR4276398877\0\177\0\0$\0\0\0\t=\6\0Im\326\30\347\20\0\0workerid39844886\0\0ti<\0\0\0\t=\6\0Im\326\30\347\20\0\0CLIENT_FUNDS_ESTONIA_SBMB\0bankwithdrawal\0\375&\224(\0\0\0\t=\6\0hm\326\30\247)\0\0RPC Worker\000423645730\0\0\0\0\30\0\0\0\0\0\0\0hm\326\30\247)\0\0\0\0i_cach", 8192) = 8192
39287 12:53:50.161139 close(7)          = 0
39287 12:53:50.161183 openat(AT_FDCWD, "pg_notify", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 7
39287 12:53:50.161253 getdents(7, /* 3 entries */, 32768) = 72
39287 12:53:50.161320 unlink("pg_notify/3AA1") = 0
39287 12:53:50.161529 getdents(7, /* 0 entries */, 32768) = 0
39287 12:53:50.161569 close(7)          = 0
39287 12:53:49.621551 sendto(10, "A\0\0\0'\0\0]0RPC Worker Scheduler\000423645724\0A\0\0\0\32\0\0S\24workerid48102010\0\0A\0\0\0\32\0\0S\24workerid48102037\0\0A\0\0\0\32\0\0S\24workerid48102034\0\0A\0\0\0\32\0\0S\24workerid48101204\0\0A\0\0\0\32\0\0S\24workerid48102192\0\0A\0\0\0\32\0\0S\24workerid48102181\0\0A\0\0\0\32\0\0S\24workerid48101868\0\0A\0\0\0\32\0\0S\24workerid48101897\0\0A\0\0\0\32\0\0S\24workerid48101599\0\0A\0\0\0\32\0\0S\24workerid48102331\0\0A\0\0\0\32\0\0S\24workerid48011787\0\0A\0\0\0\32\0\0S\24workerid48101851\0\0A\0\0\0\32\0\0S\24workerid48102096\0\0A\0\0\0\32\0\0S\24workerid48101966\0\0A\0\0\0\32\0\0S\24workerid48101904\0\0A\0\0\0\32\0\0S\24workerid48101655\0\0A\0\0\0\32\0\0S\24workerid48102097\0\0A\0\0\0\32\0\0S\24workerid48102098\0\0A\0\0\0\32\0\0S\24workerid48101724\0\0A\0\0\0\32\0\0S\24workerid48102332\0\0A\0\0\0\32\0\0S\24workerid48092738\0\0A\0\0\0\32\0\0S\24workerid48092682\0\0", 634, 0, NULL, 0) = 634
39287 12:53:49.621622 rt_sigreturn()    = 45
39287 12:53:49.621665 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.621864 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=21268, si_uid=105} ---
39287 12:53:49.621949 semop(11108376, {{5, 1, 0}}, 1) = 0
39287 12:53:49.622081 sendto(10, "A\0\0\0-\0\0005;api_cache_invalidation\0GSR3338462872\0A\0\0\0\32\0\0005;workerid48011111\0\0", 73, 0, NULL, 0) = 73
39287 12:53:49.622140 rt_sigreturn()    = 45
39287 12:53:49.622183 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.622638 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=13627, si_uid=105} ---
39287 12:53:49.622692 rt_sigreturn()    = 45
39287 12:53:49.622805 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.623654 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=36256, si_uid=105} ---
39287 12:53:49.623974 rt_sigreturn()    = 45
39287 12:53:49.624066 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.624591 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=40578, si_uid=105} ---
39287 12:53:49.624646 rt_sigreturn()    = 45
39287 12:53:49.624709 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.625327 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=40009, si_uid=105} ---
39287 12:53:49.625379 rt_sigreturn()    = 45
39287 12:53:49.625439 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.626173 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=40272, si_uid=105} ---
39287 12:53:49.626233 rt_sigreturn()    = 45
39287 12:53:49.626279 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.627303 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=4327, si_uid=105} ---
39287 12:53:49.627359 sendto(10, "A\0\0\0'\0\0\2325RPC Worker Scheduler\000423645725\0", 40, 0, NULL, 0) = 40
39287 12:53:49.627423 rt_sigreturn()    = 45
39287 12:53:49.627495 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.628184 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=39477, si_uid=105} ---
39287 12:53:49.628289 sendto(10, "A\0\0\0-\0\0d\327api_cache_invalidation\0GSR1262529471\0A\0\0\0\32\0\0d\327workerid48101670\0\0", 73, 0, NULL, 0) = 73
39287 12:53:49.628369 rt_sigreturn()    = 45
39287 12:53:49.628413 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.628934 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=25815, si_uid=105} ---
39287 12:53:49.628985 rt_sigreturn()    = 45
39287 12:53:49.629094 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.634562 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=37059, si_uid=105} ---
39287 12:53:49.634637 sendto(10, "A\0\0\0-\0\0\220\303api_cache_invalidation\0GSR2136479381\0A\0\0\0\32\0\0\220\303workerid42803423\0\0", 73, 0, NULL, 0) = 73


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Attachments:

  [text/plain] strace1.txt (87.1K, ../../CAL9smLAUy-QHS9j9QoK4kEA8ReKmL9szfAg2gUSpAsui=ge=2A@mail.gmail.com/3-strace1.txt)
  download

  [text/plain] strace2.txt (4.1K, ../../CAL9smLAUy-QHS9j9QoK4kEA8ReKmL9szfAg2gUSpAsui=ge=2A@mail.gmail.com/4-strace2.txt)
  download | inline:
39287 12:53:49.621551 sendto(10, "A\0\0\0'\0\0]0RPC Worker Scheduler\000423645724\0A\0\0\0\32\0\0S\24workerid48102010\0\0A\0\0\0\32\0\0S\24workerid48102037\0\0A\0\0\0\32\0\0S\24workerid48102034\0\0A\0\0\0\32\0\0S\24workerid48101204\0\0A\0\0\0\32\0\0S\24workerid48102192\0\0A\0\0\0\32\0\0S\24workerid48102181\0\0A\0\0\0\32\0\0S\24workerid48101868\0\0A\0\0\0\32\0\0S\24workerid48101897\0\0A\0\0\0\32\0\0S\24workerid48101599\0\0A\0\0\0\32\0\0S\24workerid48102331\0\0A\0\0\0\32\0\0S\24workerid48011787\0\0A\0\0\0\32\0\0S\24workerid48101851\0\0A\0\0\0\32\0\0S\24workerid48102096\0\0A\0\0\0\32\0\0S\24workerid48101966\0\0A\0\0\0\32\0\0S\24workerid48101904\0\0A\0\0\0\32\0\0S\24workerid48101655\0\0A\0\0\0\32\0\0S\24workerid48102097\0\0A\0\0\0\32\0\0S\24workerid48102098\0\0A\0\0\0\32\0\0S\24workerid48101724\0\0A\0\0\0\32\0\0S\24workerid48102332\0\0A\0\0\0\32\0\0S\24workerid48092738\0\0A\0\0\0\32\0\0S\24workerid48092682\0\0", 634, 0, NULL, 0) = 634
39287 12:53:49.621622 rt_sigreturn()    = 45
39287 12:53:49.621665 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.621864 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=21268, si_uid=105} ---
39287 12:53:49.621949 semop(11108376, {{5, 1, 0}}, 1) = 0
39287 12:53:49.622081 sendto(10, "A\0\0\0-\0\0005;api_cache_invalidation\0GSR3338462872\0A\0\0\0\32\0\0005;workerid48011111\0\0", 73, 0, NULL, 0) = 73
39287 12:53:49.622140 rt_sigreturn()    = 45
39287 12:53:49.622183 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.622638 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=13627, si_uid=105} ---
39287 12:53:49.622692 rt_sigreturn()    = 45
39287 12:53:49.622805 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.623654 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=36256, si_uid=105} ---
39287 12:53:49.623974 rt_sigreturn()    = 45
39287 12:53:49.624066 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.624591 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=40578, si_uid=105} ---
39287 12:53:49.624646 rt_sigreturn()    = 45
39287 12:53:49.624709 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.625327 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=40009, si_uid=105} ---
39287 12:53:49.625379 rt_sigreturn()    = 45
39287 12:53:49.625439 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.626173 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=40272, si_uid=105} ---
39287 12:53:49.626233 rt_sigreturn()    = 45
39287 12:53:49.626279 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.627303 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=4327, si_uid=105} ---
39287 12:53:49.627359 sendto(10, "A\0\0\0'\0\0\2325RPC Worker Scheduler\000423645725\0", 40, 0, NULL, 0) = 40
39287 12:53:49.627423 rt_sigreturn()    = 45
39287 12:53:49.627495 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.628184 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=39477, si_uid=105} ---
39287 12:53:49.628289 sendto(10, "A\0\0\0-\0\0d\327api_cache_invalidation\0GSR1262529471\0A\0\0\0\32\0\0d\327workerid48101670\0\0", 73, 0, NULL, 0) = 73
39287 12:53:49.628369 rt_sigreturn()    = 45
39287 12:53:49.628413 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.628934 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=25815, si_uid=105} ---
39287 12:53:49.628985 rt_sigreturn()    = 45
39287 12:53:49.629094 recvfrom(10, 0x7fad3970d760, 8192, 0, 0, 0) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
39287 12:53:49.634562 --- SIGUSR1 {si_signo=SIGUSR1, si_code=SI_USER, si_pid=37059, si_uid=105} ---
39287 12:53:49.634637 sendto(10, "A\0\0\0-\0\0\220\303api_cache_invalidation\0GSR2136479381\0A\0\0\0\32\0\0\220\303workerid42803423\0\0", 73, 0, NULL, 0) = 73

^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-03 03:00  Michael Paquier <michael.paquier@gmail.com>
  parent: marko@joh.to
  1 sibling, 1 reply; 15+ messages in thread

From: Michael Paquier @ 2017-10-03 03:00 UTC (permalink / raw)
  To: Marko Tiikkaja <marko@joh.to>; +Cc: pgsql-bugs; Tom Lane <tgl@sss.pgh.pa.us>

On Wed, Sep 27, 2017 at 3:29 AM,  <marko@joh.to> wrote:
> So.. any ideas?  Unfortunately I can't reproduce this in an isolated
> environment, and in production this seems to be taking some time before it
> builds up into a proper issue.

Hm. Could it be some side-effect from 2bbe8a68? This has been
backpatched on all branches and it is part of 9.1.24.
-- 
Michael


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-09 14:52  Marko Tiikkaja <marko@joh.to>
  parent: Michael Paquier <michael.paquier@gmail.com>
  0 siblings, 2 replies; 15+ messages in thread

From: Marko Tiikkaja @ 2017-10-09 14:52 UTC (permalink / raw)
  To: Michael Paquier <michael.paquier@gmail.com>; +Cc: pgsql-bugs; Tom Lane <tgl@sss.pgh.pa.us>

On Tue, Oct 3, 2017 at 5:00 AM, Michael Paquier <michael.paquier@gmail.com>
wrote:

> On Wed, Sep 27, 2017 at 3:29 AM,  <marko@joh.to> wrote:
> > So.. any ideas?  Unfortunately I can't reproduce this in an isolated
> > environment, and in production this seems to be taking some time before
> it
> > builds up into a proper issue.
>
> Hm. Could it be some side-effect from 2bbe8a68? This has been
> backpatched on all branches and it is part of 9.1.24.


Unsure.  I can at least reproduce this with only one session ever listening
on anything.

I've produced a test case[1] for this which matches roughly what the code
does in production.  The code isn't the most pretty code out there, but
basically what it does is that it has one session LISTENing on a channel,
and 24 sessions sending messages with a prefix in order, so for  example:

   session 1 sends A_1, A_2, A_3, etc.
   session 2 sends B_1, B_2, B_3, ...

and the listener has a map recording what the last received number is for
each prefix, checking that all notifications are received and in the right
order.

After running it for a few days I start getting logged messages such as:

  out of order notification Q_97882353: 97882353 != 97882349 + 1 (prefix Q)
  out of order notification F_97947433: 97947433 != 97947429 + 1 (prefix F)
  out of order notification F_97947439: 97947439 != 97947436 + 1 (prefix F)

I did it on both 9.1.24 and 9.6.5 and they both exhibit the same behavior:
it takes days to get into this state, but then notifications are missed all
the time.  I currently have both systems in this state, so any idea what to
look at to try and debug this further?


.m

[1]: https://github.com/johto/notify-test

^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-09 15:15  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Marko Tiikkaja <marko@joh.to>
  1 sibling, 0 replies; 15+ messages in thread

From: Tom Lane @ 2017-10-09 15:15 UTC (permalink / raw)
  To: Marko Tiikkaja <marko@joh.to>; +Cc: Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

Marko Tiikkaja <marko@joh.to> writes:
> After running it for a few days I start getting logged messages such as:

>   out of order notification Q_97882353: 97882353 != 97882349 + 1 (prefix Q)
>   out of order notification F_97947433: 97947433 != 97947429 + 1 (prefix F)
>   out of order notification F_97947439: 97947439 != 97947436 + 1 (prefix F)

> I did it on both 9.1.24 and 9.6.5 and they both exhibit the same behavior:
> it takes days to get into this state, but then notifications are missed all
> the time.  I currently have both systems in this state, so any idea what to
> look at to try and debug this further?

You might try gdb'ing the recipient and stepping through
asyncQueueProcessPageEntries to see what happens.  Are the missing
entries present in the queue but it decides to ignore them for some
reason, or are they just not there?

An interesting black-box test might be to do this with two receiver
processes and see if they miss identical sets of messages.  That
would be a different way of triangulating on question number 1,
which is whether the sender or the recipient is at fault.

I wonder whether the long ramp-up time indicates that you have to
wrap around some counter somewhere before things go south.  Although
the only obvious candidate is wrapping the pg_notify SLRU queue,
and I'd think that would have happened many times already.

			regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-10 12:39  Marko Tiikkaja <marko@joh.to>
  parent: Marko Tiikkaja <marko@joh.to>
  1 sibling, 1 reply; 15+ messages in thread

From: Marko Tiikkaja @ 2017-10-10 12:39 UTC (permalink / raw)
  To: Michael Paquier <michael.paquier@gmail.com>; +Cc: pgsql-bugs; Tom Lane <tgl@sss.pgh.pa.us>

So I managed to accidentally kill and/or restart both servers while trying
to install debug symbols, but I'm doing a new run now and I noticed
something interesting: the listening backend's RecentXmin doesn't seem to
ever go forward.  By my reading of this code, that would mean trouble for
this piece of code in TransactionIdIsInProgress:

  if (TransactionIdPrecedes(xid, RecentXmin))
      return false;

Is this not an obvious bug?  async.c would think the transaction is not
running, and did not commit -> throw away the notification.  That would
also explain why it takes days for this to happen (since we need to run
through enough xids to wrap around).


.m

^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-10 13:58  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Marko Tiikkaja <marko@joh.to>
  0 siblings, 2 replies; 15+ messages in thread

From: Tom Lane @ 2017-10-10 13:58 UTC (permalink / raw)
  To: Marko Tiikkaja <marko@joh.to>; +Cc: Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

Marko Tiikkaja <marko@joh.to> writes:
> So I managed to accidentally kill and/or restart both servers while trying
> to install debug symbols, but I'm doing a new run now and I noticed
> something interesting: the listening backend's RecentXmin doesn't seem to
> ever go forward.  By my reading of this code, that would mean trouble for
> this piece of code in TransactionIdIsInProgress:

>   if (TransactionIdPrecedes(xid, RecentXmin))
>       return false;

Hmm ... I suppose it's possible that that happens if the listening
backend isn't executing any SQL commands but is just sitting.
While that might describe your test harness, does it describe any
real-world application?

			regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-10 14:06  Alvaro Herrera <alvherre@alvh.no-ip.org>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 1 reply; 15+ messages in thread

From: Alvaro Herrera @ 2017-10-10 14:06 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Marko Tiikkaja <marko@joh.to>; Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

Tom Lane wrote:
> Marko Tiikkaja <marko@joh.to> writes:
> > So I managed to accidentally kill and/or restart both servers while trying
> > to install debug symbols, but I'm doing a new run now and I noticed
> > something interesting: the listening backend's RecentXmin doesn't seem to
> > ever go forward.  By my reading of this code, that would mean trouble for
> > this piece of code in TransactionIdIsInProgress:
> 
> >   if (TransactionIdPrecedes(xid, RecentXmin))
> >       return false;
> 
> Hmm ... I suppose it's possible that that happens if the listening
> backend isn't executing any SQL commands but is just sitting.
> While that might describe your test harness, does it describe any
> real-world application?

I think it's not totally unreasonable to have processes sitting idle for
long periods of time.  One example: a pooler configured to have more
connections that are actually needed most of the time (I'm fairly sure
I've seen this).  Would they not recompute RecentXmin if they did a
sinval reset?  Also, a listener daemon for which notifications are very
infrequent.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-10 14:09  Marko Tiikkaja <marko@joh.to>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 0 replies; 15+ messages in thread

From: Marko Tiikkaja @ 2017-10-10 14:09 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

On Tue, Oct 10, 2017 at 3:58 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Marko Tiikkaja <marko@joh.to> writes:
> > So I managed to accidentally kill and/or restart both servers while
> trying
> > to install debug symbols, but I'm doing a new run now and I noticed
> > something interesting: the listening backend's RecentXmin doesn't seem to
> > ever go forward.  By my reading of this code, that would mean trouble for
> > this piece of code in TransactionIdIsInProgress:
>
> >   if (TransactionIdPrecedes(xid, RecentXmin))
> >       return false;
>
> Hmm ... I suppose it's possible that that happens if the listening
> backend isn't executing any SQL commands but is just sitting.
> While that might describe your test harness, does it describe any
> real-world application?
>

Yes.  It's a service which only does LISTEN and UNLISTEN and forwards
notifications to its clients.


.m

^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-10 14:19  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
  0 siblings, 1 reply; 15+ messages in thread

From: Tom Lane @ 2017-10-10 14:19 UTC (permalink / raw)
  To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Marko Tiikkaja <marko@joh.to>; Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> Tom Lane wrote:
>> Hmm ... I suppose it's possible that that happens if the listening
>> backend isn't executing any SQL commands but is just sitting.
>> While that might describe your test harness, does it describe any
>> real-world application?

> I think it's not totally unreasonable to have processes sitting idle for
> long periods of time.

Sure, but if they're truly idle, they aren't looking at RecentXmin ;-)

> Would they not recompute RecentXmin if they did a
> sinval reset?

Not sure, but if not, maybe making that code path update it would be
a suitable fix in practice?  I'm a bit hesitant to put a GetSnapshotData
call into ProcessIncomingNotify, because of the possibility of adding a
lot of contention on the ProcArray if there's very heavy notify traffic.

			regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-10 14:54  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 15+ messages in thread

From: Tom Lane @ 2017-10-10 14:54 UTC (permalink / raw)
  To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Marko Tiikkaja <marko@joh.to>; Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

I wrote:
> ... I'm a bit hesitant to put a GetSnapshotData
> call into ProcessIncomingNotify, because of the possibility of adding a
> lot of contention on the ProcArray if there's very heavy notify traffic.

Wait, I have an idea.  Let's fetch a snapshot at the top of
ProcessIncomingNotify, and then use the snapshot to decide whether
xids are running, instead of calling TransactionIdIsInProgress.

GetSnapshotData is probably a bit heavier than a single
TransactionIdIsInProgress call, but if we have to process multiple
messages then this should be a net savings in ProcArray traffic.

			regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-10 18:07  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 15+ messages in thread

From: Tom Lane @ 2017-10-10 18:07 UTC (permalink / raw)
  To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Marko Tiikkaja <marko@joh.to>; Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

I wrote:
> Wait, I have an idea.  Let's fetch a snapshot at the top of
> ProcessIncomingNotify, and then use the snapshot to decide whether
> xids are running, instead of calling TransactionIdIsInProgress.

Concretely, I suggest the attached patch.  I tried Marko's testbed
against this, and it seems indeed a bit faster than before ---
running it for 100000 notifies takes about 19.25 seconds, rather than
19.5 seconds with HEAD.  But the testbed is probably about the best
case because it has a bunch of threads sending notifies to one
receiver, so that the receiver is quite likely to have multiple
messages to read at a time.  OTOH, in a situation where the NOTIFY
traffic isn't so high, it probably hardly matters anyway.

			regards, tom lane

-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Attachments:

  [text/x-diff] fix-notify-xact-testing.patch (6.4K, ../../7833.1507658856@sss.pgh.pa.us/2-fix-notify-xact-testing.patch)
  download | inline diff:
diff --git a/src/backend/commands/async.c b/src/backend/commands/async.c
index bacc08e..bec5185 100644
*** a/src/backend/commands/async.c
--- b/src/backend/commands/async.c
***************
*** 137,143 ****
--- 137,145 ----
  #include "utils/builtins.h"
  #include "utils/memutils.h"
  #include "utils/ps_status.h"
+ #include "utils/snapmgr.h"
  #include "utils/timestamp.h"
+ #include "utils/tqual.h"
  
  
  /*
*************** static bool SignalBackends(void);
*** 387,393 ****
  static void asyncQueueReadAllNotifications(void);
  static bool asyncQueueProcessPageEntries(volatile QueuePosition *current,
  							 QueuePosition stop,
! 							 char *page_buffer);
  static void asyncQueueAdvanceTail(void);
  static void ProcessIncomingNotify(void);
  static bool AsyncExistsPendingNotify(const char *channel, const char *payload);
--- 389,396 ----
  static void asyncQueueReadAllNotifications(void);
  static bool asyncQueueProcessPageEntries(volatile QueuePosition *current,
  							 QueuePosition stop,
! 							 char *page_buffer,
! 							 Snapshot snapshot);
  static void asyncQueueAdvanceTail(void);
  static void ProcessIncomingNotify(void);
  static bool AsyncExistsPendingNotify(const char *channel, const char *payload);
*************** asyncQueueReadAllNotifications(void)
*** 1744,1749 ****
--- 1747,1753 ----
  	volatile QueuePosition pos;
  	QueuePosition oldpos;
  	QueuePosition head;
+ 	Snapshot	snapshot;
  	bool		advanceTail;
  
  	/* page_buffer must be adequately aligned, so use a union */
*************** asyncQueueReadAllNotifications(void)
*** 1767,1772 ****
--- 1771,1779 ----
  		return;
  	}
  
+ 	/* Get snapshot we'll use to decide which xacts are still in progress */
+ 	snapshot = RegisterSnapshot(GetLatestSnapshot());
+ 
  	/*----------
  	 * Note that we deliver everything that we see in the queue and that
  	 * matches our _current_ listening state.
*************** asyncQueueReadAllNotifications(void)
*** 1854,1860 ****
  			 * while sending the notifications to the frontend.
  			 */
  			reachedStop = asyncQueueProcessPageEntries(&pos, head,
! 													   page_buffer.buf);
  		} while (!reachedStop);
  	}
  	PG_CATCH();
--- 1861,1868 ----
  			 * while sending the notifications to the frontend.
  			 */
  			reachedStop = asyncQueueProcessPageEntries(&pos, head,
! 													   page_buffer.buf,
! 													   snapshot);
  		} while (!reachedStop);
  	}
  	PG_CATCH();
*************** asyncQueueReadAllNotifications(void)
*** 1882,1887 ****
--- 1890,1898 ----
  	/* If we were the laziest backend, try to advance the tail pointer */
  	if (advanceTail)
  		asyncQueueAdvanceTail();
+ 
+ 	/* Done with snapshot */
+ 	UnregisterSnapshot(snapshot);
  }
  
  /*
*************** asyncQueueReadAllNotifications(void)
*** 1903,1909 ****
  static bool
  asyncQueueProcessPageEntries(volatile QueuePosition *current,
  							 QueuePosition stop,
! 							 char *page_buffer)
  {
  	bool		reachedStop = false;
  	bool		reachedEndOfPage;
--- 1914,1921 ----
  static bool
  asyncQueueProcessPageEntries(volatile QueuePosition *current,
  							 QueuePosition stop,
! 							 char *page_buffer,
! 							 Snapshot snapshot)
  {
  	bool		reachedStop = false;
  	bool		reachedEndOfPage;
*************** asyncQueueProcessPageEntries(volatile Qu
*** 1928,1934 ****
  		/* Ignore messages destined for other databases */
  		if (qe->dboid == MyDatabaseId)
  		{
! 			if (TransactionIdIsInProgress(qe->xid))
  			{
  				/*
  				 * The source transaction is still in progress, so we can't
--- 1940,1947 ----
  		/* Ignore messages destined for other databases */
  		if (qe->dboid == MyDatabaseId)
  		{
! 			if (TransactionIdIsCurrentTransactionId(qe->xid) ||
! 				XidInMVCCSnapshot(qe->xid, snapshot))
  			{
  				/*
  				 * The source transaction is still in progress, so we can't
*************** asyncQueueProcessPageEntries(volatile Qu
*** 1939,1948 ****
  				 * this advance-then-back-up behavior when dealing with an
  				 * uncommitted message.)
  				 *
! 				 * Note that we must test TransactionIdIsInProgress before we
! 				 * test TransactionIdDidCommit, else we might return a message
! 				 * from a transaction that is not yet visible to snapshots;
! 				 * compare the comments at the head of tqual.c.
  				 */
  				*current = thisentry;
  				reachedStop = true;
--- 1952,1965 ----
  				 * this advance-then-back-up behavior when dealing with an
  				 * uncommitted message.)
  				 *
! 				 * Note that we must test XidInMVCCSnapshot before we test
! 				 * TransactionIdDidCommit, else we might return a message from
! 				 * a transaction that is not yet visible to snapshots; compare
! 				 * the comments at the head of tqual.c.  Furthermore, because
! 				 * our own xact won't be listed in the snapshot, we must test
! 				 * TransactionIdIsCurrentTransactionId as well.  (It is
! 				 * possible to get here inside a transaction that has sent
! 				 * notifications, see Exec_ListenPreCommit.)
  				 */
  				*current = thisentry;
  				reachedStop = true;
diff --git a/src/backend/utils/time/tqual.c b/src/backend/utils/time/tqual.c
index bbac408..28060c5 100644
*** a/src/backend/utils/time/tqual.c
--- b/src/backend/utils/time/tqual.c
***************
*** 81,88 ****
  SnapshotData SnapshotSelfData = {HeapTupleSatisfiesSelf};
  SnapshotData SnapshotAnyData = {HeapTupleSatisfiesAny};
  
- /* local functions */
- static bool XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot);
  
  /*
   * SetHintBits()
--- 81,86 ----
*************** HeapTupleIsSurelyDead(HeapTuple htup, Tr
*** 1482,1488 ****
   * TransactionIdIsCurrentTransactionId first, except for known-committed
   * XIDs which could not be ours anyway.
   */
! static bool
  XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot)
  {
  	uint32		i;
--- 1480,1486 ----
   * TransactionIdIsCurrentTransactionId first, except for known-committed
   * XIDs which could not be ours anyway.
   */
! bool
  XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot)
  {
  	uint32		i;
diff --git a/src/include/utils/tqual.h b/src/include/utils/tqual.h
index 9a3b56e..96eaf01 100644
*** a/src/include/utils/tqual.h
--- b/src/include/utils/tqual.h
*************** extern HTSV_Result HeapTupleSatisfiesVac
*** 78,83 ****
--- 78,84 ----
  						 TransactionId OldestXmin, Buffer buffer);
  extern bool HeapTupleIsSurelyDead(HeapTuple htup,
  					  TransactionId OldestXmin);
+ extern bool XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot);
  
  extern void HeapTupleSetHintBits(HeapTupleHeader tuple, Buffer buffer,
  					 uint16 infomask, TransactionId xid);

^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-10 21:25  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 15+ messages in thread

From: Tom Lane @ 2017-10-10 21:25 UTC (permalink / raw)
  To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Marko Tiikkaja <marko@joh.to>; Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

I wrote:
> Concretely, I suggest the attached patch.  I tried Marko's testbed
> against this, and it seems indeed a bit faster than before ---
> running it for 100000 notifies takes about 19.25 seconds, rather than
> 19.5 seconds with HEAD.  But the testbed is probably about the best
> case because it has a bunch of threads sending notifies to one
> receiver, so that the receiver is quite likely to have multiple
> messages to read at a time.  OTOH, in a situation where the NOTIFY
> traffic isn't so high, it probably hardly matters anyway.

It occurred to me that it's easy to measure the worst case, ie always
one message per notify interrupt:

$ cat bnch.txt
LISTEN foo\; NOTIFY foo;
$ pgbench -n -c 1 -T 10 -f bnch.txt

On this case it seems that the patch is circa 2% slower than HEAD,
though that's only barely above the noise level in my measurements.
I think that's probably acceptable.

			regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-11 17:00  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 15+ messages in thread

From: Tom Lane @ 2017-10-11 17:00 UTC (permalink / raw)
  To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Marko Tiikkaja <marko@joh.to>; Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

I wrote:
> It occurred to me that it's easy to measure the worst case, ie always
> one message per notify interrupt:
> 
> $ cat bnch.txt
> LISTEN foo\; NOTIFY foo;
> $ pgbench -n -c 1 -T 10 -f bnch.txt
> 
> On this case it seems that the patch is circa 2% slower than HEAD,
> though that's only barely above the noise level in my measurements.

On looking at the patch again, I realized that it's not really necessary
to check TransactionIdIsCurrentTransactionId in the per-message loop.
Although it is possible to run the message-scanning code in a transaction
that's executed NOTIFY (if it also executed the session's first LISTEN),
at that point we will not yet have pushed the transaction's messages
into the queue.  Therefore there is no case where the loop can see
messages queued by its own transaction.

Although TransactionIdIsCurrentTransactionId isn't really all that
expensive, removing it brings the above test case to a statistical
tie: median of 3 60-second runs is 26054 tps in HEAD or 26034 tps
with patch.  (The cross-run variation is a couple hundred tps.)

So I'm now satisfied with the attached form of the patch, and
will proceed to make back-ported versions.

			regards, tom lane

-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Attachments:

  [text/x-diff] fix-notify-xact-testing-2.patch (7.4K, ../../9369.1507741218@sss.pgh.pa.us/2-fix-notify-xact-testing-2.patch)
  download | inline diff:
diff --git a/src/backend/commands/async.c b/src/backend/commands/async.c
index bacc08e..a93c81b 100644
*** a/src/backend/commands/async.c
--- b/src/backend/commands/async.c
***************
*** 137,143 ****
--- 137,145 ----
  #include "utils/builtins.h"
  #include "utils/memutils.h"
  #include "utils/ps_status.h"
+ #include "utils/snapmgr.h"
  #include "utils/timestamp.h"
+ #include "utils/tqual.h"
  
  
  /*
*************** static bool SignalBackends(void);
*** 387,393 ****
  static void asyncQueueReadAllNotifications(void);
  static bool asyncQueueProcessPageEntries(volatile QueuePosition *current,
  							 QueuePosition stop,
! 							 char *page_buffer);
  static void asyncQueueAdvanceTail(void);
  static void ProcessIncomingNotify(void);
  static bool AsyncExistsPendingNotify(const char *channel, const char *payload);
--- 389,396 ----
  static void asyncQueueReadAllNotifications(void);
  static bool asyncQueueProcessPageEntries(volatile QueuePosition *current,
  							 QueuePosition stop,
! 							 char *page_buffer,
! 							 Snapshot snapshot);
  static void asyncQueueAdvanceTail(void);
  static void ProcessIncomingNotify(void);
  static bool AsyncExistsPendingNotify(const char *channel, const char *payload);
*************** PreCommit_Notify(void)
*** 798,804 ****
  		}
  	}
  
! 	/* Queue any pending notifies */
  	if (pendingNotifies)
  	{
  		ListCell   *nextNotify;
--- 801,807 ----
  		}
  	}
  
! 	/* Queue any pending notifies (must happen after the above) */
  	if (pendingNotifies)
  	{
  		ListCell   *nextNotify;
*************** Exec_ListenPreCommit(void)
*** 987,993 ****
  	 * have already committed before we started to LISTEN.
  	 *
  	 * Note that we are not yet listening on anything, so we won't deliver any
! 	 * notification to the frontend.
  	 *
  	 * This will also advance the global tail pointer if possible.
  	 */
--- 990,998 ----
  	 * have already committed before we started to LISTEN.
  	 *
  	 * Note that we are not yet listening on anything, so we won't deliver any
! 	 * notification to the frontend.  Also, although our transaction might
! 	 * have executed NOTIFY, those message(s) aren't queued yet so we can't
! 	 * see them in the queue.
  	 *
  	 * This will also advance the global tail pointer if possible.
  	 */
*************** asyncQueueReadAllNotifications(void)
*** 1744,1749 ****
--- 1749,1755 ----
  	volatile QueuePosition pos;
  	QueuePosition oldpos;
  	QueuePosition head;
+ 	Snapshot	snapshot;
  	bool		advanceTail;
  
  	/* page_buffer must be adequately aligned, so use a union */
*************** asyncQueueReadAllNotifications(void)
*** 1767,1772 ****
--- 1773,1781 ----
  		return;
  	}
  
+ 	/* Get snapshot we'll use to decide which xacts are still in progress */
+ 	snapshot = RegisterSnapshot(GetLatestSnapshot());
+ 
  	/*----------
  	 * Note that we deliver everything that we see in the queue and that
  	 * matches our _current_ listening state.
*************** asyncQueueReadAllNotifications(void)
*** 1854,1860 ****
  			 * while sending the notifications to the frontend.
  			 */
  			reachedStop = asyncQueueProcessPageEntries(&pos, head,
! 													   page_buffer.buf);
  		} while (!reachedStop);
  	}
  	PG_CATCH();
--- 1863,1870 ----
  			 * while sending the notifications to the frontend.
  			 */
  			reachedStop = asyncQueueProcessPageEntries(&pos, head,
! 													   page_buffer.buf,
! 													   snapshot);
  		} while (!reachedStop);
  	}
  	PG_CATCH();
*************** asyncQueueReadAllNotifications(void)
*** 1882,1887 ****
--- 1892,1900 ----
  	/* If we were the laziest backend, try to advance the tail pointer */
  	if (advanceTail)
  		asyncQueueAdvanceTail();
+ 
+ 	/* Done with snapshot */
+ 	UnregisterSnapshot(snapshot);
  }
  
  /*
*************** asyncQueueReadAllNotifications(void)
*** 1903,1909 ****
  static bool
  asyncQueueProcessPageEntries(volatile QueuePosition *current,
  							 QueuePosition stop,
! 							 char *page_buffer)
  {
  	bool		reachedStop = false;
  	bool		reachedEndOfPage;
--- 1916,1923 ----
  static bool
  asyncQueueProcessPageEntries(volatile QueuePosition *current,
  							 QueuePosition stop,
! 							 char *page_buffer,
! 							 Snapshot snapshot)
  {
  	bool		reachedStop = false;
  	bool		reachedEndOfPage;
*************** asyncQueueProcessPageEntries(volatile Qu
*** 1928,1934 ****
  		/* Ignore messages destined for other databases */
  		if (qe->dboid == MyDatabaseId)
  		{
! 			if (TransactionIdIsInProgress(qe->xid))
  			{
  				/*
  				 * The source transaction is still in progress, so we can't
--- 1942,1948 ----
  		/* Ignore messages destined for other databases */
  		if (qe->dboid == MyDatabaseId)
  		{
! 			if (XidInMVCCSnapshot(qe->xid, snapshot))
  			{
  				/*
  				 * The source transaction is still in progress, so we can't
*************** asyncQueueProcessPageEntries(volatile Qu
*** 1939,1948 ****
  				 * this advance-then-back-up behavior when dealing with an
  				 * uncommitted message.)
  				 *
! 				 * Note that we must test TransactionIdIsInProgress before we
! 				 * test TransactionIdDidCommit, else we might return a message
! 				 * from a transaction that is not yet visible to snapshots;
! 				 * compare the comments at the head of tqual.c.
  				 */
  				*current = thisentry;
  				reachedStop = true;
--- 1953,1967 ----
  				 * this advance-then-back-up behavior when dealing with an
  				 * uncommitted message.)
  				 *
! 				 * Note that we must test XidInMVCCSnapshot before we test
! 				 * TransactionIdDidCommit, else we might return a message from
! 				 * a transaction that is not yet visible to snapshots; compare
! 				 * the comments at the head of tqual.c.
! 				 *
! 				 * Also, while our own xact won't be listed in the snapshot,
! 				 * we need not check for TransactionIdIsCurrentTransactionId
! 				 * because our transaction cannot (yet) have queued any
! 				 * messages.
  				 */
  				*current = thisentry;
  				reachedStop = true;
diff --git a/src/backend/utils/time/tqual.c b/src/backend/utils/time/tqual.c
index bbac408..28060c5 100644
*** a/src/backend/utils/time/tqual.c
--- b/src/backend/utils/time/tqual.c
***************
*** 81,88 ****
  SnapshotData SnapshotSelfData = {HeapTupleSatisfiesSelf};
  SnapshotData SnapshotAnyData = {HeapTupleSatisfiesAny};
  
- /* local functions */
- static bool XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot);
  
  /*
   * SetHintBits()
--- 81,86 ----
*************** HeapTupleIsSurelyDead(HeapTuple htup, Tr
*** 1482,1488 ****
   * TransactionIdIsCurrentTransactionId first, except for known-committed
   * XIDs which could not be ours anyway.
   */
! static bool
  XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot)
  {
  	uint32		i;
--- 1480,1486 ----
   * TransactionIdIsCurrentTransactionId first, except for known-committed
   * XIDs which could not be ours anyway.
   */
! bool
  XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot)
  {
  	uint32		i;
diff --git a/src/include/utils/tqual.h b/src/include/utils/tqual.h
index 9a3b56e..96eaf01 100644
*** a/src/include/utils/tqual.h
--- b/src/include/utils/tqual.h
*************** extern HTSV_Result HeapTupleSatisfiesVac
*** 78,83 ****
--- 78,84 ----
  						 TransactionId OldestXmin, Buffer buffer);
  extern bool HeapTupleIsSurelyDead(HeapTuple htup,
  					  TransactionId OldestXmin);
+ extern bool XidInMVCCSnapshot(TransactionId xid, Snapshot snapshot);
  
  extern void HeapTupleSetHintBits(HeapTupleHeader tuple, Buffer buffer,
  					 uint16 infomask, TransactionId xid);

^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24
@ 2017-10-11 21:16  Marko Tiikkaja <marko@joh.to>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 15+ messages in thread

From: Marko Tiikkaja @ 2017-10-11 21:16 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Michael Paquier <michael.paquier@gmail.com>; pgsql-bugs

On Wed, Oct 11, 2017 at 7:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> So I'm now satisfied with the attached form of the patch, and
> will proceed to make back-ported versions.
>

Thank you very much!  I'll let the test case run against patched master for
a few days just to make sure we didn't miss anything obvious.


.m

^ permalink  raw  reply  [nested|flat] 15+ messages in thread


end of thread, other threads:[~2017-10-11 21:16 UTC | newest]

Thread overview: 15+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2017-09-26 18:29 BUG #14830: Missed NOTIFications, PostgreSQL 9.1.24 marko@joh.to
2017-09-26 18:32 ` Marko Tiikkaja <marko@joh.to>
2017-10-03 03:00 ` Michael Paquier <michael.paquier@gmail.com>
2017-10-09 14:52   ` Marko Tiikkaja <marko@joh.to>
2017-10-09 15:15     ` Tom Lane <tgl@sss.pgh.pa.us>
2017-10-10 12:39     ` Marko Tiikkaja <marko@joh.to>
2017-10-10 13:58       ` Tom Lane <tgl@sss.pgh.pa.us>
2017-10-10 14:06         ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2017-10-10 14:19           ` Tom Lane <tgl@sss.pgh.pa.us>
2017-10-10 14:54             ` Tom Lane <tgl@sss.pgh.pa.us>
2017-10-10 18:07               ` Tom Lane <tgl@sss.pgh.pa.us>
2017-10-10 21:25                 ` Tom Lane <tgl@sss.pgh.pa.us>
2017-10-11 17:00                   ` Tom Lane <tgl@sss.pgh.pa.us>
2017-10-11 21:16                     ` Marko Tiikkaja <marko@joh.to>
2017-10-10 14:09         ` Marko Tiikkaja <marko@joh.to>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox