pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedshm_mq inconsistent behavior of SHM_MQ_DETACHED
4+ messages / 2 participants
[nested] [flat]
* shm_mq inconsistent behavior of SHM_MQ_DETACHED
@ 2014-04-22 13:55 Petr Jelinek <petr@2ndquadrant.com>
2014-04-28 13:36 ` Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Petr Jelinek @ 2014-04-22 13:55 UTC (permalink / raw)
To: pgsql-hackers
Hi,
I was playing with shm_mq and found a little odd behavior with detaching
after sending messages.
Following sequence behaves as expected (receiver gets 2 messages):
P1 -> set_sender
P1 -> attach
P2 -> set_receiver
P2 -> attach
P1 -> send
P2 -> receive
P1 -> send
P1 -> detach
P2 -> receive
P2 -> detach
But if I do first receive after detach like in this sequence:
P1 -> set_sender
P1 -> attach
P2 -> set_receiver
P2 -> attach
P1 -> send
P1 -> send
P1 -> detach
P2 -> receive
I get SHM_MQ_DETACHED on the receiver even though there are messages in
the ring buffer.
The reason for this behavior is that mqh_counterparty_attached is only
set by shm_mq_receive. This does not seem to be consistent - I would
either expect to get SHM_MQ_DETACHED always when other party has
detached or always get all remaining messages that are in queue (and I
would strongly prefer the latter).
Maybe the shm_mq_get_bytes_written should be used to determine if there
is something left for us to read in the receiver if we hit the
!mqh_counterparty_attached code path with detached sender?
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED
2014-04-22 13:55 shm_mq inconsistent behavior of SHM_MQ_DETACHED Petr Jelinek <petr@2ndquadrant.com>
@ 2014-04-28 13:36 ` Robert Haas <robertmhaas@gmail.com>
2014-04-28 20:24 ` Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED Petr Jelinek <petr@2ndquadrant.com>
0 siblings, 1 reply; 4+ messages in thread
From: Robert Haas @ 2014-04-28 13:36 UTC (permalink / raw)
To: Petr Jelinek <petr@2ndquadrant.com>; +Cc: pgsql-hackers
On Tue, Apr 22, 2014 at 9:55 AM, Petr Jelinek <petr@2ndquadrant.com> wrote:
> I was playing with shm_mq and found a little odd behavior with detaching
> after sending messages.
>
> Following sequence behaves as expected (receiver gets 2 messages):
> P1 -> set_sender
> P1 -> attach
> P2 -> set_receiver
> P2 -> attach
> P1 -> send
> P2 -> receive
> P1 -> send
> P1 -> detach
> P2 -> receive
> P2 -> detach
>
> But if I do first receive after detach like in this sequence:
> P1 -> set_sender
> P1 -> attach
> P2 -> set_receiver
> P2 -> attach
> P1 -> send
> P1 -> send
> P1 -> detach
> P2 -> receive
>
> I get SHM_MQ_DETACHED on the receiver even though there are messages in the
> ring buffer.
That's a bug.
> The reason for this behavior is that mqh_counterparty_attached is only set
> by shm_mq_receive. This does not seem to be consistent - I would either
> expect to get SHM_MQ_DETACHED always when other party has detached or always
> get all remaining messages that are in queue (and I would strongly prefer
> the latter).
>
> Maybe the shm_mq_get_bytes_written should be used to determine if there is
> something left for us to read in the receiver if we hit the
> !mqh_counterparty_attached code path with detached sender?
That's probably not a good idea, because there could be just a partial
message left in the buffer, if the sender died midway through writing
it. I suspect that attacking the problem that way will lead to a
bunch of nasty edge cases.
I'm thinking that the problem is really revolves around
shm_mq_wait_internal(). It returns true if the queue is attached but
not detached, and false if either the detach has already happened, or
if we establish via the background worker handle that it will never
come. But in the case of receiving, we want to treat
attached-then-detached as a success case, not a failure case.
Can you see if the attached patch fixes it?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
maybe-fix-shm-mq-detach.patch (text/x-patch, 606B) download
diff --git a/src/backend/storage/ipc/shm_mq.c b/src/backend/storage/ipc/shm_mq.c
index c70f3bf..4f7dd9c 100644
--- a/src/backend/storage/ipc/shm_mq.c
+++ b/src/backend/storage/ipc/shm_mq.c
@@ -406,7 +406,8 @@ shm_mq_receive(shm_mq_handle *mqh, Size *nbytesp, void **datap, bool nowait)
if (shm_mq_get_sender(mq) == NULL)
return SHM_MQ_WOULD_BLOCK;
}
- else if (!shm_mq_wait_internal(mq, &mq->mq_sender, mqh->mqh_handle))
+ else if (!shm_mq_wait_internal(mq, &mq->mq_sender, mqh->mqh_handle)
+ && shm_mq_get_sender(mq) == NULL)
{
mq->mq_detached = true;
return SHM_MQ_DETACHED;
* Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED
2014-04-22 13:55 shm_mq inconsistent behavior of SHM_MQ_DETACHED Petr Jelinek <petr@2ndquadrant.com>
2014-04-28 13:36 ` Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED Robert Haas <robertmhaas@gmail.com>
@ 2014-04-28 20:24 ` Petr Jelinek <petr@2ndquadrant.com>
2014-04-30 21:58 ` Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 4+ messages in thread
From: Petr Jelinek @ 2014-04-28 20:24 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers
On 28/04/14 15:36, Robert Haas wrote:
> On Tue, Apr 22, 2014 at 9:55 AM, Petr Jelinek <petr@2ndquadrant.com> wrote:
>>
>> But if I do first receive after detach like in this sequence:
>> P1 -> set_sender
>> P1 -> attach
>> P2 -> set_receiver
>> P2 -> attach
>> P1 -> send
>> P1 -> send
>> P1 -> detach
>> P2 -> receive
>>
>> I get SHM_MQ_DETACHED on the receiver even though there are messages in the
>> ring buffer.
>
> That's a bug.
>
> I'm thinking that the problem is really revolves around
> shm_mq_wait_internal(). It returns true if the queue is attached but
> not detached, and false if either the detach has already happened, or
> if we establish via the background worker handle that it will never
> come. But in the case of receiving, we want to treat
> attached-then-detached as a success case, not a failure case.
>
> Can you see if the attached patch fixes it?
>
Yes, the patch fixes it for me.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED
2014-04-22 13:55 shm_mq inconsistent behavior of SHM_MQ_DETACHED Petr Jelinek <petr@2ndquadrant.com>
2014-04-28 13:36 ` Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED Robert Haas <robertmhaas@gmail.com>
2014-04-28 20:24 ` Re: shm_mq inconsistent behavior of SHM_MQ_DETACHED Petr Jelinek <petr@2ndquadrant.com>
@ 2014-04-30 21:58 ` Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 4+ messages in thread
From: Robert Haas @ 2014-04-30 21:58 UTC (permalink / raw)
To: Petr Jelinek <petr@2ndquadrant.com>; +Cc: pgsql-hackers
On Mon, Apr 28, 2014 at 4:24 PM, Petr Jelinek <petr@2ndquadrant.com> wrote:
> Yes, the patch fixes it for me.
OK. I committed it. Thanks for the report.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2014-04-30 21:58 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-04-22 13:55 shm_mq inconsistent behavior of SHM_MQ_DETACHED Petr Jelinek <petr@2ndquadrant.com>
2014-04-28 13:36 ` Robert Haas <robertmhaas@gmail.com>
2014-04-28 20:24 ` Petr Jelinek <petr@2ndquadrant.com>
2014-04-30 21:58 ` Robert Haas <robertmhaas@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox