public inbox for [email protected]help / color / mirror / Atom feed
libpq v17 PQsocketPoll timeout is not granular enough 4+ messages / 4 participants [nested] [flat]
* libpq v17 PQsocketPoll timeout is not granular enough @ 2024-06-10 14:36 Dominique Devienne <[email protected]> 2024-06-10 22:59 ` Re: libpq v17 PQsocketPoll timeout is not granular enough Thomas Munro <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Dominique Devienne @ 2024-06-10 14:36 UTC (permalink / raw) To: pgsql-general Hi. I've noticed [that libpq API in v17 beta1][1], and wanted to use it to replace an existing Boost.ASIO-based async polling of the connection's socket, waiting for notifications. The use case being using PostgreSQL LISTEN/NOTIFY for a simple message queue. The code needs to be cross-platform Windows and Linux. My goal would be to eliminate that Boost.ASIO dependency for that, to use just libpq. PQsocketPoll() being based on time_t, it has only second resolution, AFAIK. Despite the [underlying implementation in fe-misc.c][2] supporting at least milliseconds. My use case is clients posting "requests" to the "queue" (i.e. a PostgreSQL table), which trigger NOTIFY messages, waited on by "servers"; and those servers informing back clients via further notifications (on per-request channels) about the processing status of their requests. In that use case, second-resolution on long-lived servers is OK. But OTOH, for the client side, waiting 1s or more to know whether a server picked up their request is "too long / slow". I'd need millisecond timeouts for that. The doc for PQsocketPoll() mentions a different use case for that API. But I think it would a pity if that unreleased API couldn't be made to accomodate sub-second timeouts and more use-cases, like above. Especially given that libpq v17 isn't out yet. I may come late to the game, but hopefully it is not too late. Thoughts? Thanks, --DD [1]: https://www.postgresql.org/docs/17/libpq-connect.html#LIBPQ-PQSOCKETPOLL [2]: https://github.com/postgres/postgres/blob/master/src/interfaces/libpq/fe-misc.c#L1086 ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: libpq v17 PQsocketPoll timeout is not granular enough 2024-06-10 14:36 libpq v17 PQsocketPoll timeout is not granular enough Dominique Devienne <[email protected]> @ 2024-06-10 22:59 ` Thomas Munro <[email protected]> 2024-06-10 23:49 ` Re: libpq v17 PQsocketPoll timeout is not granular enough Tom Lane <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Thomas Munro @ 2024-06-10 22:59 UTC (permalink / raw) To: Dominique Devienne <[email protected]>; +Cc: pgsql-general On Tue, Jun 11, 2024 at 2:36 AM Dominique Devienne <[email protected]> wrote: > Hi. I've noticed [that libpq API in v17 beta1][1], and wanted to use > it to replace an existing Boost.ASIO-based async polling of the > connection's socket, waiting for notifications. The use case being > using PostgreSQL LISTEN/NOTIFY for a simple message queue. The code > needs to be cross-platform Windows and Linux. My goal would be to > eliminate that Boost.ASIO dependency for that, to use just libpq. One idea I have wondered about is why you wouldn't just use poll() directly, if you want a facility that works on all known operating systems without extra portability libraries. Windows has WSApoll(), which AFAIK was designed to be Unix-compatible and a drop-in replacement, requiring just a rename but otherwise having the same macros and struct etc. For some period of time, people who had to deal with socket connection events (that includes us) avoided it, with the Curl guys' blog being the most often cited public explanation for why: https://daniel.haxx.se/blog/2012/10/10/wsapoll-is-broken/ However, as far as I know, that was fixed ~4 years ago: https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsapoll "Note As of Windows 10 version 2004, when a TCP socket fails to connect, (POLLHUP \| POLLERR \| POLLWRNORM) is indicated." I wonder if that means that it's now completely usable on all reasonable versions of the OS. I think so? I don't use Windows myself, my interest in this topic is that I have a slow moving background project to figure out how and when to remove all remaining uses of select() from our tree, and this one is on my hit list. > PQsocketPoll() being based on time_t, it has only second resolution, AFAIK. > Despite the [underlying implementation in fe-misc.c][2] supporting at > least milliseconds. Yeah, that is not nice and your complaint is very reasonable, and we should probably do something like what Tom suggested. Hmm, but if what I speculated above is true, I wonder if the extern function is even worth its bits... but I don't know how to determine that completely. ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: libpq v17 PQsocketPoll timeout is not granular enough 2024-06-10 14:36 libpq v17 PQsocketPoll timeout is not granular enough Dominique Devienne <[email protected]> 2024-06-10 22:59 ` Re: libpq v17 PQsocketPoll timeout is not granular enough Thomas Munro <[email protected]> @ 2024-06-10 23:49 ` Tom Lane <[email protected]> 2024-06-11 07:19 ` Re: libpq v17 PQsocketPoll timeout is not granular enough Francisco Olarte <[email protected]> 0 siblings, 1 reply; 4+ messages in thread From: Tom Lane @ 2024-06-10 23:49 UTC (permalink / raw) To: Thomas Munro <[email protected]>; +Cc: Dominique Devienne <[email protected]>; pgsql-general Thomas Munro <[email protected]> writes: > On Tue, Jun 11, 2024 at 2:36 AM Dominique Devienne <[email protected]> wrote: >> PQsocketPoll() being based on time_t, it has only second resolution, AFAIK. >> Despite the [underlying implementation in fe-misc.c][2] supporting at >> least milliseconds. > Yeah, that is not nice and your complaint is very reasonable, and we > should probably do something like what Tom suggested. > Hmm, but if what I speculated above is true, I wonder if the extern > function is even worth its bits... but I don't know how to determine > that completely. I think if we're going to change anything at all here, we should define the external API in microseconds not milliseconds. The lesson we need to be taking from this is that system calls come and go, but libpq API is forever ;-). Somebody could conceivably want sub-millisecond wait resolution within the lifespan of libpq. regards, tom lane ^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: libpq v17 PQsocketPoll timeout is not granular enough 2024-06-10 14:36 libpq v17 PQsocketPoll timeout is not granular enough Dominique Devienne <[email protected]> 2024-06-10 22:59 ` Re: libpq v17 PQsocketPoll timeout is not granular enough Thomas Munro <[email protected]> 2024-06-10 23:49 ` Re: libpq v17 PQsocketPoll timeout is not granular enough Tom Lane <[email protected]> @ 2024-06-11 07:19 ` Francisco Olarte <[email protected]> 0 siblings, 0 replies; 4+ messages in thread From: Francisco Olarte @ 2024-06-11 07:19 UTC (permalink / raw) To: Tom Lane <[email protected]>; +Cc: Thomas Munro <[email protected]>; Dominique Devienne <[email protected]>; pgsql-general Tom: On Tue, 11 Jun 2024 at 01:49, Tom Lane <[email protected]> wrote: > I think if we're going to change anything at all here, we should > define the external API in microseconds not milliseconds. The lesson > we need to be taking from this is that system calls come and go, > but libpq API is forever ;-). Somebody could conceivably want > sub-millisecond wait resolution within the lifespan of libpq. I was thinking on that, but since you need at least 20 bits, so 32, for microseconds I would suggest nanoseconds, which fit in 32 too. Sure, nanos seems too much for the current time but it pushes the future problem further down, nearly forever for comms between different machines until someone develops FTL networks. Francisco Olarte. ^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2024-06-11 07:19 UTC | newest] Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-06-10 14:36 libpq v17 PQsocketPoll timeout is not granular enough Dominique Devienne <[email protected]> 2024-06-10 22:59 ` Thomas Munro <[email protected]> 2024-06-10 23:49 ` Tom Lane <[email protected]> 2024-06-11 07:19 ` Francisco Olarte <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox