agora inbox for pgsql-admin@postgresql.org
help / color / mirror / Atom feedConnection pooler / LDAP auth / Load Balancing on read-only queries
7+ messages / 4 participants
[nested] [flat]
* Connection pooler / LDAP auth / Load Balancing on read-only queries
@ 2024-07-03 20:25 Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
2024-07-03 20:32 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Scott Ribe <scott_ribe@elevated-dev.com>
2024-07-03 20:32 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Holger Jakobs <holger@jakobs.com>
0 siblings, 2 replies; 7+ messages in thread
From: Achilleas Mantzios @ 2024-07-03 20:25 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Dear Members
I am searching (like many others) of the holy grail of PostgreSQL
pooling + Load Balancing . Ideally I would like :
- A pgbouncer with better/native LDAP (not PAM) / and eventually
Kerberos support
- A pgbouncer enhanced with load balancing read-only queries (like pgpool)
or
- A pgpool-II with better resources utilization / more efficient pooling
(like pgbouncer)
There are other solutions available pgcat (no LDAP), odyssey (no load
balancing) , pgagroal, supavisor, none of which seem to cover the above.
Some ppl advice , keeping pgbouncer close to the app(s) and pgpool close
to the DB. But kinda had mixed feelings putting the two work together.
I'd like to ask if there any thoughts or even hopes that some of the
above will be available in a single software, or otherwise how do people
tackle this.
Thank you
--
Achilleas Mantzios
IT DEV - HEAD
IT DEPT
Dynacom Tankers Mgmt (as agents only)
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
2024-07-03 20:25 Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
@ 2024-07-03 20:32 ` Scott Ribe <scott_ribe@elevated-dev.com>
2024-07-04 06:38 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
1 sibling, 1 reply; 7+ messages in thread
From: Scott Ribe @ 2024-07-03 20:32 UTC (permalink / raw)
To: Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>; +Cc: pgsql-admin@lists.postgresql.org
the read only detection is tricky--consider transaction level pooling, and the first query is read only
--
Scott Ribe
scott_ribe@elevated-dev.com
https://www.linkedin.com/in/scottribe/
> On Jul 3, 2024, at 2:25 PM, Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> wrote:
>
> Dear Members
>
> I am searching (like many others) of the holy grail of PostgreSQL pooling + Load Balancing . Ideally I would like :
>
> - A pgbouncer with better/native LDAP (not PAM) / and eventually Kerberos support
>
> - A pgbouncer enhanced with load balancing read-only queries (like pgpool)
>
> or
>
> - A pgpool-II with better resources utilization / more efficient pooling (like pgbouncer)
>
>
> There are other solutions available pgcat (no LDAP), odyssey (no load balancing) , pgagroal, supavisor, none of which seem to cover the above.
>
> Some ppl advice , keeping pgbouncer close to the app(s) and pgpool close to the DB. But kinda had mixed feelings putting the two work together.
>
> I'd like to ask if there any thoughts or even hopes that some of the above will be available in a single software, or otherwise how do people tackle this.
>
> Thank you
>
> --
> Achilleas Mantzios
> IT DEV - HEAD
> IT DEPT
> Dynacom Tankers Mgmt (as agents only)
>
>
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
2024-07-03 20:25 Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
2024-07-03 20:32 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Scott Ribe <scott_ribe@elevated-dev.com>
@ 2024-07-04 06:38 ` Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
2024-07-04 12:47 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Scott Ribe <scott_ribe@elevated-dev.com>
0 siblings, 1 reply; 7+ messages in thread
From: Achilleas Mantzios - cloud @ 2024-07-04 06:38 UTC (permalink / raw)
To: Scott Ribe <scott_ribe@elevated-dev.com>; +Cc: pgsql-admin@lists.postgresql.org
On 7/3/24 23:32, Scott Ribe wrote:
> the read only detection is tricky--consider transaction level pooling, and the first query is read only
Hello
pgpool does a great job at that. pgpool does load balancing to the first
statement that is considered a write statement, for that point on, the
rest of the transaction is routed to the primary.
But the question here is how to achieve all of the aforementioned
features, of all those great tools combined, or ideally combined in one.
>
> --
> Scott Ribe
> scott_ribe@elevated-dev.com
> https://www.linkedin.com/in/scottribe/
>
>
>
>> On Jul 3, 2024, at 2:25 PM, Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> wrote:
>>
>> Dear Members
>>
>> I am searching (like many others) of the holy grail of PostgreSQL pooling + Load Balancing . Ideally I would like :
>>
>> - A pgbouncer with better/native LDAP (not PAM) / and eventually Kerberos support
>>
>> - A pgbouncer enhanced with load balancing read-only queries (like pgpool)
>>
>> or
>>
>> - A pgpool-II with better resources utilization / more efficient pooling (like pgbouncer)
>>
>>
>> There are other solutions available pgcat (no LDAP), odyssey (no load balancing) , pgagroal, supavisor, none of which seem to cover the above.
>>
>> Some ppl advice , keeping pgbouncer close to the app(s) and pgpool close to the DB. But kinda had mixed feelings putting the two work together.
>>
>> I'd like to ask if there any thoughts or even hopes that some of the above will be available in a single software, or otherwise how do people tackle this.
>>
>> Thank you
>>
>> --
>> Achilleas Mantzios
>> IT DEV - HEAD
>> IT DEPT
>> Dynacom Tankers Mgmt (as agents only)
>>
>>
>>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
2024-07-03 20:25 Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
2024-07-03 20:32 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Scott Ribe <scott_ribe@elevated-dev.com>
2024-07-04 06:38 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
@ 2024-07-04 12:47 ` Scott Ribe <scott_ribe@elevated-dev.com>
2024-07-04 14:28 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
0 siblings, 1 reply; 7+ messages in thread
From: Scott Ribe @ 2024-07-04 12:47 UTC (permalink / raw)
To: Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>; +Cc: pgsql-admin@lists.postgresql.org
> Hello
>
> pgpool does a great job at that. pgpool does load balancing to the first statement that is considered a write statement, for that point on, the rest of the transaction is routed to the primary.
>
> But the question here is how to achieve all of the aforementioned features, of all those great tools combined, or ideally combined in one.
AFAIK, pgpool still doesn't do what I'd consider true pooling, that is MxN multiplexing of connections. (Every client connection has a connection from pgpool -> server, but the server connections become available for resuse when clients disconnect.)
And to me, the mechanism for routing queries in a transaction is highly suspect, as the initial reads vs later writes could potentially be using different snapshots of the data. (There is a safeguard there, involving looking at replication delay, but that's not a transactional guarantee, just "here's how out of date the replica can be to get read queries".)
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
2024-07-03 20:25 Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
2024-07-03 20:32 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Scott Ribe <scott_ribe@elevated-dev.com>
2024-07-04 06:38 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
2024-07-04 12:47 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Scott Ribe <scott_ribe@elevated-dev.com>
@ 2024-07-04 14:28 ` Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
0 siblings, 0 replies; 7+ messages in thread
From: Achilleas Mantzios - cloud @ 2024-07-04 14:28 UTC (permalink / raw)
To: Scott Ribe <scott_ribe@elevated-dev.com>; +Cc: pgsql-admin@lists.postgresql.org
On 7/4/24 15:47, Scott Ribe wrote:
>> Hello
>>
>> pgpool does a great job at that. pgpool does load balancing to the first statement that is considered a write statement, for that point on, the rest of the transaction is routed to the primary.
>>
>> But the question here is how to achieve all of the aforementioned features, of all those great tools combined, or ideally combined in one.
> AFAIK, pgpool still doesn't do what I'd consider true pooling, that is MxN multiplexing of connections. (Every client connection has a connection from pgpool -> server, but the server connections become available for resuse when clients disconnect.)
Yes, unfortunately, pgbouncer shines in this department.
>
> And to me, the mechanism for routing queries in a transaction is highly suspect, as the initial reads vs later writes could potentially be using different snapshots of the data. (There is a safeguard there, involving looking at replication delay, but that's not a transactional guarantee, just "here's how out of date the replica can be to get read queries".)
It seems pgpool supports snapshot_isolation mode, which is similar to
streaming_replication, + it adds visibility consistency, but at the
expense of running with default_transaction_isolation = 'repeatable
read' :
https://www.pgpool.net/docs/latest/en/html/runtime-config-running-mode.html#GUC-SNAPSHOT-ISOLATION-M...
Read latency is an issue with asynchronous physical replication, but
then again, the problem is there no matter the HA/pooling solution.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
2024-07-03 20:25 Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
@ 2024-07-03 20:32 ` Holger Jakobs <holger@jakobs.com>
2024-07-04 06:44 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
1 sibling, 1 reply; 7+ messages in thread
From: Holger Jakobs @ 2024-07-03 20:32 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Am 03.07.24 um 22:25 schrieb Achilleas Mantzios:
> Dear Members
>
> I am searching (like many others) of the holy grail of PostgreSQL
> pooling + Load Balancing . Ideally I would like :
>
> - A pgbouncer with better/native LDAP (not PAM) / and eventually
> Kerberos support
>
> - A pgbouncer enhanced with load balancing read-only queries (like
> pgpool)
>
> or
>
> - A pgpool-II with better resources utilization / more efficient
> pooling (like pgbouncer)
>
>
> There are other solutions available pgcat (no LDAP), odyssey (no load
> balancing) , pgagroal, supavisor, none of which seem to cover the above.
>
> Some ppl advice , keeping pgbouncer close to the app(s) and pgpool
> close to the DB. But kinda had mixed feelings putting the two work
> together.
>
> I'd like to ask if there any thoughts or even hopes that some of the
> above will be available in a single software, or otherwise how do
> people tackle this.
>
> Thank you
>
Have a look at pgCat, a newcomer which might meet your needs.
Cheers
Holger
--
Holger Jakobs, Bergisch Gladbach
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
2024-07-03 20:25 Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
2024-07-03 20:32 ` Re: Connection pooler / LDAP auth / Load Balancing on read-only queries Holger Jakobs <holger@jakobs.com>
@ 2024-07-04 06:44 ` Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
0 siblings, 0 replies; 7+ messages in thread
From: Achilleas Mantzios - cloud @ 2024-07-04 06:44 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
On 7/3/24 23:32, Holger Jakobs wrote:
> Am 03.07.24 um 22:25 schrieb Achilleas Mantzios:
>> Dear Members
>>
>> I am searching (like many others) of the holy grail of PostgreSQL
>> pooling + Load Balancing . Ideally I would like :
>>
>> - A pgbouncer with better/native LDAP (not PAM) / and eventually
>> Kerberos support
>>
>> - A pgbouncer enhanced with load balancing read-only queries (like
>> pgpool)
>>
>> or
>>
>> - A pgpool-II with better resources utilization / more efficient
>> pooling (like pgbouncer)
>>
>>
>> There are other solutions available pgcat (no LDAP), odyssey (no load
>> balancing) , pgagroal, supavisor, none of which seem to cover the above.
>>
>> Some ppl advice , keeping pgbouncer close to the app(s) and pgpool
>> close to the DB. But kinda had mixed feelings putting the two work
>> together.
>>
>> I'd like to ask if there any thoughts or even hopes that some of the
>> above will be available in a single software, or otherwise how do
>> people tackle this.
>>
>> Thank you
>>
>
> Have a look at pgCat, a newcomer which might meet your needs.
Hello,
I already mentioned pgCat, and the reason it won't cut it. (no PAM, no
LDAP, only MD5/SCRAM-SHA-256)
>
> Cheers
>
> Holger
>
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2024-07-04 14:28 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-07-03 20:25 Connection pooler / LDAP auth / Load Balancing on read-only queries Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>
2024-07-03 20:32 ` Scott Ribe <scott_ribe@elevated-dev.com>
2024-07-04 06:38 ` Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
2024-07-04 12:47 ` Scott Ribe <scott_ribe@elevated-dev.com>
2024-07-04 14:28 ` Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
2024-07-03 20:32 ` Holger Jakobs <holger@jakobs.com>
2024-07-04 06:44 ` Achilleas Mantzios - cloud <a.mantzios@cloud.gatewaynet.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox