pg.ddx.io pgsql-interfaces@postgresql.org mailing list archive
help / color / mirror / Atom feedlibpq + multiple connections ...
11+ messages / 5 participants
[nested] [flat]
* libpq + multiple connections ...
@ 1999-11-29 07:01 The Hermit Hacker <scrappy@hub.org>
0 siblings, 1 reply; 11+ messages in thread
From: The Hermit Hacker @ 1999-11-29 07:01 UTC (permalink / raw)
To: pgsql-interfaces
Anyone able to comment on this? I haven' tused libpq in so long that I
can't :(
======
Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
function to libpq and I stumbled across some problems.
It seems that libpq makes use of some static variables, meaning i'm not
sure if it's safe to use libpq for multiple database connections.
What i'm refering to is:
postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
line 79 has a structure that seems to be shared amongst the entire
library, am I likely to stumble upon more stuff that makes it somewhat
dangerous to have more than one active database connection in my program?
=======
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-29 07:45 Tom Lane <tgl@sss.pgh.pa.us>
parent: The Hermit Hacker <scrappy@hub.org>
0 siblings, 3 replies; 11+ messages in thread
From: Tom Lane @ 1999-11-29 07:45 UTC (permalink / raw)
To: The Hermit Hacker <scrappy@hub.org>; +Cc: pgsql-interfaces
The Hermit Hacker <scrappy@hub.org> forwards:
> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> function to libpq and I stumbled across some problems.
Someone else was already working on that --- check the archives from
a few months back.
> It seems that libpq makes use of some static variables, meaning i'm not
> sure if it's safe to use libpq for multiple database connections.
> What i'm refering to is:
> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> line 79 has a structure that seems to be shared amongst the entire
> library, am I likely to stumble upon more stuff that makes it somewhat
> dangerous to have more than one active database connection in my program?
The PQconnectdb() function uses that static array, meaning that you can't
safely run two PQconnectdb() calls in parallel. But you can open two
connections in sequence and then use them in parallel; and you can do
the opens in parallel if you use one of the older, less-friendly
connection-opening calls. There aren't any other non-constant statics
in libpq AFAIR.
The static array sucks, I agree, but I don't see any way to get rid of
it without changing libpq's API for PQconnectdb() and PQconndefaults().
Do we want to consider doing that (and breaking some apps) for 7.0?
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-29 07:48 The Hermit Hacker <scrappy@hub.org>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 0 replies; 11+ messages in thread
From: The Hermit Hacker @ 1999-11-29 07:48 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-interfaces
On Mon, 29 Nov 1999, Tom Lane wrote:
> The Hermit Hacker <scrappy@hub.org> forwards:
> > Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> > function to libpq and I stumbled across some problems.
>
> Someone else was already working on that --- check the archives from
> a few months back.
>
> > It seems that libpq makes use of some static variables, meaning i'm not
> > sure if it's safe to use libpq for multiple database connections.
> > What i'm refering to is:
> > postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> > line 79 has a structure that seems to be shared amongst the entire
> > library, am I likely to stumble upon more stuff that makes it somewhat
> > dangerous to have more than one active database connection in my program?
>
> The PQconnectdb() function uses that static array, meaning that you can't
> safely run two PQconnectdb() calls in parallel. But you can open two
> connections in sequence and then use them in parallel; and you can do
> the opens in parallel if you use one of the older, less-friendly
> connection-opening calls. There aren't any other non-constant statics
> in libpq AFAIR.
>
> The static array sucks, I agree, but I don't see any way to get rid of
> it without changing libpq's API for PQconnectdb() and PQconndefaults().
> Do we want to consider doing that (and breaking some apps) for 7.0?
IMHO...7.0 let's us pretty much go hog-wild as far as API changes are
concerned...if it is something that a) can be done and b) shoudl be
done...now is about the best time to do it...
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-29 10:42 E.E. Mellor <eem21@cam.ac.uk>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 1 reply; 11+ messages in thread
From: E.E. Mellor @ 1999-11-29 10:42 UTC (permalink / raw)
To: pgsql-interfaces; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; The Hermit Hacker <scrappy@hub.org>
--On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
> The Hermit Hacker <scrappy@hub.org> forwards:
>> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
>> function to libpq and I stumbled across some problems.
>
> Someone else was already working on that --- check the archives from
> a few months back.
'twas me. In fact, it's been ready for ages, but I've had so much on that
I've not had time to write the docs. I'll do them today for you.
>> It seems that libpq makes use of some static variables, meaning i'm not
>> sure if it's safe to use libpq for multiple database connections.
>> What i'm refering to is:
>> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
>> line 79 has a structure that seems to be shared amongst the entire
>> library, am I likely to stumble upon more stuff that makes it somewhat
>> dangerous to have more than one active database connection in my program?
>
> The PQconnectdb() function uses that static array, meaning that you can't
> safely run two PQconnectdb() calls in parallel. But you can open two
> connections in sequence and then use them in parallel; and you can do
> the opens in parallel if you use one of the older, less-friendly
> connection-opening calls. There aren't any other non-constant statics
> in libpq AFAIR.
This is correct. Someone else was looking at thread-safety across the
whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
thread-safe anywhere. My changes allow one to use libpq in a single
thread, but with multiple connections and without blocking the thread at
all.
> The static array sucks, I agree, but I don't see any way to get rid of
> it without changing libpq's API for PQconnectdb() and PQconndefaults().
> Do we want to consider doing that (and breaking some apps) for 7.0?
I think it should be possible to arrange it so that these functions still
exist (but are deprecated and marked as non-thread-safe). I'll leave that
for others to decide though.
Ewan Mellor.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-29 12:19 Roberto Joao Lopes Garcia <roberto@mha.com.br>
parent: Tom Lane <tgl@sss.pgh.pa.us>
2 siblings, 1 reply; 11+ messages in thread
From: Roberto Joao Lopes Garcia @ 1999-11-29 12:19 UTC (permalink / raw)
To: pgsql-interfaces
>> safely run two PQconnectdb() calls in parallel. But you can open two
>> connections in sequence and then use them in parallel;
>
So I could open, for exemple, 5 db connections (with PQconnectdb()) and
then create 5 threads and use those connections in eache thread. But how
about use PQreset() or PQfinish() inside the new created threads?
I used to test connection before send a query like bellow:
if(PQstatus(con)==CONNECTION_BAD) {
PQreset(con);
if(PQstatus(con)==CONNECTION_BAD) erro_fatal();
}
Should I protect the calls to those functions by an mutex?? Could it
work??? Which functions read or alter Qconnectdb()'s static array??
I think it is important to mark wich functions in Libpq is thread safe or
not.
Roberto
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-29 21:05 The Hermit Hacker <scrappy@hub.org>
parent: E.E. Mellor <eem21@cam.ac.uk>
0 siblings, 1 reply; 11+ messages in thread
From: The Hermit Hacker @ 1999-11-29 21:05 UTC (permalink / raw)
To: E.E. Mellor <eem21@cam.ac.uk>; +Cc: pgsql-interfaces@postgreSQL.org, Tom Lane <tgl@sss.pgh.pa.us>
Is this something that could *safely* be back-patched to a v6.5.4 release?
On Mon, 29 Nov 1999, E.E. Mellor wrote:
> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> > The Hermit Hacker <scrappy@hub.org> forwards:
> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> >> function to libpq and I stumbled across some problems.
> >
> > Someone else was already working on that --- check the archives from
> > a few months back.
>
> 'twas me. In fact, it's been ready for ages, but I've had so much on that
> I've not had time to write the docs. I'll do them today for you.
>
> >> It seems that libpq makes use of some static variables, meaning i'm not
> >> sure if it's safe to use libpq for multiple database connections.
> >> What i'm refering to is:
> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> >> line 79 has a structure that seems to be shared amongst the entire
> >> library, am I likely to stumble upon more stuff that makes it somewhat
> >> dangerous to have more than one active database connection in my program?
> >
> > The PQconnectdb() function uses that static array, meaning that you can't
> > safely run two PQconnectdb() calls in parallel. But you can open two
> > connections in sequence and then use them in parallel; and you can do
> > the opens in parallel if you use one of the older, less-friendly
> > connection-opening calls. There aren't any other non-constant statics
> > in libpq AFAIR.
>
> This is correct. Someone else was looking at thread-safety across the
> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
> thread-safe anywhere. My changes allow one to use libpq in a single
> thread, but with multiple connections and without blocking the thread at
> all.
>
> > The static array sucks, I agree, but I don't see any way to get rid of
> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
> > Do we want to consider doing that (and breaking some apps) for 7.0?
>
> I think it should be possible to arrange it so that these functions still
> exist (but are deprecated and marked as non-thread-safe). I'll leave that
> for others to decide though.
>
> Ewan Mellor.
>
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-29 21:18 eem21@cam.ac.uk
parent: The Hermit Hacker <scrappy@hub.org>
0 siblings, 1 reply; 11+ messages in thread
From: eem21@cam.ac.uk @ 1999-11-29 21:18 UTC (permalink / raw)
To: scrappy@hub.org; +Cc: pgsql-interfaces; tgl@sss.pgh.pa.us
On 29 Nov, The Hermit Hacker wrote:
>
> Is this something that could *safely* be back-patched to a v6.5.4 release?
Well, it's not _supposed_ to have changed the behaviour for existing
functions. However, there's been quite a lot of chopping and changing,
and I might have lost something somewhere. That said, it would be of
great help to me if it made it into a 6.5.x release, and I would thus
be willing to test it. I can only test ix86/Linux, unfortunately.
When would it be released, if that were the case?
BTW, how do I find out about the SGML tags we are using (I've never
written any SGML before, only HTML).
Ewan.
>
> On Mon, 29 Nov 1999, E.E. Mellor wrote:
>
>> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>
>> > The Hermit Hacker <scrappy@hub.org> forwards:
>> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
>> >> function to libpq and I stumbled across some problems.
>> >
>> > Someone else was already working on that --- check the archives from
>> > a few months back.
>>
>> 'twas me. In fact, it's been ready for ages, but I've had so much on that
>> I've not had time to write the docs. I'll do them today for you.
>>
>> >> It seems that libpq makes use of some static variables, meaning i'm not
>> >> sure if it's safe to use libpq for multiple database connections.
>> >> What i'm refering to is:
>> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
>> >> line 79 has a structure that seems to be shared amongst the entire
>> >> library, am I likely to stumble upon more stuff that makes it somewhat
>> >> dangerous to have more than one active database connection in my program?
>> >
>> > The PQconnectdb() function uses that static array, meaning that you can't
>> > safely run two PQconnectdb() calls in parallel. But you can open two
>> > connections in sequence and then use them in parallel; and you can do
>> > the opens in parallel if you use one of the older, less-friendly
>> > connection-opening calls. There aren't any other non-constant statics
>> > in libpq AFAIR.
>>
>> This is correct. Someone else was looking at thread-safety across the
>> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
>> thread-safe anywhere. My changes allow one to use libpq in a single
>> thread, but with multiple connections and without blocking the thread at
>> all.
>>
>> > The static array sucks, I agree, but I don't see any way to get rid of
>> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
>> > Do we want to consider doing that (and breaking some apps) for 7.0?
>>
>> I think it should be possible to arrange it so that these functions still
>> exist (but are deprecated and marked as non-thread-safe). I'll leave that
>> for others to decide though.
>>
>> Ewan Mellor.
>>
>
> Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
> Systems Administrator @ hub.org
> primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
>
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-29 21:25 The Hermit Hacker <scrappy@hub.org>
parent: eem21@cam.ac.uk
0 siblings, 2 replies; 11+ messages in thread
From: The Hermit Hacker @ 1999-11-29 21:25 UTC (permalink / raw)
To: eem21@cam.ac.uk; +Cc: pgsql-interfaces; tgl@sss.pgh.pa.us
On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
> On 29 Nov, The Hermit Hacker wrote:
> >
> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>
> Well, it's not _supposed_ to have changed the behaviour for existing
> functions. However, there's been quite a lot of chopping and changing,
> and I might have lost something somewhere. That said, it would be of
> great help to me if it made it into a 6.5.x release, and I would thus
> be willing to test it. I can only test ix86/Linux, unfortunately.
> When would it be released, if that were the case?
If you can get a patch in so that we can test it, would love to get a
release out by ... oh ... end of week? :)
Anyone object overly?
> BTW, how do I find out about the SGML tags we are using (I've never
> written any SGML before, only HTML).
>
> Ewan.
>
> >
> > On Mon, 29 Nov 1999, E.E. Mellor wrote:
> >
> >> --On 29 November 1999, 02:45 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >>
> >> > The Hermit Hacker <scrappy@hub.org> forwards:
> >> >> Hmm, sorta, I'm a bit troubled, I was trying to add an async connection
> >> >> function to libpq and I stumbled across some problems.
> >> >
> >> > Someone else was already working on that --- check the archives from
> >> > a few months back.
> >>
> >> 'twas me. In fact, it's been ready for ages, but I've had so much on that
> >> I've not had time to write the docs. I'll do them today for you.
> >>
> >> >> It seems that libpq makes use of some static variables, meaning i'm not
> >> >> sure if it's safe to use libpq for multiple database connections.
> >> >> What i'm refering to is:
> >> >> postgresql-6.5.3/src/interfaces/libpq/fe-connect.c
> >> >> line 79 has a structure that seems to be shared amongst the entire
> >> >> library, am I likely to stumble upon more stuff that makes it somewhat
> >> >> dangerous to have more than one active database connection in my program?
> >> >
> >> > The PQconnectdb() function uses that static array, meaning that you can't
> >> > safely run two PQconnectdb() calls in parallel. But you can open two
> >> > connections in sequence and then use them in parallel; and you can do
> >> > the opens in parallel if you use one of the older, less-friendly
> >> > connection-opening calls. There aren't any other non-constant statics
> >> > in libpq AFAIR.
> >>
> >> This is correct. Someone else was looking at thread-safety across the
> >> whole of libpq, but as far as I'm aware, it's not _guaranteed_ to be
> >> thread-safe anywhere. My changes allow one to use libpq in a single
> >> thread, but with multiple connections and without blocking the thread at
> >> all.
> >>
> >> > The static array sucks, I agree, but I don't see any way to get rid of
> >> > it without changing libpq's API for PQconnectdb() and PQconndefaults().
> >> > Do we want to consider doing that (and breaking some apps) for 7.0?
> >>
> >> I think it should be possible to arrange it so that these functions still
> >> exist (but are deprecated and marked as non-thread-safe). I'll leave that
> >> for others to decide though.
> >>
> >> Ewan Mellor.
> >>
> >
> > Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
> > Systems Administrator @ hub.org
> > primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
> >
>
>
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-29 21:29 eem21@cam.ac.uk
parent: The Hermit Hacker <scrappy@hub.org>
1 sibling, 0 replies; 11+ messages in thread
From: eem21@cam.ac.uk @ 1999-11-29 21:29 UTC (permalink / raw)
To: scrappy@hub.org; +Cc: pgsql-interfaces; tgl@sss.pgh.pa.us
On 29 Nov, The Hermit Hacker wrote:
> On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
>
>> On 29 Nov, The Hermit Hacker wrote:
>> >
>> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>>
>> Well, it's not _supposed_ to have changed the behaviour for existing
>> functions. However, there's been quite a lot of chopping and changing,
>> and I might have lost something somewhere. That said, it would be of
>> great help to me if it made it into a 6.5.x release, and I would thus
>> be willing to test it. I can only test ix86/Linux, unfortunately.
>> When would it be released, if that were the case?
>
> If you can get a patch in so that we can test it
I'm doin' it, I'm doin' it... ;-)
> would love to get a release out by ... oh ... end of week? :)
>
> Anyone object overly?
Nope.
> [Snip async functions]
Ewan.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-30 02:18 eem21@cam.ac.uk
parent: The Hermit Hacker <scrappy@hub.org>
1 sibling, 0 replies; 11+ messages in thread
From: eem21@cam.ac.uk @ 1999-11-30 02:18 UTC (permalink / raw)
To: scrappy@hub.org; +Cc: pgsql-interfaces; tgl@sss.pgh.pa.us
On 29 Nov, The Hermit Hacker wrote:
> On Mon, 29 Nov 1999 eem21@cam.ac.uk wrote:
>
>> On 29 Nov, The Hermit Hacker wrote:
>> >
>> > Is this something that could *safely* be back-patched to a v6.5.4 release?
>>
>> Well, it's not _supposed_ to have changed the behaviour for existing
>> functions. However, there's been quite a lot of chopping and changing,
>> and I might have lost something somewhere. That said, it would be of
>> great help to me if it made it into a 6.5.x release, and I would thus
>> be willing to test it. I can only test ix86/Linux, unfortunately.
>> When would it be released, if that were the case?
>
> If you can get a patch in so that we can test it, would love to get a
> release out by ... oh ... end of week? :)
Patch submitted.
> [Snip]
Ewan.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: [INTERFACES] libpq + multiple connections ...
@ 1999-11-30 12:08 E.E. Mellor <eem21@cam.ac.uk>
parent: Roberto Joao Lopes Garcia <roberto@mha.com.br>
0 siblings, 0 replies; 11+ messages in thread
From: E.E. Mellor @ 1999-11-30 12:08 UTC (permalink / raw)
To: Roberto Joao Lopes Garcia <roberto@mha.com.br>; +Cc: pgsql-interfaces
On Mon, 29 Nov 1999, Roberto Joao Lopes Garcia wrote:
> >> safely run two PQconnectdb() calls in parallel. But you can open two
> >> connections in sequence and then use them in parallel;
> >
>
> So I could open, for exemple, 5 db connections (with PQconnectdb()) and
> then create 5 threads and use those connections in eache thread. But how
> about use PQreset() or PQfinish() inside the new created threads?
>
> I used to test connection before send a query like bellow:
>
> if(PQstatus(con)==CONNECTION_BAD) {
> PQreset(con);
> if(PQstatus(con)==CONNECTION_BAD) erro_fatal();
> }
>
> Should I protect the calls to those functions by an mutex?? Could it
> work??? Which functions read or alter Qconnectdb()'s static array??
You cannot mix calls to _any_ of PQconnectdb, PQreset, or their
newly-added asynchronous analogues in a thread-safe manner. You need to
use a mutex to protect the calls above.
> I think it is important to mark wich functions in Libpq is thread safe or
> not.
I agree. I'll add a note to that effect to the docs today.
Ewan.
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~1999-11-30 12:08 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
1999-11-29 07:01 libpq + multiple connections ... The Hermit Hacker <scrappy@hub.org>
1999-11-29 07:45 ` Tom Lane <tgl@sss.pgh.pa.us>
1999-11-29 07:48 ` The Hermit Hacker <scrappy@hub.org>
1999-11-29 10:42 ` E.E. Mellor <eem21@cam.ac.uk>
1999-11-29 21:05 ` The Hermit Hacker <scrappy@hub.org>
1999-11-29 21:18 ` eem21@cam.ac.uk
1999-11-29 21:25 ` The Hermit Hacker <scrappy@hub.org>
1999-11-29 21:29 ` eem21@cam.ac.uk
1999-11-30 02:18 ` eem21@cam.ac.uk
1999-11-29 12:19 ` Roberto Joao Lopes Garcia <roberto@mha.com.br>
1999-11-30 12:08 ` E.E. Mellor <eem21@cam.ac.uk>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox