pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedRE : RE: Postgresql vs SQLserver for this application ?
5+ messages / 4 participants
[nested] [flat]
* RE : RE: Postgresql vs SQLserver for this application ?
@ 2005-04-06 07:17 bsimon@loxane.com
0 siblings, 1 reply; 5+ messages in thread
From: bsimon@loxane.com @ 2005-04-06 07:17 UTC (permalink / raw)
To: pgsql-performance; +Cc: Mohan, Ross <RMohan@arbinet.com>
Unfortunately.
But we are in the the process to choose Postgresql with pgcluster. I'm
currently running some tests (performance, stability...)
Save the money on the license fees, you get it for your hardware ;-)
I still welcome any advices or comments and I'll let you know how the
project is going on.
Benjamin.
"Mohan, Ross" <RMohan@arbinet.com>
05/04/2005 20:48
Pour : <bsimon@loxane.com>
cc :
Objet : RE: [PERFORM] Postgresql vs SQLserver for this application ?
You never got answers on this? Apologies, I don't have one, but'd be
curious to hear about any you did get....
thx
Ross
-----Original Message-----
From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org] On Behalf Of bsimon@loxane.com
Sent: Monday, April 04, 2005 4:02 AM
To: pgsql-performance@postgresql.org
Subject: [PERFORM] Postgresql vs SQLserver for this application ?
hi all.
We are designing a quite big application that requires a high-performance
database backend.
The rates we need to obtain are at least 5000 inserts per second and 15
selects per second for one connection. There should only be 3 or 4
simultaneous connections.
I think our main concern is to deal with the constant flow of data coming
from the inserts that must be available for selection as fast as possible.
(kind of real time access ...)
As a consequence, the database should rapidly increase up to more than one
hundred gigs. We still have to determine how and when we shoud backup old
data to prevent the application from a performance drop. We intend to
develop some kind of real-time partionning on our main table keep the
flows up.
At first, we were planning to use SQL Server as it has features that in my
opinion could help us a lot :
- replication
- clustering
Recently we started to study Postgresql as a solution for our project :
- it also has replication
- Postgis module can handle geographic datatypes (which would
facilitate our developments)
- We do have a strong knowledge on Postgresql administration (we
use it for production processes)
- it is free (!) and we could save money for hardware purchase.
Is SQL server clustering a real asset ? How reliable are Postgresql
replication tools ? Should I trust Postgresql performance for this kind
of needs ?
My question is a bit fuzzy but any advices are most welcome...
hardware,tuning or design tips as well :))
Thanks a lot.
Benjamin.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: RE : RE: Postgresql vs SQLserver for this application ?
@ 2005-04-06 15:37 Alex Turner <armtuk@gmail.com>
parent: bsimon@loxane.com
0 siblings, 0 replies; 5+ messages in thread
From: Alex Turner @ 2005-04-06 15:37 UTC (permalink / raw)
To: bsimon@loxane.com <bsimon@loxane.com>; +Cc: pgsql-performance@postgresql.org, "Mohan, Ross" <RMohan@arbinet.com>
I think everyone was scared off by the 5000 inserts per second number.
I've never seen even Oracle do this on a top end Dell system with
copious SCSI attached storage.
Alex Turner
netEconomist
On Apr 6, 2005 3:17 AM, bsimon@loxane.com <bsimon@loxane.com> wrote:
>
> Unfortunately.
>
> But we are in the the process to choose Postgresql with pgcluster. I'm
> currently running some tests (performance, stability...)
> Save the money on the license fees, you get it for your hardware ;-)
>
> I still welcome any advices or comments and I'll let you know how the
> project is going on.
>
> Benjamin.
>
>
>
> "Mohan, Ross" <RMohan@arbinet.com>
>
> 05/04/2005 20:48
>
> Pour : <bsimon@loxane.com>
> cc :
> Objet : RE: [PERFORM] Postgresql vs SQLserver for this
> application ?
>
>
> You never got answers on this? Apologies, I don't have one, but'd be curious
> to hear about any you did get....
>
> thx
>
> Ross
>
> -----Original Message-----
> From: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] On Behalf
> Of bsimon@loxane.com
> Sent: Monday, April 04, 2005 4:02 AM
> To: pgsql-performance@postgresql.org
> Subject: [PERFORM] Postgresql vs SQLserver for this application ?
>
>
> hi all.
>
> We are designing a quite big application that requires a high-performance
> database backend.
> The rates we need to obtain are at least 5000 inserts per second and 15
> selects per second for one connection. There should only be 3 or 4
> simultaneous connections.
> I think our main concern is to deal with the constant flow of data coming
> from the inserts that must be available for selection as fast as possible.
> (kind of real time access ...)
>
> As a consequence, the database should rapidly increase up to more than one
> hundred gigs. We still have to determine how and when we shoud backup old
> data to prevent the application from a performance drop. We intend to
> develop some kind of real-time partionning on our main table keep the flows
> up.
>
> At first, we were planning to use SQL Server as it has features that in my
> opinion could help us a lot :
> - replication
> - clustering
>
> Recently we started to study Postgresql as a solution for our project :
> - it also has replication
> - Postgis module can handle geographic datatypes (which would
> facilitate our developments)
> - We do have a strong knowledge on Postgresql administration (we use
> it for production processes)
> - it is free (!) and we could save money for hardware purchase.
>
> Is SQL server clustering a real asset ? How reliable are Postgresql
> replication tools ? Should I trust Postgresql performance for this kind of
> needs ?
>
> My question is a bit fuzzy but any advices are most welcome...
> hardware,tuning or design tips as well :))
>
> Thanks a lot.
>
> Benjamin.
>
>
>
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: RE : RE: Postgresql vs SQLserver for this application ?
@ 2005-04-06 16:12 Mohan, Ross <RMohan@arbinet.com>
0 siblings, 1 reply; 5+ messages in thread
From: Mohan, Ross @ 2005-04-06 16:12 UTC (permalink / raw)
To: pgsql-performance
I wish I had a Dell system and run case to show you Alex, but I don't...
however...using Oracle's "direct path" feature, it's pretty straightforward.
We've done 110,000 rows per second into index-less tables on a big system
(IBM Power5 chips, Hitachi SAN). ( Yes, I am sure: over 100K a second. Sustained
for almost 9 minutes. )
Yes, this is an exception, but oracle directpath/InsertAppend/BulkLoad
feature enabled us to migrate a 4 TB database...really quickly.
Now...if you ask me "can this work without Power5 and Hitachi SAN?"
my answer is..you give me a top end Dell and SCSI III on 15K disks
and I'll likely easily match it, yea.
I'd love to see PG get into this range..i am a big fan of PG (just a
rank newbie) but I gotta think the underlying code to do this has
to be not-too-complex.....
Best,
Ross
-----Original Message-----
From: Alex Turner [mailto:armtuk@gmail.com]
Sent: Wednesday, April 06, 2005 11:38 AM
To: bsimon@loxane.com
Cc: pgsql-performance@postgresql.org; Mohan, Ross
Subject: Re: RE : RE: [PERFORM] Postgresql vs SQLserver for this application ?
I think everyone was scared off by the 5000 inserts per second number.
I've never seen even Oracle do this on a top end Dell system with copious SCSI attached storage.
Alex Turner
netEconomist
On Apr 6, 2005 3:17 AM, bsimon@loxane.com <bsimon@loxane.com> wrote:
>
> Unfortunately.
>
> But we are in the the process to choose Postgresql with pgcluster. I'm
> currently running some tests (performance, stability...) Save the
> money on the license fees, you get it for your hardware ;-)
>
> I still welcome any advices or comments and I'll let you know how the
> project is going on.
>
> Benjamin.
>
>
>
> "Mohan, Ross" <RMohan@arbinet.com>
>
> 05/04/2005 20:48
>
> Pour : <bsimon@loxane.com>
> cc :
> Objet : RE: [PERFORM] Postgresql vs SQLserver for this
> application ?
>
>
> You never got answers on this? Apologies, I don't have one, but'd be
> curious to hear about any you did get....
>
> thx
>
> Ross
>
> -----Original Message-----
> From: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org] On Behalf
> Of bsimon@loxane.com
> Sent: Monday, April 04, 2005 4:02 AM
> To: pgsql-performance@postgresql.org
> Subject: [PERFORM] Postgresql vs SQLserver for this application ?
>
>
> hi all.
>
> We are designing a quite big application that requires a
> high-performance database backend. The rates we need to obtain are at
> least 5000 inserts per second and 15 selects per second for one
> connection. There should only be 3 or 4 simultaneous connections.
> I think our main concern is to deal with the constant flow of data coming
> from the inserts that must be available for selection as fast as possible.
> (kind of real time access ...)
>
> As a consequence, the database should rapidly increase up to more
> than one hundred gigs. We still have to determine how and when we
> shoud backup old data to prevent the application from a performance
> drop. We intend to develop some kind of real-time partionning on our
> main table keep the flows up.
>
> At first, we were planning to use SQL Server as it has features that
> in my opinion could help us a lot :
> - replication
> - clustering
>
> Recently we started to study Postgresql as a solution for our project :
> - it also has replication
> - Postgis module can handle geographic datatypes (which would
> facilitate our developments)
> - We do have a strong knowledge on Postgresql administration
> (we use it for production processes)
> - it is free (!) and we could save money for hardware
> purchase.
>
> Is SQL server clustering a real asset ? How reliable are Postgresql
> replication tools ? Should I trust Postgresql performance for this
> kind of needs ?
>
> My question is a bit fuzzy but any advices are most welcome...
> hardware,tuning or design tips as well :))
>
> Thanks a lot.
>
> Benjamin.
>
>
>
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: RE : RE: Postgresql vs SQLserver for this application
@ 2005-04-06 16:38 Steve Wampler <swampler@noao.edu>
parent: Mohan, Ross <RMohan@arbinet.com>
0 siblings, 0 replies; 5+ messages in thread
From: Steve Wampler @ 2005-04-06 16:38 UTC (permalink / raw)
To: Mohan, Ross <RMohan@arbinet.com>; +Cc: pgsql-performance
Mohan, Ross wrote:
> I wish I had a Dell system and run case to show you Alex, but I don't...
> however...using Oracle's "direct path" feature, it's pretty straightforward.
>
> We've done 110,000 rows per second into index-less tables on a big system
> (IBM Power5 chips, Hitachi SAN). ( Yes, I am sure: over 100K a second. Sustained
> for almost 9 minutes. )
>
> Yes, this is an exception, but oracle directpath/InsertAppend/BulkLoad
> feature enabled us to migrate a 4 TB database...really quickly.
How close to this is PG's COPY? I get surprisingly good results using
COPY with jdbc on smallish systems (now if that patch would make into
the mainstream PG jdbc support!) I think COPY has a bit more overhead
than what a Bulkload feature may have, but I suspect it's not that
much more.
> Now...if you ask me "can this work without Power5 and Hitachi SAN?"
> my answer is..you give me a top end Dell and SCSI III on 15K disks
> and I'll likely easily match it, yea.
>
> I'd love to see PG get into this range..i am a big fan of PG (just a
> rank newbie) but I gotta think the underlying code to do this has
> to be not-too-complex.....
It may not be that far off if you can use COPY instead of INSERT.
But comparing Bulkload to INSERT is a bit apples<->orangish.
--
Steve Wampler -- swampler@noao.edu
The gods that smiled on your birth are now laughing out loud.
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: RE : RE: Postgresql vs SQLserver for this application ?
@ 2005-04-06 16:43 Mohan, Ross <RMohan@arbinet.com>
0 siblings, 0 replies; 5+ messages in thread
From: Mohan, Ross @ 2005-04-06 16:43 UTC (permalink / raw)
To: pgsql-performance
How close to this is PG's COPY? I get surprisingly good results using COPY with jdbc on smallish systems (now if that patch would make into the mainstream PG jdbc support!) I think COPY has a bit more overhead than what a Bulkload feature may have, but I suspect it's not that much more.
|| Steve, I do not know. But am reading the docs now, and should figure it out. Ask
me later if you remember. Oracle's "direct path" is a way of just slamming blocks
filled with rows into the table, above the high water mark. It sidesteps freelist
management and all manner of intrablock issues. There is a "payback", but the benefits
far far outweigh the costs.
> Now...if you ask me "can this work without Power5 and Hitachi SAN?" my
> answer is..you give me a top end Dell and SCSI III on 15K disks and
> I'll likely easily match it, yea.
>
> I'd love to see PG get into this range..i am a big fan of PG (just a
> rank newbie) but I gotta think the underlying code to do this has to
> be not-too-complex.....
It may not be that far off if you can use COPY instead of INSERT. But comparing Bulkload to INSERT is a bit apples<->orangish.
|| Oh! I see! I had no idea I was doing that! Thanks for pointing it out clearly to me. Yea, I would
say a full transactional INSERT of 5K rows/sec into an indexed-table is a near-mythology without significant
caveats (parallelized, deferred buffering, etc.)
--
Steve Wampler -- swampler@noao.edu
The gods that smiled on your birth are now laughing out loud.
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2005-04-06 16:43 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-04-06 07:17 RE : RE: Postgresql vs SQLserver for this application ? bsimon@loxane.com
2005-04-06 15:37 ` Alex Turner <armtuk@gmail.com>
2005-04-06 16:12 Re: RE : RE: Postgresql vs SQLserver for this application ? Mohan, Ross <RMohan@arbinet.com>
2005-04-06 16:38 ` Re: RE : RE: Postgresql vs SQLserver for this application Steve Wampler <swampler@noao.edu>
2005-04-06 16:43 Re: RE : RE: Postgresql vs SQLserver for this application ? Mohan, Ross <RMohan@arbinet.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox