From: Ivan Voras <ivoras@freebsd.org>
To: pgsql-performance@postgresql.org
Subject: Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins
Date: Thu, 18 Nov 2010 00:27:35 +0100
Message-ID: <ic1ode$mt8$1@dough.gmane.org> (raw)
In-Reply-To: <35FABCF85D99464FB00BC5123DC2A70A0B7EFC38@s228130hz1ew09.apptix-01.savvis.net>
References: <ic0onf$3fi$1@news.hub.org>
<AANLkTiky7UGdCbpdrNMAiwo69WFgv0sEah=_ov0nCN0J@mail.gmail.com>
<35FABCF85D99464FB00BC5123DC2A70A0B7EFC38@s228130hz1ew09.apptix-01.savvis.net>
On 11/17/10 22:11, Eric Comeau wrote:
>> *) what kind of data do you expect to be writing out at this speed?
>>
> Large Video files ... our s/w is used to displace FTP.
>>
>> *) how many transactions per second will you expect to have?
>>
> Ideally 1 large file, but it may have to be multiple. We find that if we
> send multiple files it just causes the disk to thrash more so we get
> better throughput by sending one large file.
>
>> *) what is the architecture of the client? how many connections will
>> be open to postgres writing?
>>
> Our s/w can do multiple streams, but I believe we get better performance
> with 1 stream handling one large file, you could have 4 streams with 4
> files in flight, but the disk thrashes more... postgres is not be
> writing the file data, our agent reports back to postgres stats on the
> transfer rate being achieved ... postgres transactions is not the issue.
> The client and server are written in C and use UDP (with our own error
> correction) to achieve high network throughput as opposed to TCP.
I hope you know what you are doing, there is a large list of tricks used
by modern high performance FTP and web servers to get maximum
performance from hardware and the operating system while minimizing CPU
usage - and most of them don't work with UDP.
Before you test with real hardware, try simply sending dummy data or
/dev/null data (i.e. not from disks, not from file systems) and see how
it goes.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: ivoras@freebsd.org
Subject: Re: How to achieve sustained disk performance of 1.25 GB write for 5 mins
In-Reply-To: <ic1ode$mt8$1@dough.gmane.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox