DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	b=kBu+faRDY5vsT7YkQ45MeEw4yz99i+Bi5XrSmV5LXlJw36US2NNf9LJPH76hWcVH87
	dtUwOW4TF4qhVsMq6VnKVs62fUR7b/W8kHLB9IteszgaDkbCyd3E/pDkuFhOgJfzSTpg
	YDLp881Til148tvcDR/153VWkpRvfinj79rMQ=
MIME-Version: 1.0
In-Reply-To: <60B0F2124D07B942988329B5B7CA393D01E5B93FF7@mail2.FireEye.com>
References: <95652FF3B9EF3045AC4AE8FE5AF8671EED65CF6B@mail2.FireEye.com>
	<60B0F2124D07B942988329B5B7CA393D01E5B93FF7@mail2.FireEye.com>
Date: Tue, 3 Nov 2009 20:35:19 -0700
Message-ID: <dcc563d10911031935q5edf8961ye7812c161ea6e762@mail.gmail.com>
Subject: Re: High Frequency Inserts to Postgres Database vs Writing to a File
From: Scott Marlowe <scott.marlowe@gmail.com>
To: Jay Manni <JManni@fireeye.com>
Cc: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On Tue, Nov 3, 2009 at 8:12 PM, Jay Manni <JManni@fireeye.com> wrote:
> Hi:
>
>
>
> I have an application wherein a process needs to read data from a stream =
and
> store the records for further analysis and reporting. The data in the str=
eam
> is in the form of variable length records with clearly defined fields =96=
 so
> it can be stored in a database or in a file. The only caveat is that the
> rate of records coming in the stream could be several 1000 records a seco=
nd.
>
>
>
> The design choice I am faced with currently is whether to use a postgres
> database or a flat file for this purpose. My application already maintain=
s a

A common approach is to store them in flat files, then insert the flat
files at a later time so that if the db falls behind no data is lost.