pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Ron Johnson <ron.l.johnson@cox.net>
To: Postgresql General <pgsql-general@postgresql.org>
Subject: Re: realtime data inserts
Date: 16 May 2003 17:29:05 -0500
Message-ID: <1053124145.11845.17.camel@haggis> (raw)
In-Reply-To: <1053117234.15138.23.camel@localhost.localdomain>
References: <b9j57m$236m$1@news.hub.org>
	<6754.1052582402@sss.pgh.pa.us>
	<1052591490.26574.161.camel@haggis>
	<10704.1052621185@sss.pgh.pa.us>
	<1052672090.995.13.camel@haggis>
	<m3llxdju7w.fsf@varsoon.wireboard.com>
	<3EC549BE.5040403@gmx.de>
	<1053117234.15138.23.camel@localhost.localdomain>

On Fri, 2003-05-16 at 15:33, Ericson Smith wrote:
> You probably want to have a process that constantly stores the data in a
> text file. Every "n" minutes, you will cause the logger to rotate the
> text file, then process that batch. 

Does the logger spawn the DB writer?

> Over here, we are able to dump around 5,000 records per second in one of
> our tables using that methodology.
> 
> - Ericson Smith
> eric@did-it.com
> 
> On Fri, 2003-05-16 at 16:27, alex b. wrote:
> > Doug McNaught wrote:
> > > Ron Johnson <ron.l.johnson@cox.net> writes:
> > > 
> > > 
> > >>On Sat, 2003-05-10 at 21:46, Tom Lane wrote:
> > >>
> > >>>Ron Johnson <ron.l.johnson@cox.net> writes:
> > >>>
> > >>>>On Sat, 2003-05-10 at 11:00, Tom Lane wrote:
> > >>>>
> > >>>>>Have you thought about using COPY?
> > >>>
> > >>>>Generate a temporary file, and then system("COPY /tmp/foobar ...") ?
> > >>>
> > >>>No, copy from stdin.  No need for a temp file.
> > >>
> > >>But wouldn't that only work if the input stream is acceptable to
> > >>COPY ?
> > > 
> > > 
> > > Yes, but you could always pipe it through a script or C program to
> > > make it so...
> > 
> > lets say I have an about 1kb/s continuus datastream comming in for many 
> > hours and I'd like to store this data in my db using COPY table FROM stdin.
> > 
> > At what time should I COMMIT or close the stream to feed the database 
> > and COPY FROM again?
> > 
> > 
> > 
> > 
> > ---------------------------(end of broadcast)---------------------------
> > TIP 5: Have you checked our extensive FAQ?
> > 
> > http://www.postgresql.org/docs/faqs/FAQ.html
-- 
+---------------------------------------------------------------+
| Ron Johnson, Jr.        mailto:ron.l.johnson@cox.net          |
| Jefferson, LA  USA      http://members.cox.net/ron.l.johnson  |
|                                                               |
| The purpose of the military isn't to pay your college tuition |
| or give you a little extra income; it's to "kill people and   |
| break things".  Surprisingly, not everyone understands that.  |
+---------------------------------------------------------------+




view thread (13+ messages)

Message-ID: <1053124145.11845.17.camel@haggis>
Permalink:  ../1053124145.11845.17.camel@haggis/
Also on:    postgresql.org/message-id/1053124145.11845.17.camel@haggis

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: ron.l.johnson@cox.net, pgsql-general@postgresql.org
  Subject: Re: realtime data inserts
  In-Reply-To: <1053124145.11845.17.camel@haggis>
  References: <b9j57m$236m$1@news.hub.org> <6754.1052582402@sss.pgh.pa.us> <1052591490.26574.161.camel@haggis> <10704.1052621185@sss.pgh.pa.us> <1052672090.995.13.camel@haggis> <m3llxdju7w.fsf@varsoon.wireboard.com> <3EC549BE.5040403@gmx.de> <1053117234.15138.23.camel@localhost.localdomain> <1053124145.11845.17.camel@haggis>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox