X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 6D0A0534CB for ; Thu, 31 Mar 2005 03:12:01 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 36610-03 for ; Thu, 31 Mar 2005 02:11:57 +0000 (GMT) Received: from server07.icaen.uiowa.edu (server07.icaen.uiowa.edu [128.255.17.47]) by svr1.postgresql.org (Postfix) with ESMTP id 64DA55348D for ; Thu, 31 Mar 2005 03:11:57 +0100 (BST) Received: from server11.icaen.uiowa.edu (server11.icaen.uiowa.edu [128.255.17.51]) by server07.icaen.uiowa.edu (8.13.3/8.12.9) with ESMTP id j2V2C1qm015819 for ; (envelope-from ) Wed, 30 Mar 2005 20:12:01 -0600 (CST) Received: from [192.168.1.13] (12-215-118-172.client.mchsi.com [12.215.118.172]) (authenticated user=jfmeinel) by server11.icaen.uiowa.edu (8.13.2/smtp-serv-1.7) with ESMTP id j2V2Bx4l013986 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256) for ; (envelope-from ) Wed, 30 Mar 2005 20:12:00 -0600 (CST) Message-ID: <424B5C6F.8090609@arbash-meinel.com> Date: Wed, 30 Mar 2005 20:11:59 -0600 From: John Arbash Meinel User-Agent: Mozilla Thunderbird 1.0 (Macintosh/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: pgsql-performance@postgresql.org Subject: Re: fine tuning for logging server References: <1112233809.10313.31.camel@stantz.corp.sgi.com> <1112234574.12159.1.camel@stantz.corp.sgi.com> In-Reply-To: <1112234574.12159.1.camel@stantz.corp.sgi.com> X-Enigmail-Version: 0.89.6.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigFC3852802768A3D7D9EEC78C" X-Virus-Scanned: ClamAV version 0.83, clamav-milter version 0.83 on clamav.icaen.uiowa.edu X-Virus-Scanned: ClamAV version 0.83, clamav-milter version 0.83 on clamav.icaen.uiowa.edu X-Virus-Status: Clean X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests= X-Spam-Level: X-Archive-Number: 200503/669 X-Sequence-Number: 11374 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigFC3852802768A3D7D9EEC78C Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Florin Andrei wrote: >On Wed, 2005-03-30 at 17:50 -0800, Florin Andrei wrote: > > > >>Function: Essentially a logging server. There are two applications (like >>syslog) on the same box that are logging to pgsql, each one to its own >>database. There are a few tables in one DB, and exactly one table in the >>other. >>Most of the time, the apps are just doing mindless INSERTs to the DB. >>Every now and then, an admin performs some SELECTs via a PHP interface. >> >> > >For performance reasons, i was thinking to keep the tables append-only, >and simply rotate them out every so often (daily?) and delete those >tables that are too old. Is that a good idea? > > > If you aren't doing updates, then I'm pretty sure the data stays packed pretty well. I don't know that you need daily rotations, but you certainly could consider some sort of rotation schedule. The biggest performance improvement, though, is probably to group inserts into transactions. I had an application (in a different db, but it should be relevant), where using a transaction changed the time from 6min -> 6 sec. It was just thrashing on all the little inserts that it had to fsync to disk. How fast is fast? How many log messages are you expecting? 1/s 100/s 1000/s? I think the hardware should be capable of the 10-100 range if things are properly configured. Naturally that depends on all sorts of factors, but it should give you an idea. John =:-> --------------enigFC3852802768A3D7D9EEC78C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCS1xvJdeBCYSNAAMRAjWqAJ9vP92bvumQ3PhmzPGtPoQ37NmZTACgzuQ+ 4pzUmCtPvBWkgrGtz1dDYfo= =ArJ+ -----END PGP SIGNATURE----- --------------enigFC3852802768A3D7D9EEC78C--