From aa@devshock.com Sat Sep 10 17:03:07 2011 Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id E9ACAB5DC1E for ; Sat, 10 Sep 2011 14:03:07 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 87849-10 for ; Sat, 10 Sep 2011 17:03:01 +0000 (UTC) X-Greylist: delayed 00:07:40.793275 by SQLgrey-1.7.6 Received: from mail-yi0-f46.google.com (mail-yi0-f46.google.com [209.85.218.46]) by mail.postgresql.org (Postfix) with ESMTP id 7305AB5DC19 for ; Sat, 10 Sep 2011 14:03:01 -0300 (ADT) Received: by yie30 with SMTP id 30so1683310yie.19 for ; Sat, 10 Sep 2011 10:03:01 -0700 (PDT) Received: by 10.150.8.11 with SMTP id 11mr475198ybh.215.1315673720254; Sat, 10 Sep 2011 09:55:20 -0700 (PDT) Received: from AnibalVAIO (host-1-30.b9.cvc.com.py [186.2.200.30]) by mx.google.com with ESMTPS id u31sm273410ybu.24.2011.09.10.09.55.17 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 10 Sep 2011 09:55:18 -0700 (PDT) From: "Anibal David Acosta" To: Subject: should i expected performance degradation over time Date: Sat, 10 Sep 2011 12:55:46 -0400 Message-ID: <000001cc6fda$7e55fd20$7b01f760$@devshock.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0001_01CC6FB8.F74595A0" X-Mailer: Microsoft Outlook 14.0 Thread-Index: Acxv2PQmuYgvE3ubQOC0iy0cyA0eiA== Content-Language: es-py X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.584 tagged_above=-10 required=5 tests=BAYES_00=-1.9, HTML_MESSAGE=0.001, SARE_MILLIONSOF=0.315 X-Spam-Level: X-Archive-Number: 201109/102 X-Sequence-Number: 44878 This is a multipart message in MIME format. ------=_NextPart_000_0001_01CC6FB8.F74595A0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sometimes I read that postgres performance is degraded over the time and something people talk about backup and restore database solve the problem. It is really true? I have postgres 9.0 on a windows machine with The autovacuum is ON I have some configuration tables And a couple of transactional table. Transactional table has about 4 millions of rows inserted per day. In the midnight all rows are moved to a historical table and in the historical table rows are about 2 months, any transaction older than 2 months are deleted daily. So, my question is, if Should I expect same performance over time (example: after 1 year) or should I expect a degradation and must implements come technics like backup restore every certain time? Thanks!! ------=_NextPart_000_0001_01CC6FB8.F74595A0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Sometimes = I read that postgres performance is degraded over the time and something = people talk about backup and restore database solve the = problem.

 

It is really true?

 

I have = postgres 9.0 on a windows machine with The autovacuum is = ON

 

I have some configuration tables

And a couple of transactional table.

Transactional table has about 4 millions of rows = inserted per day.

In the midnight all = rows are moved to a historical table and in the historical table rows = are about 2 months, any transaction older than 2 months are deleted = daily.

 

 

So, my = question is, if Should I expect same performance over time (example: = after 1 year) or should I expect a degradation and must implements come = technics like backup restore every certain time?

 

Thanks!!

 

 

 

 

 

 

------=_NextPart_000_0001_01CC6FB8.F74595A0-- From andy@squeakycode.net Sat Sep 10 17:20:50 2011 Received: from maia.hub.org (maia-2.hub.org [200.46.204.251]) by mail.postgresql.org (Postfix) with ESMTP id 3A75EB5DC6A for ; Sat, 10 Sep 2011 14:20:50 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.251]) (amavisd-maia, port 10024) with ESMTP id 31333-04 for ; Sat, 10 Sep 2011 17:20:44 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from marajade.localdomain (97-64-237-59.client.mchsi.com [97.64.237.59]) by mail.postgresql.org (Postfix) with ESMTP id E8404B5DC6D for ; Sat, 10 Sep 2011 14:20:43 -0300 (ADT) Received: from [192.168.1.116] (173-31-201-111.client.mchsi.com [173.31.201.111]) (Authenticated sender: andy) by marajade.localdomain (Postfix) with ESMTPA id 68F9748698; Sat, 10 Sep 2011 12:20:42 -0500 (CDT) Message-ID: <4E6B9C69.7000205@squeakycode.net> Date: Sat, 10 Sep 2011 12:20:41 -0500 From: Andy Colson User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.21) Gecko/20110903 Lightning/1.0b2 Thunderbird/3.1.13 MIME-Version: 1.0 To: Anibal David Acosta CC: pgsql-performance@postgresql.org Subject: Re: should i expected performance degradation over time References: <000001cc6fda$7e55fd20$7b01f760$@devshock.com> In-Reply-To: <000001cc6fda$7e55fd20$7b01f760$@devshock.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-0.603 tagged_above=-10 required=5 tests=BAYES_00=-1.9, RDNS_DYNAMIC=0.982, SARE_MILLIONSOF=0.315 X-Spam-Level: X-Archive-Number: 201109/103 X-Sequence-Number: 44879 On 09/10/2011 11:55 AM, Anibal David Acosta wrote: > Sometimes I read that postgres performance is degraded over the time and something people talk about backup and restore database solve the problem. > > It is really true? > > I have postgres 9.0 on a windows machine with The autovacuum is ON > > I have some configuration tables > > And a couple of transactional table. > > Transactional table has about 4 millions of rows inserted per day. > > In the midnight all rows are moved to a historical table and in the historical table rows are about 2 months, any transaction older than 2 months are deleted daily. > > So, my question is, if Should I expect same performance over time (example: after 1 year) or should I expect a degradation and must implements come technics like backup restore every certain time? > > Thanks!! > Yes. And no. Things have changed over that last few versions. In older version of PG I recall hearing about table bloat problems that were really bad, and backup/restore would fix it. (Vacuum full would probably also have fixed it). "Vacuum full", in older versions was a last chance, bring a gun to a knife fight, nothing else has worked, fix table bloat solution. Its not dis-similar from backup/restore. In newer versions of PG, autovacuum, vacuum and vacuum full are all much nicer and work better. I really doubt you'll need to resort to backup/restore to fix problems. Just remember: the harder you hit a table, the less chance autovacuum will have to clean it up. So you might need manual vacuum. autovacuum will cancel itself if the table is getting hit, where-as manual vacuum wont. Keeping on top of vacuum will keep your tables slim and trim. If things get out of hand, they'll balloon into problems. Vacuum full at that point should clean it up. But, if you ignore the problem for two years, and have super really bad table bloat, well, maybe backup/restore is best. -Andy From scott.marlowe@gmail.com Sat Sep 10 18:30:29 2011 Received: from maia.hub.org (maia-5.hub.org [200.46.204.29]) by mail.postgresql.org (Postfix) with ESMTP id 5E468B5DC48 for ; Sat, 10 Sep 2011 15:30:29 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024) with ESMTP id 55204-05 for ; Sat, 10 Sep 2011 18:30:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-bw0-f46.google.com (mail-bw0-f46.google.com [209.85.214.46]) by mail.postgresql.org (Postfix) with ESMTP id A6F08B5DC44 for ; Sat, 10 Sep 2011 15:30:22 -0300 (ADT) Received: by bkbzt4 with SMTP id zt4so1262324bkb.19 for ; Sat, 10 Sep 2011 11:30:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=aiU7znthH+1f1AELqiWfta+n09FIwinYxKIsjYfcp7Y=; b=hHU9oCkf20nlig+/WmcDIx8/iocbGXruUaw/ft01/B7hcApfbBC2ynnocouCojJUbx u8IWOocBWDsA2uVAuWXQE91BagC0r4+JEgi/2vfKFokIiFdVHW6WSVyXIGVsiOiRFz6n Ei/9dhYW31eFRLTPZ4NnEZNQ4FQdJpp/EOCfI= MIME-Version: 1.0 Received: by 10.204.133.2 with SMTP id d2mr2063687bkt.165.1315679420710; Sat, 10 Sep 2011 11:30:20 -0700 (PDT) Received: by 10.205.81.206 with HTTP; Sat, 10 Sep 2011 11:30:20 -0700 (PDT) In-Reply-To: <000001cc6fda$7e55fd20$7b01f760$@devshock.com> References: <000001cc6fda$7e55fd20$7b01f760$@devshock.com> Date: Sat, 10 Sep 2011 12:30:20 -0600 Message-ID: Subject: Re: should i expected performance degradation over time From: Scott Marlowe To: Anibal David Acosta Cc: pgsql-performance@postgresql.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.584 tagged_above=-10 required=5 tests=BAYES_00=-1.9, FREEMAIL_FROM=0.001, SARE_MILLIONSOF=0.315 X-Spam-Level: X-Archive-Number: 201109/104 X-Sequence-Number: 44880 On Sat, Sep 10, 2011 at 10:55 AM, Anibal David Acosta wrote: > Sometimes I read that postgres performance is degraded over the time and > something people talk about backup and restore database solve the problem. > > It is really true? Yes and no. If you let things get out of hand, a backup and restore may be your best choice. > I have postgres 9.0 on a windows machine with The autovacuum is ON Good start > Transactional table has about 4 millions of rows inserted per day. > > In the midnight all rows are moved to a historical table and in the > historical table rows are about 2 months, any transaction older than 2 > months are deleted daily. You should look into table partitioning then. but as long as vacuum keeps up you're probably still ok. Look at the check_postgresql.pl script by the same guy who wrote Bucardo. It'll keep you advised of how much bloat your tables have. > So, my question is, if Should I expect same performance over time (example: > after 1 year) or should I expect a degradation and must implements come > technics like backup restore every certain time? If you maintain your db properly, performance should stay good. If you ignore bloat issues you might have some issues. From aa@devshock.com Tue Oct 11 18:28:34 2011 Received: from maia.hub.org (maia-5.hub.org [200.46.204.29]) by mail.postgresql.org (Postfix) with ESMTP id 04D68B5DBD8 for ; Tue, 11 Oct 2011 15:28:34 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024) with ESMTP id 80150-09 for ; Tue, 11 Oct 2011 18:28:27 +0000 (UTC) X-Greylist: delayed 00:08:05.972657 by SQLgrey-1.8.0-rc2 Received: from mail-yw0-f46.google.com (mail-yw0-f46.google.com [209.85.213.46]) by mail.postgresql.org (Postfix) with ESMTP id 31186B5D803 for ; Tue, 11 Oct 2011 15:28:27 -0300 (ADT) Received: by ywb5 with SMTP id 5so6859065ywb.19 for ; Tue, 11 Oct 2011 11:28:26 -0700 (PDT) Received: by 10.236.145.74 with SMTP id o50mr31908357yhj.36.1318357220395; Tue, 11 Oct 2011 11:20:20 -0700 (PDT) Received: from AnibalVAIO ([190.128.222.90]) by mx.google.com with ESMTPS id r30sm31936474yhj.20.2011.10.11.11.20.14 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 11 Oct 2011 11:20:15 -0700 (PDT) From: "Anibal David Acosta" To: "'Scott Marlowe'" Cc: References: <000001cc6fda$7e55fd20$7b01f760$@devshock.com> In-Reply-To: Subject: Re: should i expected performance degradation over time Date: Tue, 11 Oct 2011 15:20:12 -0300 Message-ID: <003001cc8842$6ced6ca0$46c845e0$@devshock.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQHsI1FKu/mxvl84UJ0Y4KoiPGDhvALrDJ2tlSDlLAA= Content-Language: es-py X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.585 tagged_above=-10 required=5 tests=BAYES_00=-1.9, SARE_MILLIONSOF=0.315 X-Spam-Level: X-Archive-Number: 201110/108 X-Sequence-Number: 45330 Do you know if check_postgresql.pl can run on windows (with perl = installed)? Because our postgres installation is running on a Windows 2008 R2 server = but can't find any tool like this for windows :( Thanks! -----Mensaje original----- De: Scott Marlowe [mailto:scott.marlowe@gmail.com]=20 Enviado el: s=E1bado, 10 de septiembre de 2011 02:30 p.m. Para: Anibal David Acosta CC: pgsql-performance@postgresql.org Asunto: Re: [PERFORM] should i expected performance degradation over = time On Sat, Sep 10, 2011 at 10:55 AM, Anibal David Acosta wrote: > Sometimes I read that postgres performance is degraded over the time=20 > and something people talk about backup and restore database solve the problem. > > It is really true? Yes and no. If you let things get out of hand, a backup and restore may = be your best choice. > I have postgres 9.0 on a windows machine with The autovacuum is ON Good start > Transactional table has about 4 millions of rows inserted per day. > > In the midnight all rows are moved to a historical table and in the=20 > historical table rows are about 2 months, any transaction older than 2 = > months are deleted daily. You should look into table partitioning then. but as long as vacuum = keeps up you're probably still ok. Look at the check_postgresql.pl script by = the same guy who wrote Bucardo. It'll keep you advised of how much bloat = your tables have. > So, my question is, if Should I expect same performance over time (example: > after 1 year) or should I expect a degradation and must implements=20 > come technics like backup restore every certain time? If you maintain your db properly, performance should stay good. If you ignore bloat issues you might have some issues. From robertmhaas@gmail.com Fri Oct 28 19:03:21 2011 Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id A52ECB5DC4B for ; Fri, 28 Oct 2011 16:03:21 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 75792-01 for ; Fri, 28 Oct 2011 19:03:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0-rc2 Received: from mail-qw0-f46.google.com (mail-qw0-f46.google.com [209.85.216.46]) by mail.postgresql.org (Postfix) with ESMTP id E4BFAB5DC51 for ; Fri, 28 Oct 2011 16:03:14 -0300 (ADT) Received: by qabj40 with SMTP id j40so3955630qab.19 for ; Fri, 28 Oct 2011 12:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=q6NFSzW8VOrjuQqpS/A7NXSL9cfEHEMAcZDyzZAdP+U=; b=fOgUEdJ7yNrWhy+iBtWMqHo+Kpb5XkzOXXEmGdYTJ1A70PJbuTXOMx7elV+vZ/p4mm r8Gc8puWECK1YVmQ0NZ1lqGIHbryuA5RbPBhTm8M76xg25PCBE+z0KeZq8F5NQYxH0Lb mcAOsKQQ2WprONJnhkHPCWO3z4V3lO45Zy3XM= MIME-Version: 1.0 Received: by 10.224.185.16 with SMTP id cm16mr4063573qab.28.1319828594632; Fri, 28 Oct 2011 12:03:14 -0700 (PDT) Received: by 10.224.45.133 with HTTP; Fri, 28 Oct 2011 12:03:14 -0700 (PDT) In-Reply-To: <003001cc8842$6ced6ca0$46c845e0$@devshock.com> References: <000001cc6fda$7e55fd20$7b01f760$@devshock.com> <003001cc8842$6ced6ca0$46c845e0$@devshock.com> Date: Fri, 28 Oct 2011 15:03:14 -0400 Message-ID: Subject: Re: should i expected performance degradation over time From: Robert Haas To: Anibal David Acosta Cc: Scott Marlowe , pgsql-performance@postgresql.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.899 tagged_above=-10 required=5 tests=BAYES_00=-1.9, FREEMAIL_FROM=0.001 X-Spam-Level: X-Archive-Number: 201110/261 X-Sequence-Number: 45483 On Tue, Oct 11, 2011 at 2:20 PM, Anibal David Acosta wrote: > Do you know if check_postgresql.pl can run on windows (with perl installed)? > > Because our postgres installation is running on a Windows 2008 R2 server but > can't find any tool like this for windows :( > > Thanks! It's written in Perl, so I would think you could get it to work. But if not, you can always extract the big ol' query that it runs from the script and run it some other way. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company