Received: from localhost (unknown [200.46.204.184]) by postgresql.org (Postfix) with ESMTP id DEC862E01A6 for ; Fri, 7 Mar 2008 09:00:03 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 58865-04 for ; Fri, 7 Mar 2008 08:59:44 -0400 (AST) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.5 Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.225]) by postgresql.org (Postfix) with ESMTP id E0B422E018E for ; Fri, 7 Mar 2008 08:59:50 -0400 (AST) Received: by wx-out-0506.google.com with SMTP id r21so598013wxc.28 for ; Fri, 07 Mar 2008 04:59:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; bh=uOS+EjV3Iywq03bDwSDkp65V9yOiyfTZGNveEA3iEA8=; b=QQ86DDXHRdcE/lkf1sSc7K6RDKMPc3bXSxmqihYrdGytS22LND7kr3EYgb254JdxAcd48dOeXJh5cxIsFWCEgCpHkfc63mHRPU7zMtCKBjQSHgiX70fJzH/5aGQd4+ABoG2qkopBwnC3Jpwpyv22gtlHa7Wq36OTnP37iBRyUvM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=yEhCybxqKLIv0YNbEk9r0PJo+kVO1z80pKLaMbSdTdf+2bSdS/lgnrxdEXHn0whRYfniNgVVf617IT2UWWIv3jJuY++vbCS8rhciHeOJMf0Su1+grqygYiTdVLgxUjJPYdeGuykcwEN76QyxrR1oPNpCoeC/lxKa+MEryn6FvqY= Received: by 10.65.180.9 with SMTP id h9mr884216qbp.31.1204894786815; Fri, 07 Mar 2008 04:59:46 -0800 (PST) Received: by 10.64.3.7 with HTTP; Fri, 7 Mar 2008 04:59:46 -0800 (PST) Message-ID: <3556c08c0803070459i8598423t64e0704e48af533@mail.gmail.com> Date: Fri, 7 Mar 2008 13:59:46 +0100 From: "Pavel Rotek" To: pgsql-performance@postgresql.org Subject: Re: Toast space grows In-Reply-To: <20080307065314.1248665b.wmoran@collaborativefusion.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_25608_6550223.1204894786751" References: <3556c08c0803070035p6d8b4aa8m6318f54728f61022@mail.gmail.com> <47D10722.4080508@archonet.com> <3556c08c0803070129y6ffabf32i1c8bae445ec460e0@mail.gmail.com> <20080307065314.1248665b.wmoran@collaborativefusion.com> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.001 tagged_above=0 required=5 tests=HTML_MESSAGE=0.001 X-Spam-Level: X-Archive-Number: 200803/157 X-Sequence-Number: 29111 ------=_Part_25608_6550223.1204894786751 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline 2008/3/7, Bill Moran : > > In response to "Pavel Rotek" : > > > > > There are inserts and few updates (but what do you mean with update?? > > > He means adding or changing data in the table. I understand, but i don't have deep understanding of mechanism, that affects toast space. I don't know if toast space is affected only with commited transactions or also with uncommitted transactions. This is the reason why i asked. > committed update??, because there are many updates of the log attribute in > > trasaction, we do periodical flush during transaction) > > > Are you saying you have long-running transactions? How long does a > single transaction take? > > Transactions prevent vacuum from being able to clean up. Long running > transactions tend to render vacuum ineffective. No i do not mean long running transactions... Update of log entry (update of row in dataaction) is performed in series of short transactions, but during short transaction there is a lot of change log value, flush, change log value ,flush ..... change log value, flush actions (flush means perform flush operation via JDBC driver). I'm not sure if this flush affects toast space... Maybe this is the reason. >, sum takes > > approximately 1,2G, and i mean vacuum full (but there is no lock on the > > table when running vacuum full). I haven't try plain vacuum. > > > Don't do vacuum full on this table. Do frequent vacuums. The table will > bloat some, but not 10x the required size, once you find a reasonable > frequency for vacuums. You might find it practical to manually vacuum > this table from your application after insert and update operations. I perform autovacuum daily. > > > 2008/3/7, Richard Huxton : > > > > > > Pavel Rotek wrote: > > > > Hello, > > > > > > > > i have problem with following table... > > > > > > > > create table dataaction ( > > > > id INT4 not null, > > > > log text, > > > > primary key (id) > > > > ); > > > > > > > > It is the table for storing results of long running jobs. The log > > > attribute > > > > takes approximately 5MB for one row (there is about 300 rows). My > > > problem > > > > is, that table dataaction takes after restoring about 1,5G, but in > few > > > days > > > > grows to 79G(Toast space)... > > > > > > > > > 1. What is happening with this table - just inserts, lots of updates? > > > > > > 2. What does SELECT sum(length(log)) FROM dataaction; show? > > > > > > > > > > Vacuum on the table doesn't finish. > > > > > > > > > A plain vacuum doesn't finish, or vacuum full doesn't finish? > > > > > > > > > -- > > > Richard Huxton > > > Archonet Ltd > > > > > > > > > -- > Bill Moran > Collaborative Fusion Inc. > http://people.collaborativefusion.com/~wmoran/ > > wmoran@collaborativefusion.com > Phone: 412-422-3463x4023 > > **************************************************************** > IMPORTANT: This message contains confidential information and is > intended only for the individual named. If the reader of this > message is not an intended recipient (or the individual > responsible for the delivery of this message to an intended > recipient), please be advised that any re-use, dissemination, > distribution or copying of this message is prohibited. Please > notify the sender immediately by e-mail if you have received > this e-mail by mistake and delete this e-mail from your system. > E-mail transmission cannot be guaranteed to be secure or > error-free as information could be intercepted, corrupted, lost, > destroyed, arrive late or incomplete, or contain viruses. The > sender therefore does not accept liability for any errors or > omissions in the contents of this message, which arise as a > result of e-mail transmission. > **************************************************************** > ------=_Part_25608_6550223.1204894786751 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline

2008/3/7, Bill Moran <wmoran@collaborativefusion.com>:
In response to "Pavel Rotek" <pavel.rotek@gmail.com>:

>
> There are inserts and few updates (but what do you mean with update??


He means adding or changing data in the table.

I understand, but i don't have deep understanding of mechanism, that affects toast space. I don't know if toast space is affected only with commited transactions or also with uncommitted transactions. This is the reason why i asked.

> committed update??, because there are many updates of the log attribute in
> trasaction, we do periodical flush during transaction)


Are you saying you have long-running transactions?  How long does a
single transaction take?

Transactions prevent vacuum from being able to clean up.  Long running
transactions tend to render vacuum ineffective.

No i do not mean long running transactions... Update of log entry (update of row in dataaction) is performed in series of short transactions, but during short transaction there is a lot of change log value, flush, change log value ,flush ..... change log value, flush actions (flush means perform flush operation via JDBC driver). I'm not sure if this flush affects toast space... Maybe this is the reason.

>, sum takes
> approximately 1,2G, and i mean vacuum full (but there is no lock on the
> table when running vacuum full). I haven't try plain vacuum.


Don't do vacuum full on this table.  Do frequent vacuums.  The table will
bloat some, but not 10x the required size, once you find a reasonable
frequency for vacuums.  You might find it practical to manually vacuum
this table from your application after insert and update operations.

I perform autovacuum daily.

>
> 2008/3/7, Richard Huxton <dev@archonet.com>:
> >
> > Pavel Rotek wrote:
> > > Hello,
> > >
> > > i have problem with following table...
> > >
> > > create table dataaction (
> > >    id INT4 not null,
> > >    log text,
> > >    primary key (id)
> > > );
> > >
> > > It is the table for storing results of long running jobs. The log
> > attribute
> > > takes approximately 5MB for one row (there is about 300 rows). My
> > problem
> > > is, that table dataaction takes after restoring about 1,5G, but in few
> > days
> > > grows to 79G(Toast space)...
> >
> >
> > 1. What is happening with this table - just inserts, lots of updates?
> >
> > 2. What does SELECT sum(length(log)) FROM dataaction; show?
> >
> >
> >   > Vacuum on the table doesn't finish.
> >
> >
> > A plain vacuum doesn't finish, or vacuum full doesn't finish?
> >
> >
> > --
> >    Richard Huxton
> >    Archonet Ltd
> >
>



--
Bill Moran
Collaborative Fusion Inc.
http://people.collaborativefusion.com/~wmoran/

wmoran@collaborativefusion.com
Phone: 412-422-3463x4023

****************************************************************
IMPORTANT: This message contains confidential information and is
intended only for the individual named. If the reader of this
message is not an intended recipient (or the individual
responsible for the delivery of this message to an intended
recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received
this e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The
sender therefore does not accept liability for any errors or
omissions in the contents of this message, which arise as a
result of e-mail transmission.
****************************************************************

------=_Part_25608_6550223.1204894786751--