pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
vacuumdb -a -f
9+ messages / 6 participants
[nested] [flat]

* vacuumdb -a -f
@ 2005-08-15 20:16  D Kavan <bitsandbytes88@hotmail.com>
  0 siblings, 1 reply; 9+ messages in thread

From: D Kavan @ 2005-08-15 20:16 UTC (permalink / raw)
  To: pgsql-admin

Hello,

Even though I run vacuumdb -a -f every night with no exceptions or problems, 
my database size remains 5.6 GB.  After I do a dump/restore, the new 
database size is 4.0 GB.  How could that be possible?   That's a significant 
amount of space.  If I can clean up my existing databases by 30%, I'd be in 
much better shape.

Has anyone else noticed this?

Thanks,
~DjK





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: vacuumdb -a -f
@ 2005-08-16 01:31  Tom Lane <tgl@sss.pgh.pa.us>
  parent: D Kavan <bitsandbytes88@hotmail.com>
  0 siblings, 0 replies; 9+ messages in thread

From: Tom Lane @ 2005-08-16 01:31 UTC (permalink / raw)
  To: D Kavan <bitsandbytes88@hotmail.com>; +Cc: pgsql-admin

"D Kavan" <bitsandbytes88@hotmail.com> writes:
> Even though I run vacuumdb -a -f every night with no exceptions or problems, 
> my database size remains 5.6 GB.  After I do a dump/restore, the new 
> database size is 4.0 GB.  How could that be possible?

The extra 1.6GB probably represents the amount of junk you generate in
one day.  So, forget the -f and instead do plain vacuums on a more
frequent basis.  Make sure your FSM settings are large enough, too.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: vacuumdb -a -f
@ 2005-08-17 15:24  D Kavan <bitsandbytes88@hotmail.com>
  0 siblings, 2 replies; 9+ messages in thread

From: D Kavan @ 2005-08-17 15:24 UTC (permalink / raw)
  To: pgsql-admin

Hi,

Thanks for the tips.

Unfortunatley for me,  even after started doing vacuumdb -a 3 times a day 
and increasing fsm dramatically , the size of the database won't go down 
even 1 MB.  It's stil at 5.6 GB, size after restore = 4 GB.   I even did a 
stop/start instead of a re-load to make sure the settings took affect. Would 
a reboot help?

max_fsm_pages = 16000001
max_fsm_relations = 1000000

shared_buffers = 65536
work_mem = 32768
maintenance work mem = 786432

checkpoint_segments = 18


##/etc/sysctl.conf

nel.shmall = 524288
#kernel.shmall = 2097152
#kernel.shmmax = 2147483648
#kernel.shmmax = 1073741824
kernel.shmmax = 6979321856
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
vm.overcommit_memory = 2


~DjK





>From: Tom Lane <tgl@sss.pgh.pa.us>
>To: "D Kavan" <bitsandbytes88@hotmail.com>
>CC: pgsql-admin@postgresql.org
>Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 -0400
>
>"D Kavan" <bitsandbytes88@hotmail.com> writes:
> > Even though I run vacuumdb -a -f every night with no exceptions or 
>problems,
> > my database size remains 5.6 GB.  After I do a dump/restore, the new
> > database size is 4.0 GB.  How could that be possible?
>
>The extra 1.6GB probably represents the amount of junk you generate in
>one day.  So, forget the -f and instead do plain vacuums on a more
>frequent basis.  Make sure your FSM settings are large enough, too.
>
>			regards, tom lane





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: vacuumdb -a -f
@ 2005-08-17 15:57  Guido Barosio <gbarosio@gmail.com>
  parent: D Kavan <bitsandbytes88@hotmail.com>
  1 sibling, 2 replies; 9+ messages in thread

From: Guido Barosio @ 2005-08-17 15:57 UTC (permalink / raw)
  To: D Kavan <bitsandbytes88@hotmail.com>; +Cc: pgsql-admin

How are you finding out the DB size?
 G.-

 On 8/17/05, D Kavan <bitsandbytes88@hotmail.com> wrote: 
> 
> Hi,
> 
> Thanks for the tips.
> 
> Unfortunatley for me, even after started doing vacuumdb -a 3 times a day
> and increasing fsm dramatically , the size of the database won't go down
> even 1 MB. It's stil at 5.6 GB, size after restore = 4 GB. I even did a
> stop/start instead of a re-load to make sure the settings took affect. 
> Would
> a reboot help?
> 
> max_fsm_pages = 16000001
> max_fsm_relations = 1000000
> 
> shared_buffers = 65536
> work_mem = 32768
> maintenance work mem = 786432
> 
> checkpoint_segments = 18
> 
> 
> ##/etc/sysctl.conf
> 
> nel.shmall = 524288
> #kernel.shmall = 2097152
> #kernel.shmmax = 2147483648
> #kernel.shmmax = 1073741824
> kernel.shmmax = 6979321856
> kernel.shmmni = 4096
> kernel.sem = 250 32000 100 128
> fs.file-max = 65536
> net.ipv4.ip_local_port_range = 1024 65000
> vm.overcommit_memory = 2
> 
> 
> ~DjK
> 
> 
> 
> 
> 
> >From: Tom Lane <tgl@sss.pgh.pa.us>
> >To: "D Kavan" <bitsandbytes88@hotmail.com>
> >CC: pgsql-admin@postgresql.org
> >Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 -0400
> >
> >"D Kavan" <bitsandbytes88@hotmail.com> writes:
> > > Even though I run vacuumdb -a -f every night with no exceptions or
> >problems,
> > > my database size remains 5.6 GB. After I do a dump/restore, the new
> > > database size is 4.0 GB. How could that be possible?
> >
> >The extra 1.6GB probably represents the amount of junk you generate in
> >one day. So, forget the -f and instead do plain vacuums on a more
> >frequent basis. Make sure your FSM settings are large enough, too.
> >
> > regards, tom lane
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
> 



-- 
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.

^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: vacuumdb -a -f
@ 2005-08-17 16:15  Joshua D. Drake <jd@commandprompt.com>
  parent: D Kavan <bitsandbytes88@hotmail.com>
  1 sibling, 0 replies; 9+ messages in thread

From: Joshua D. Drake @ 2005-08-17 16:15 UTC (permalink / raw)
  To: D Kavan <bitsandbytes88@hotmail.com>; +Cc: pgsql-admin

D Kavan wrote:
> Hi,
> 
> Thanks for the tips.
> 
> Unfortunatley for me,  even after started doing vacuumdb -a 3 times a 
> day and increasing fsm dramatically , the size of the database won't go 
> down even 1 MB.  It's stil at 5.6 GB, size after restore = 4 GB.   I 
> even did a stop/start instead of a re-load to make sure the settings 
> took affect. Would a reboot help?

A vacuum full will decrease the size but I am not sure why the size is a 
problem. I normal vacuum will not remove rows, it only make them 
reusable. So at any given point your database is going to grow after use.

I didn't see this whole thread but is there something specific that 
seems to  be the issue beyond size?

Sincerely,

Joshua D. Drake




> 
> max_fsm_pages = 16000001
> max_fsm_relations = 1000000
> 
> shared_buffers = 65536
> work_mem = 32768
> maintenance work mem = 786432
> 
> checkpoint_segments = 18
> 
> 
> ##/etc/sysctl.conf
> 
> nel.shmall = 524288
> #kernel.shmall = 2097152
> #kernel.shmmax = 2147483648
> #kernel.shmmax = 1073741824
> kernel.shmmax = 6979321856
> kernel.shmmni = 4096
> kernel.sem = 250 32000 100 128
> fs.file-max = 65536
> net.ipv4.ip_local_port_range = 1024 65000
> vm.overcommit_memory = 2
> 
> 
> ~DjK
> 
> 
> 
> 
> 
>> From: Tom Lane <tgl@sss.pgh.pa.us>
>> To: "D Kavan" <bitsandbytes88@hotmail.com>
>> CC: pgsql-admin@postgresql.org
>> Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 -0400
>>
>> "D Kavan" <bitsandbytes88@hotmail.com> writes:
>> > Even though I run vacuumdb -a -f every night with no exceptions or 
>> problems,
>> > my database size remains 5.6 GB.  After I do a dump/restore, the new
>> > database size is 4.0 GB.  How could that be possible?
>>
>> The extra 1.6GB probably represents the amount of junk you generate in
>> one day.  So, forget the -f and instead do plain vacuums on a more
>> frequent basis.  Make sure your FSM settings are large enough, too.
>>
>>             regards, tom lane
> 
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend


-- 
Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240
PostgreSQL Replication, Consulting, Custom Programming, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: vacuumdb -a -f
@ 2005-08-17 16:57  Chris Hoover <revoohc@gmail.com>
  parent: Guido Barosio <gbarosio@gmail.com>
  1 sibling, 1 reply; 9+ messages in thread

From: Chris Hoover @ 2005-08-17 16:57 UTC (permalink / raw)
  To: Guido Barosio <gbarosio@gmail.com>; +Cc: D Kavan <bitsandbytes88@hotmail.com>; pgsql-admin

You might want to try doing a reindexdb -a -e (reindexdb is in the
contrib directory of your pg source).  The first time I ran this, I
gained back a significant amount of space.

I now run a vacuumdb -v -f -a and then a reindexdb -a -e every weekend
to have PostgreSQL give back as much space as it can.  I have ended up
doing this do to space and i/o constraints.

Give the reindexdb script a try and see if you don't get your space back.

HTH,

Chris

On 8/17/05, Guido Barosio <gbarosio@gmail.com> wrote:
> How are you finding out the DB size? 
>   
> G.-
> 
>  
>  
> On 8/17/05, D Kavan <bitsandbytes88@hotmail.com> wrote: 
> > Hi,
> > 
> > Thanks for the tips.
> > 
> > Unfortunatley for me,  even after started doing vacuumdb -a 3 times a day 
> > and increasing fsm dramatically , the size of the database won't go down
> > even 1 MB.  It's stil at 5.6 GB, size after restore = 4 GB.   I even did a
> > stop/start instead of a re-load to make sure the settings took affect.
> Would 
> > a reboot help?
> > 
> > max_fsm_pages = 16000001
> > max_fsm_relations = 1000000
> > 
> > shared_buffers = 65536
> > work_mem = 32768
> > maintenance work mem = 786432
> > 
> > checkpoint_segments = 18
> > 
> > 
> > ##/etc/sysctl.conf 
> > 
> > nel.shmall = 524288
> > #kernel.shmall = 2097152
> > #kernel.shmmax = 2147483648
> > #kernel.shmmax = 1073741824
> > kernel.shmmax = 6979321856
> > kernel.shmmni = 4096
> > kernel.sem = 250 32000 100 128
> > fs.file-max = 65536 
> > net.ipv4.ip_local_port_range = 1024 65000
> > vm.overcommit_memory = 2
> > 
> > 
> > ~DjK
> > 
> > 
> > 
> > 
> > 
> > >From: Tom Lane <tgl@sss.pgh.pa.us>
> > >To: "D Kavan" < bitsandbytes88@hotmail.com>
> > >CC: pgsql-admin@postgresql.org
> > >Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 -0400
> > >
> > >"D Kavan" <bitsandbytes88@hotmail.com> writes:
> > > > Even though I run vacuumdb -a -f every night with no exceptions or
> > >problems, 
> > > > my database size remains 5.6 GB.  After I do a dump/restore, the new
> > > > database size is 4.0 GB.  How could that be possible?
> > >
> > >The extra 1.6GB probably represents the amount of junk you generate in 
> > >one day.  So, forget the -f and instead do plain vacuums on a more
> > >frequent basis.  Make sure your FSM settings are large enough, too.
> > >
> > >                       regards, tom lane
> > 
> > 
> > 
> > ---------------------------(end of
> broadcast)---------------------------
> > TIP 6: explain analyze is your friend
> > 
> 
> 
> 
> -- 
> "Adopting the position that you are smarter than an automatic 
> optimization algorithm is generally a good way to achieve less
> performance, not more" - Tom Lane.



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: vacuumdb -a -f
@ 2005-08-17 17:41  D Kavan <bitsandbytes88@hotmail.com>
  parent: Chris Hoover <revoohc@gmail.com>
  0 siblings, 0 replies; 9+ messages in thread

From: D Kavan @ 2005-08-17 17:41 UTC (permalink / raw)
  To: pgsql-admin



CASE CLOSED!

Thanks for the tip on the reindex script.  That did the trick.  It was one 
table in particuliar.

~DjK
















>From: Chris Hoover <revoohc@gmail.com>
>To: Guido Barosio <gbarosio@gmail.com>
>CC: D Kavan <bitsandbytes88@hotmail.com>, pgsql-admin@postgresql.org
>Subject: Re: [ADMIN] vacuumdb -a -f
>Date: Wed, 17 Aug 2005 12:57:16 -0400
>
>You might want to try doing a reindexdb -a -e (reindexdb is in the
>contrib directory of your pg source).  The first time I ran this, I
>gained back a significant amount of space.
>
>I now run a vacuumdb -v -f -a and then a reindexdb -a -e every weekend
>to have PostgreSQL give back as much space as it can.  I have ended up
>doing this do to space and i/o constraints.
>
>Give the reindexdb script a try and see if you don't get your space back.
>
>HTH,
>
>Chris
>
>On 8/17/05, Guido Barosio <gbarosio@gmail.com> wrote:
> > How are you finding out the DB size?
> >
> > G.-
> >
> >
> >
> > On 8/17/05, D Kavan <bitsandbytes88@hotmail.com> wrote:
> > > Hi,
> > >
> > > Thanks for the tips.
> > >
> > > Unfortunatley for me,  even after started doing vacuumdb -a 3 times a 
>day
> > > and increasing fsm dramatically , the size of the database won't go 
>down
> > > even 1 MB.  It's stil at 5.6 GB, size after restore = 4 GB.   I even 
>did a
> > > stop/start instead of a re-load to make sure the settings took affect.
> > Would
> > > a reboot help?
> > >
> > > max_fsm_pages = 16000001
> > > max_fsm_relations = 1000000
> > >
> > > shared_buffers = 65536
> > > work_mem = 32768
> > > maintenance work mem = 786432
> > >
> > > checkpoint_segments = 18
> > >
> > >
> > > ##/etc/sysctl.conf
> > >
> > > nel.shmall = 524288
> > > #kernel.shmall = 2097152
> > > #kernel.shmmax = 2147483648
> > > #kernel.shmmax = 1073741824
> > > kernel.shmmax = 6979321856
> > > kernel.shmmni = 4096
> > > kernel.sem = 250 32000 100 128
> > > fs.file-max = 65536
> > > net.ipv4.ip_local_port_range = 1024 65000
> > > vm.overcommit_memory = 2
> > >
> > >
> > > ~DjK
> > >
> > >
> > >
> > >
> > >
> > > >From: Tom Lane <tgl@sss.pgh.pa.us>
> > > >To: "D Kavan" < bitsandbytes88@hotmail.com>
> > > >CC: pgsql-admin@postgresql.org
> > > >Subject: Re: [ADMIN] vacuumdb -a -f Date: Mon, 15 Aug 2005 21:31:01 
>-0400
> > > >
> > > >"D Kavan" <bitsandbytes88@hotmail.com> writes:
> > > > > Even though I run vacuumdb -a -f every night with no exceptions or
> > > >problems,
> > > > > my database size remains 5.6 GB.  After I do a dump/restore, the 
>new
> > > > > database size is 4.0 GB.  How could that be possible?
> > > >
> > > >The extra 1.6GB probably represents the amount of junk you generate 
>in
> > > >one day.  So, forget the -f and instead do plain vacuums on a more
> > > >frequent basis.  Make sure your FSM settings are large enough, too.
> > > >
> > > >                       regards, tom lane
> > >
> > >
> > >
> > > ---------------------------(end of
> > broadcast)---------------------------
> > > TIP 6: explain analyze is your friend
> > >
> >
> >
> >
> > --
> > "Adopting the position that you are smarter than an automatic
> > optimization algorithm is generally a good way to achieve less
> > performance, not more" - Tom Lane.





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: vacuumdb -a -f
@ 2005-08-17 18:15  D Kavan <bitsandbytes88@hotmail.com>
  parent: Guido Barosio <gbarosio@gmail.com>
  1 sibling, 1 reply; 9+ messages in thread

From: D Kavan @ 2005-08-17 18:15 UTC (permalink / raw)
  To: pgsql-admin


'du -h' in the base directory.

>How are you finding out the DB size?
>  G.-





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: vacuumdb -a -f
@ 2005-08-17 18:59  Alvaro Herrera <alvherre@alvh.no-ip.org>
  parent: D Kavan <bitsandbytes88@hotmail.com>
  0 siblings, 0 replies; 9+ messages in thread

From: Alvaro Herrera @ 2005-08-17 18:59 UTC (permalink / raw)
  To: D Kavan <bitsandbytes88@hotmail.com>; +Cc: pgsql-admin

On Wed, Aug 17, 2005 at 02:15:42PM -0400, D Kavan wrote:
> 
> 'du -h' in the base directory.
> 
> >How are you finding out the DB size?

You are considering the difference in XLog segment size, right?  The
pg_xlog directory may start almost empty and then start filling.  The
space will not be recovered by any maintenance command.

-- 
Alvaro Herrera (<alvherre[a]alvh.no-ip.org>)
Y dijo Dios: "Que sea Satanás, para que la gente no me culpe de todo a mí."
"Y que hayan abogados, para que la gente no culpe de todo a Satanás"



^ permalink  raw  reply  [nested|flat] 9+ messages in thread


end of thread, other threads:[~2005-08-17 18:59 UTC | newest]

Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-08-15 20:16 vacuumdb -a -f D Kavan <bitsandbytes88@hotmail.com>
2005-08-16 01:31 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-08-17 15:24 Re: vacuumdb -a -f D Kavan <bitsandbytes88@hotmail.com>
2005-08-17 15:57 ` Guido Barosio <gbarosio@gmail.com>
2005-08-17 16:57   ` Chris Hoover <revoohc@gmail.com>
2005-08-17 17:41     ` D Kavan <bitsandbytes88@hotmail.com>
2005-08-17 18:15   ` D Kavan <bitsandbytes88@hotmail.com>
2005-08-17 18:59     ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2005-08-17 16:15 ` Joshua D. Drake <jd@commandprompt.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox