pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Ron <ronljohnsonjr@gmail.com>
To: pgsql-general@lists.postgresql.org <pgsql-general@lists.postgresql.org>
Subject: Re: Purging few months old data and vacuuming in production
Date: Fri, 6 Jan 2023 08:57:26 -0600
Message-ID: <90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com> (raw)
In-Reply-To: <SEZPR06MB5690F3A7AE31D9EA19DA0A28C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com>
References: <SEZPR06MB569032257FDFACE5C906ADB4C2F09@SEZPR06MB5690.apcprd06.prod.outlook.com>
	<6f623e3a-d93d-bc89-b553-d2f87a82bd73@gmail.com>
	<SEZPR06MB56905EB1343165E6D1C8B5B4C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com>
	<1984ec34-26fa-a585-3312-ef38814b5f42@gmail.com>
	<SEZPR06MB5690F3A7AE31D9EA19DA0A28C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com>

On 1/6/23 08:27, Ranjith Paliyath wrote:
> Thank you very much for the response.
>
>      > Can you do online purging?
>
>      > For example, get a list of the main table's primary keys to be deleted, and
>      > then nibble away at them all day: in one transaction delete all the records
>      > for one logically related set of records.  Do that N million times, and
>      > you've purged the data without impacting production.
>
> So, with this approach, is the advantage like, manual vacuuming worry may be set aside, because auto-vacuuming would deal with the dead rows?

Theoretically, manual vacuuming is never necessary.  I'd occasionally do 
manual vacuums (after purging a couple of weeks of data, for example).

Disable autovacuum on a table, vacuum it, then reenable autovacuum.

ALTER TABLE table_name SET (autovacuum_enabled = false);
VACUUM table_name;
ALTER TABLE table_name SET (autovacuum_enabled = true);


>   This is because the deletion step is executed record by record in main table, with its connected record(s) delete executions in rest of tables?

I don't know if you have ON DELETE CASCADE.  Even if you do, you'll have to 
manually delete the tables not linked by FK.  I'd write a PL/pgSQL 
procedure: pass in a PK and then delete records from the 9 tables in the 
proper order so as to not throw FK constraint errors.

> Due to the infra capability that is there in this instance,

What is "infra capability"?

> the impact could be almost none!!??

It'll use /some/ resources, because it's a thread deleting records, but most 
of the records and index nodes won't be where new records are being inserted.

Note, though, that this will generate a lot of WAL records.

-- 
Born in Arizona, moved to Babylonia.

view thread (13+ messages)  latest in thread

Message-ID: <90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com>
Permalink:  ../90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com/
Also on:    postgresql.org/message-id/90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: ronljohnsonjr@gmail.com
  Subject: Re: Purging few months old data and vacuuming in production
  In-Reply-To: <90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com>
  References: <SEZPR06MB569032257FDFACE5C906ADB4C2F09@SEZPR06MB5690.apcprd06.prod.outlook.com> <6f623e3a-d93d-bc89-b553-d2f87a82bd73@gmail.com> <SEZPR06MB56905EB1343165E6D1C8B5B4C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com> <1984ec34-26fa-a585-3312-ef38814b5f42@gmail.com> <SEZPR06MB5690F3A7AE31D9EA19DA0A28C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com> <90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox