From: Ron <ronljohnsonjr@gmail.com>
To: pgsql-general@lists.postgresql.org <pgsql-general@lists.postgresql.org>
Subject: Re: Purging few months old data and vacuuming in production
Date: Fri, 6 Jan 2023 08:57:26 -0600
Message-ID: <90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com> (raw)
In-Reply-To: <SEZPR06MB5690F3A7AE31D9EA19DA0A28C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com>
References: <SEZPR06MB569032257FDFACE5C906ADB4C2F09@SEZPR06MB5690.apcprd06.prod.outlook.com>
<6f623e3a-d93d-bc89-b553-d2f87a82bd73@gmail.com>
<SEZPR06MB56905EB1343165E6D1C8B5B4C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com>
<1984ec34-26fa-a585-3312-ef38814b5f42@gmail.com>
<SEZPR06MB5690F3A7AE31D9EA19DA0A28C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com>
On 1/6/23 08:27, Ranjith Paliyath wrote:
> Thank you very much for the response.
>
> > Can you do online purging?
>
> > For example, get a list of the main table's primary keys to be deleted, and
> > then nibble away at them all day: in one transaction delete all the records
> > for one logically related set of records. Do that N million times, and
> > you've purged the data without impacting production.
>
> So, with this approach, is the advantage like, manual vacuuming worry may be set aside, because auto-vacuuming would deal with the dead rows?
Theoretically, manual vacuuming is never necessary. I'd occasionally do
manual vacuums (after purging a couple of weeks of data, for example).
Disable autovacuum on a table, vacuum it, then reenable autovacuum.
ALTER TABLE table_name SET (autovacuum_enabled = false);
VACUUM table_name;
ALTER TABLE table_name SET (autovacuum_enabled = true);
> This is because the deletion step is executed record by record in main table, with its connected record(s) delete executions in rest of tables?
I don't know if you have ON DELETE CASCADE. Even if you do, you'll have to
manually delete the tables not linked by FK. I'd write a PL/pgSQL
procedure: pass in a PK and then delete records from the 9 tables in the
proper order so as to not throw FK constraint errors.
> Due to the infra capability that is there in this instance,
What is "infra capability"?
> the impact could be almost none!!??
It'll use /some/ resources, because it's a thread deleting records, but most
of the records and index nodes won't be where new records are being inserted.
Note, though, that this will generate a lot of WAL records.
--
Born in Arizona, moved to Babylonia.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-general@postgresql.org
Cc: ronljohnsonjr@gmail.com
Subject: Re: Purging few months old data and vacuuming in production
In-Reply-To: <90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com>
References: <SEZPR06MB569032257FDFACE5C906ADB4C2F09@SEZPR06MB5690.apcprd06.prod.outlook.com> <6f623e3a-d93d-bc89-b553-d2f87a82bd73@gmail.com> <SEZPR06MB56905EB1343165E6D1C8B5B4C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com> <1984ec34-26fa-a585-3312-ef38814b5f42@gmail.com> <SEZPR06MB5690F3A7AE31D9EA19DA0A28C2FB9@SEZPR06MB5690.apcprd06.prod.outlook.com> <90262fac-d6c7-e253-6988-a081c9e47a15@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox