Subject: Re: Dropping column from big table

From: sud <[email-masked]>
Date: Thu, 11 Jul 2024 13:10:57 +0530

On Thu, 11 Jul, 2024, 12:46 pm Ron Johnson, <[email-masked]> wrote:

> On Wed, Jul 10, 2024 at 11:28 PM sud <[email-masked]> wrote:
>
>>
>>
>>
>> Thank you so much. When you said *"you can execute one of the forms of
>> ALTER TABLE that performs a rewrite*
>> *of the whole table."* Does it mean that post "alter table drop column"
>> the vacuum is going to run longer as it will try to clean up all the rows
>> and recreate the new rows? But then how can this be avoidable or made
>> better without impacting the system performance
>>
>
> "Impact" is a non-specific word.  "How much impact" depends on how many
> autovacuum workers you've set it to use, and how many threads you set in
> vacuumdb.
>
>
>> and blocking others?
>>
>
> VACUUM never blocks.
>
> Anyway, DROP is the easy part; it's ADD COLUMN that can take a lot of time
> (depending on whether or not you populate the column with a default value).
>
> I'd detach all the partitions from the parent table, and then add the new
> column to the not-children in multiple threads, add the column to the
> parent and then reattach all of the children.  That's the fastest method,
> though takes some time to set up.
>

Thank you so much.

Dropping will take it's own time for post vacuum however as you rightly
said, it won't be blocking which should be fine.

In regards to add column, Detaching all partitions then adding column  to
the individual partition in multiple sessions and then reattaching looks to
be a really awesome idea to make it faster. However one doubt, Will it
create issue if there already exists foreign key on this partition table or
say it's the parent to other child partition/nonpartition tables?

---

From: Laurenz Albe <[email-masked]>
Date: Thu, 11 Jul 2024 10:06:47 +0200

On Thu, 2024-07-11 at 13:10 +0530, sud wrote:
> Dropping will take it's own time for post vacuum however as you
> rightly said, it won't be blocking which should be fine. 

I am not certain if you understood this correctly.

Dropping a column is fast, but doesn't reclaim the space.
VACUUM won't block anything, but won't reclaim the space.
VACUUM (FULL) will block everything, but will also not reclaim the space.

You'd need to use a form of ALTER TABLE that rewrites the table,
as indicated in the documentation.  However, such an operation
will block all access to the table for a long time, and it will
temporarily need much more space, because it has to hold both the
old and the new copy of the table.

Yours,
Laurenz Albe

---

From: "Peter J. Holzer" <[email-masked]>
Date: Sun, 14 Jul 2024 00:05:45 +0200

On 2024-07-11 10:06:47 +0200, Laurenz Albe wrote:
> as indicated in the documentation.

Unfortunately the documentation indicates very little. It mentions that
the table will be rewritten with

* SET ACCESS METHOD
* a volatile DEFAULT
* changing the type of an existing column (unless binary coercible)

All three change something which you probably don't want to change.

The documentation also mentions some cases where the table is not
rewritten, so maybe some not explicitely mentioned options rewrite the
table, too.

I would especially expected ALTER TABLE ... CLUSTER to do this, but if
VACUUM FULL preserves the (former) content of dropped columns, maybe
CLUSTER does, too?

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | [email-masked]         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

---

From: Laurenz Albe <[email-masked]>
Date: Mon, 15 Jul 2024 13:53:25 +0200

> All three change something which you probably don't want to change.

Hm, true.

You can always do

  UPDATE tab SET id = id;

followed by

  VACUUM (FULL) tab;

Yours,
Laurenz Albe