public inbox for [email protected]help / color / mirror / Atom feed
Alter table fast 6+ messages / 5 participants [nested] [flat]
* Alter table fast @ 2025-01-09 16:25 veem v <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: veem v @ 2025-01-09 16:25 UTC (permalink / raw) To: pgsql-general <[email protected]> Hello, It's postgres version 16.1, we want to convert an existing column data type from integer to numeric and it's taking a long time. The size of the table is ~50GB and the table has ~150million rows in it and it's not partitioned. We tried running the direct alter and it's going beyond hours, so wanted to understand from experts what is the best way to achieve this? 1)Should we go with below Alter table <table_name> alter column <column_name> type numeric(15,0) USING <column_name>::NUMERIC(15,0); OR We should add a new not null column. update the data in that column from the existing column. drop the old column rename the new column to the old column. Regards Veem ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Alter table fast @ 2025-01-09 16:27 Ron Johnson <[email protected]> parent: veem v <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: Ron Johnson @ 2025-01-09 16:27 UTC (permalink / raw) To: pgsql-general <[email protected]> On Thu, Jan 9, 2025 at 11:25 AM veem v <[email protected]> wrote: > Hello, > It's postgres version 16.1, we want to convert an existing column data > type from integer to numeric and it's taking a long time. The size of the > table is ~50GB and the table has ~150million rows in it and it's not > partitioned. We tried running the direct alter and it's going beyond hours, > so wanted to understand from experts what is the best way to achieve this? > > Out of curiosity, why NUMERIC(15,0) instead of BIGINT? -- Death to <Redacted>, and butter sauce. Don't boil me, I'm still alive. <Redacted> lobster! ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Alter table fast @ 2025-01-09 19:17 veem v <[email protected]> parent: Ron Johnson <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: veem v @ 2025-01-09 19:17 UTC (permalink / raw) To: Ron Johnson <[email protected]>; +Cc: pgsql-general <[email protected]> On Thu, 9 Jan 2025 at 21:57, Ron Johnson <[email protected]> wrote: > On Thu, Jan 9, 2025 at 11:25 AM veem v <[email protected]> wrote: > >> Hello, >> It's postgres version 16.1, we want to convert an existing column data >> type from integer to numeric and it's taking a long time. The size of the >> table is ~50GB and the table has ~150million rows in it and it's not >> partitioned. We tried running the direct alter and it's going beyond hours, >> so wanted to understand from experts what is the best way to achieve this? >> >> > Out of curiosity, why NUMERIC(15,0) instead of BIGINT? > > > It's for aligning the database column types to the data model and it's happening across all the upstream downstream systems. I was thinking if this can be made faster with the single line alter statement "Alter table <table_name> alter column <column_name> type numeric(15,0) USING <column_name>::NUMERIC(15,0);" or through the UPDATE column rename column strategy. Additionally if this can be further improved using any session level parameter like "max_parallel_workers_per_gather", "max_parallel_workers", "maintenance_work_mem", "work_mem"? ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Alter table fast @ 2025-01-09 19:52 [email protected] parent: veem v <[email protected]> 0 siblings, 1 reply; 6+ messages in thread From: [email protected] @ 2025-01-09 19:52 UTC (permalink / raw) To: [email protected] Am 09.01.25 um 20:17 schrieb veem v: >> Out of curiosity, why NUMERIC(15,0) instead of BIGINT? > > It's for aligning the database column types to the data model and > it's happening across all the upstream downstream systems. I was > thinking if this can be made faster with the single line alter > statement "Alter table <table_name> alter column <column_name> type > numeric(15,0) USING <column_name>::NUMERIC(15,0);" Hmm, I would rather change numeric(15,0) to bigint if I had to "align" types across systems. ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Alter table fast @ 2025-01-12 21:19 Peter J. Holzer <[email protected]> parent: [email protected] 0 siblings, 1 reply; 6+ messages in thread From: Peter J. Holzer @ 2025-01-12 21:19 UTC (permalink / raw) To: [email protected] On 2025-01-09 20:52:27 +0100, [email protected] wrote: > Am 09.01.25 um 20:17 schrieb veem v: > > > Out of curiosity, why NUMERIC(15,0) instead of BIGINT? > > > > It's for aligning the database column types to the data model and > > it's happening across all the upstream downstream systems. I was > > thinking if this can be made faster with the single line alter > > statement "Alter table <table_name> alter column <column_name> type > > numeric(15,0) USING <column_name>::NUMERIC(15,0);" > > Hmm, I would rather change numeric(15,0) to bigint if I had to "align" types across systems. I'm also wondering what "the data model" is. If I have numeric(15,0) in an abstract data model, that means that I expect values larger than 99,999,999,999,999 but at most 999,999,999,999,999. That seems to be oddly specific and also somewhat at odds with reality when until now there apparently haven't been any values larger than 2,147,483,647. What kind of real world value could suddenly jump by more than 5 orders of magnitude but certainly not by 7? A bigint is much less precise (more than 2,147,483,647 but not more than 9,223,372,036,854,775,807) and therefore more suitable for values where you don't really know the range. However, for the problem at hand, I doubt it makes any difference. Surely converting a few million values takes much less time than rewriting a 50 GB table and all its indexes. So there isn't really a faster way to do what Veem wants. There may however be less disruptive way: He could create a new column with the new values (which takes at least as long but can be done in the background) and then switch it over and drop the old column. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | [email protected] | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" Attachments: [application/pgp-signature] signature.asc (833B, 2-signature.asc) download ^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Alter table fast @ 2025-01-12 21:36 Marco Torres <[email protected]> parent: Peter J. Holzer <[email protected]> 0 siblings, 0 replies; 6+ messages in thread From: Marco Torres @ 2025-01-12 21:36 UTC (permalink / raw) To: [email protected] This is the right approach, Peter J. Holzer, from a well season DBA perspective "ALTER TABLE working_table ADD COLUMN B INTEGER ; UPDATE working_table SET B = A;" Bare in mind the indexes or existing references to an from other tables and act accordingly-- define the new and drop the old. Good luck. On Sun, Jan 12, 2025, 2:20 PM Peter J. Holzer <[email protected]> wrote: > On 2025-01-09 20:52:27 +0100, [email protected] wrote: > > Am 09.01.25 um 20:17 schrieb veem v: > > > > Out of curiosity, why NUMERIC(15,0) instead of BIGINT? > > > > > > It's for aligning the database column types to the data model and > > > it's happening across all the upstream downstream systems. I was > > > thinking if this can be made faster with the single line alter > > > statement "Alter table <table_name> alter column <column_name> type > > > numeric(15,0) USING <column_name>::NUMERIC(15,0);" > > > > Hmm, I would rather change numeric(15,0) to bigint if I had to "align" > types across systems. > > I'm also wondering what "the data model" is. > > If I have numeric(15,0) in an abstract data model, that means that I > expect values larger than 99,999,999,999,999 but at most > 999,999,999,999,999. That seems to be oddly specific and also somewhat > at odds with reality when until now there apparently haven't been any > values larger than 2,147,483,647. What kind of real world value could > suddenly jump by more than 5 orders of magnitude but certainly not by 7? > > A bigint is much less precise (more than 2,147,483,647 but not more > than 9,223,372,036,854,775,807) and therefore more suitable for values > where you don't really know the range. > > However, for the problem at hand, I doubt it makes any difference. > Surely converting a few million values takes much less time than > rewriting a 50 GB table and all its indexes. > > So there isn't really a faster way to do what Veem wants. There may > however be less disruptive way: He could create a new column with the > new values (which takes at least as long but can be done in the > background) and then switch it over and drop the old column. > > hp > > -- > _ | Peter J. Holzer | Story must make more sense than reality. > |_|_) | | > | | | [email protected] | -- Charles Stross, "Creative writing > __/ | http://www.hjp.at/ | challenge!" > ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2025-01-12 21:36 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-01-09 16:25 Alter table fast veem v <[email protected]> 2025-01-09 16:27 ` Ron Johnson <[email protected]> 2025-01-09 19:17 ` veem v <[email protected]> 2025-01-09 19:52 ` [email protected] 2025-01-12 21:19 ` Peter J. Holzer <[email protected]> 2025-01-12 21:36 ` Marco Torres <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox