pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedUpgrading cluster with thousands of DBs
3+ messages / 3 participants
[nested] [flat]
* Upgrading cluster with thousands of DBs
@ 2014-01-14 10:45 Alejandro Martínez <zen@itram.es>
0 siblings, 2 replies; 3+ messages in thread
From: Alejandro Martínez @ 2014-01-14 10:45 UTC (permalink / raw)
To: pgsql-admin
Hi list,
We're working on upgrading from 9.1 to 9.3 some of our database
clusters, while seeking to minimize (write) downtime by making the
upgrades as fast as possible.
However, our clusters have the particularity that they get to host
many different *databases*, as in, ~2000 databases per cluster. Most
of those databases are small (only a table and PostGIS functions), so
it's more a problem of the number of DBs than the space they use. The
total size of the cluster is around 60gb.
I first started using pg_upgrade but, even when enabling
parallelization and hardlinks, as most of the steps (preliminary
check?) don't seem to be parallelized, the upgrade takes ~5 hours as
it goes database-per-database serially on most steps.
I'm experimenting with reducing the time by spawning parallel pg_dump
and pg_restore instances, so far I've reached a 3:15h mark that I
still think can reduce a bit more with some tweaking.
Does anyone have any tips on reducing the time of upgrading such a
kind of a cluster?
Thanks!
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Upgrading cluster with thousands of DBs
@ 2014-01-14 10:52 Gabriele Bartolini <Gabriele.Bartolini@2ndQuadrant.it>
parent: Alejandro Martínez <zen@itram.es>
1 sibling, 0 replies; 3+ messages in thread
From: Gabriele Bartolini @ 2014-01-14 10:52 UTC (permalink / raw)
To: Alejandro Martínez <zen@itram.es>; +Cc: pgsql-admin
Hi,
On Tue, 14 Jan 2014 11:45:33 +0100, Alejandro Martínez <zen@itram.es>
wrote:
> However, our clusters have the particularity that they get to host
> many different *databases*, as in, ~2000 databases per cluster. Most
> of those databases are small (only a table and PostGIS functions), so
> it's more a problem of the number of DBs than the space they use. The
> total size of the cluster is around 60gb.
If this is your case, I'd suggest an "old-school" approach and migrate
one database at a time on a new server with Postgres 9.3.
Cheers,
Gabriele
--
Gabriele Bartolini - 2ndQuadrant Italia
PostgreSQL Training, Services and Support
Gabriele.Bartolini@2ndQuadrant.it - www.2ndQuadrant.it
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Upgrading cluster with thousands of DBs
@ 2014-01-22 02:05 Bruce Momjian <bruce@momjian.us>
parent: Alejandro Martínez <zen@itram.es>
1 sibling, 0 replies; 3+ messages in thread
From: Bruce Momjian @ 2014-01-22 02:05 UTC (permalink / raw)
To: Alejandro Martínez <zen@itram.es>; +Cc: pgsql-admin
On Tue, Jan 14, 2014 at 11:45:33AM +0100, Alejandro Martínez wrote:
> Hi list,
>
> We're working on upgrading from 9.1 to 9.3 some of our database
> clusters, while seeking to minimize (write) downtime by making the
> upgrades as fast as possible.
>
> However, our clusters have the particularity that they get to host
> many different *databases*, as in, ~2000 databases per cluster. Most
> of those databases are small (only a table and PostGIS functions), so
> it's more a problem of the number of DBs than the space they use. The
> total size of the cluster is around 60gb.
>
> I first started using pg_upgrade but, even when enabling
> parallelization and hardlinks, as most of the steps (preliminary
> check?) don't seem to be parallelized, the upgrade takes ~5 hours as
> it goes database-per-database serially on most steps.
That is surprising. I know I see parallel hard link creation and
parallel database dump in my testing of 9.3 pg_upgrade.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2014-01-22 02:05 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-01-14 10:45 Upgrading cluster with thousands of DBs Alejandro Martínez <zen@itram.es>
2014-01-14 10:52 ` Gabriele Bartolini <Gabriele.Bartolini@2ndQuadrant.it>
2014-01-22 02:05 ` Bruce Momjian <bruce@momjian.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox