public inbox for [email protected]
help / color / mirror / Atom feedFrom: Rick Otten <[email protected]>
To: Shaun Thomas <[email protected]>
Cc: pgsql-performa. <[email protected]>
Subject: Re: partitioning materialized views
Date: Thu, 6 Jul 2017 12:05:33 -0400
Message-ID: <CAMAYy4L=-n=VzyMi6s6tg0PSJcR6mXH2D2K1vk526uLEZkC31A@mail.gmail.com> (raw)
In-Reply-To: <CAG1YDPeH+FBFv9_w1ktPn3JyJNA1F8L_6qaMcRZZQsbBjXtxww@mail.gmail.com>
References: <CAMAYy4LD9TDO6ohfZwZ7OWOfu3oUZKQxi1Y=qt=36C65DL4jOw@mail.gmail.com>
<CAG1YDPeH+FBFv9_w1ktPn3JyJNA1F8L_6qaMcRZZQsbBjXtxww@mail.gmail.com>
List-Unsubscribe: <mailto:[email protected]?body=unsub%20pgsql-performance>
On Thu, Jul 6, 2017 at 11:25 AM, Shaun Thomas <[email protected]>
wrote:
> > I'm curious if I'm overlooking other possible architectures or tools
> that might make this simpler to manage.
>
> One of the issues with materialized views is that they are based on
> views... For a concurrent update, it essentially performs a looped
> merge, which can be pretty ugly. That's the price you pay to be
> non-blocking. For this particular setup, I'd actually recommend using
> something like pglogical to just maintain a live copy of the remote
> table or wait for Postgres 10's logical replication.
Unfortunately the foreign database is Hadoop. (As A Service)
> If you _can't_ do
> that due to cloud restrictions, you'd actually be better off doing an
> atomic swap.
>
> CREATE MATERIALIZED VIEW y AS ...;
>
> BEGIN;
> ALTER MATERIALIZED VIEW x RENAME TO x_old;
> ALTER MATERIALIZED VIEW y RENAME TO x;
> DROP MATERIALIZED VIEW x_old;
> COMMIT;
>
> This is an interesting idea. Thanks! I'll ponder that one.
> You could still follow your partitioned plan if you don't want to
> update all of the data at once. Let's face it, 3-4 hours is still a
> ton of data transfer and calculation.
>
>
yup.
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: partitioning materialized views
In-Reply-To: <CAMAYy4L=-n=VzyMi6s6tg0PSJcR6mXH2D2K1vk526uLEZkC31A@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox