From: Adrian Klaver <adrian.klaver@aklaver.com>
To: sud <suds1434@gmail.com>
Cc: pgsql-general <pgsql-general@lists.postgresql.org>
Subject: Re: Question related to partitioning with pg_partman
Date: Sun, 10 Mar 2024 13:25:42 -0700
Message-ID: <14f28b7a-9ea4-49b2-9000-b88dbe0332ec@aklaver.com> (raw)
In-Reply-To: <CAD=mzVWHrzvLd9NsOC=DLEJOSiG3hSWVxyedK9yRvEWi94_9pQ@mail.gmail.com>
References: <CAD=mzVWc=HOHrccJ20BJB5sGj_pLkaCa-7qHjV9jGAbbE3X-TQ@mail.gmail.com>
<7ca9af73-6cb2-4ac0-8dfb-b83a50c64363@aklaver.com>
<CAD=mzVXP_WEXT_Zk4R8bb7ztar1N7XsG+7BgmqJJ-LgQrNAKHw@mail.gmail.com>
<46b8c853-a98c-41ee-b3f3-e1fcf11f26db@aklaver.com>
<CAD=mzVXtks_qLKJiDO11oQnkRYs+Yr2=Hh_Ctj8yQVP10jqsmw@mail.gmail.com>
<0a663c42-32fe-486e-8fca-63c0faa7b7c9@aklaver.com>
<CAD=mzVWHrzvLd9NsOC=DLEJOSiG3hSWVxyedK9yRvEWi94_9pQ@mail.gmail.com>
On 3/10/24 11:34, sud wrote:
>
> On Sun, Mar 10, 2024 at 11:31 PM Adrian Klaver
> <adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>> wrote:
>
> 1) The partition will be across one day(24 hours) it is just the times
> may confuse people. Per you example 2024-03-07 00:00:00+00 is the same
> time as 2024-03-06 19:00:00-05 for EST. The issue is that the +00 and
> -05 maybe ignored. Also it depends on the clients being consistent in
> using timestamptz.
>
> 2) You still have not answered what the datetime range(not date range)
> is that will be queried. If you have the partitions Midnight to
> Midnight
> UTC and the clients are querying Midnight to Midnight local time the
> query will not match the partitions.
>
>
> My apology if not able to clearly put the details. Actually, the query
> will always happen on a day basis i.e they can query from one day to 15
> days transactions. But as you rightly pointed , the partitions can only
> span from midnight to midnight in one timezone, and thus users who
> queries the data from another time zone will mostly scan two partitions
> (even if they just queries one days transaction data in their own
> timezone). And I don't see an easy solution for this , which will help
> all users across all time zones to scan only a single partition in the
> database, when they queries data for a single transaction date.
>
> And thus my question was, is it necessary to have the creation of
> partitions to happen on UTC time zone only? and then whatever
> transaction data inserted by the users from respective time zones will
> be stored in the database as is and will be queried based on the user
> timezone (it may span across multiple partitions though for a single
> user transaction date).
This is going to depend on many things.
1) Partitions are not free they have overhead, which is fine if the
cost(overhead) is less then the benefits. For details on that see:
https://www.postgresql.org/docs/current/ddl-partitioning.html
and partition parts of
https://www.postgresql.org/docs/current/sql-createtable.html
As part of this there is the consideration of whether daily partitions
are really what you want?
2) What you hope to get out of the partitioning?
a) If it is confining queries to the partition boundaries then you have
already stated that is not going to happen.
b) If it is for data pruning purposes, then you have something to
consider on both ends. Creating/dropping partitions with Midnight to
Midnight UTC means you will need to consider whether they cover the
range of datetimes that your users are interested in. In other words
creating a partition ahead that covers local times that resolve to a UTC
time in the 'future'. On the back end not dropping a partition until it
has gone out of scope for everybody.
To answer 1 & 2 you are probably going to need to create a test setup
and verify how the expected queries are actually going to work with your
partition scheme.
--
Adrian Klaver
adrian.klaver@aklaver.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-general@postgresql.org
Cc: adrian.klaver@aklaver.com, suds1434@gmail.com
Subject: Re: Question related to partitioning with pg_partman
In-Reply-To: <14f28b7a-9ea4-49b2-9000-b88dbe0332ec@aklaver.com>
References: <CAD=mzVWc=HOHrccJ20BJB5sGj_pLkaCa-7qHjV9jGAbbE3X-TQ@mail.gmail.com> <7ca9af73-6cb2-4ac0-8dfb-b83a50c64363@aklaver.com> <CAD=mzVXP_WEXT_Zk4R8bb7ztar1N7XsG+7BgmqJJ-LgQrNAKHw@mail.gmail.com> <46b8c853-a98c-41ee-b3f3-e1fcf11f26db@aklaver.com> <CAD=mzVXtks_qLKJiDO11oQnkRYs+Yr2=Hh_Ctj8yQVP10jqsmw@mail.gmail.com> <0a663c42-32fe-486e-8fca-63c0faa7b7c9@aklaver.com> <CAD=mzVWHrzvLd9NsOC=DLEJOSiG3hSWVxyedK9yRvEWi94_9pQ@mail.gmail.com> <14f28b7a-9ea4-49b2-9000-b88dbe0332ec@aklaver.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox