Date: Thu, 11 Jul 2024 17:19:42 +0200 (CEST)
From: Dimitrios Apostolou <jimis@gmx.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
cc: David Rowley <dgrowleyml@gmail.com>, pgsql-general@lists.postgresql.org
Subject: Re: Query on partitioned table needs memory n_partitions *
 work_mem
In-Reply-To: <2077940.1720709022@sss.pgh.pa.us>
Message-ID: <82476a55-bb26-b566-e6d9-907592ce732e@gmx.net>
References: <3603c380-d094-136e-e333-610914fb3e80@gmx.net> <CAApHDvp_u6SbyTKtk9m7ssh8H4dXMoLMZrAq8a1sJk1cj6YxzQ@mail.gmail.com> <d26e67d3-74bc-60aa-bf24-2a8fb83efe9c@gmx.net> <2077940.1720709022@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII; format=flowed
UI-OutboundReport: notjunk:1;M01:P0:mGAkaL2Tick=;tRuBqQlyai6PiWQsRMFSxTPCWkV
 2b85rPfbg8bxlDtLPpXwRToe6W2Ht3HqhhThDZ3j+deuVbDu+N9G87m9CQCxBKsGIL0uRdh2p
 G8yd8jz2l6RQTkrSdjZxrnFzXVHAYk/iHmJqLT4s2U3othMkms9eM2AC6yruHPmVkR0PKfb2H
 Do9yb4YvBSnoeSs7RgQ0d4vNk/g1WnrB18xxwQmUd5sneqKsMp2KckFqA5mo1nDvPZDdKFNNy
 WY2q1wV9g2+YKqeu20KmvFzc8cNC/kIr1Xm2C39XfIcpfIK6ySy7XVqKrDSiF2VQtbCeQAqhc
 +Vx52PMBhZoLXCACw620WUa1d68Wj6HVUFNa21duF92qQIg7Snhfyv1X00CZgRMVfZPwEXFjc
 heY7PydVm1tTnxmwIsH87vzV2gfb75FD4dTCtkr2MP5f8JJI2zQGYiK2O1FNEhrKkeHjQAI3e
 gPU0VLArUUzaXx4890eoYO3fpbjhzaLg1xM04VFb9ffmc78Yig+zrtBFUd/nUMY+lrzDVYknC
 Uk+/54VLSlfe1l42VniGislG7PFzn6MAou5snRLG58/aBvbl2doL1vgmeR5qXdGuJr0HhsQ29
 PhruhQ5AJ9hq5BcQfceJIca+fZg0yEmj9/QpAlcZGW1ouWzKy3RX3HjY5zgDRDjHXCFAaqUFG
 cJ6antDLYdQHirS+dOgtGRNz0Wyvl0HYXRf8X5QroQ50mn2ye35N9NptUrWhNM06W6BPDeS6m
 IZnjtD3EHP1J9qK7coO7DdLZWWi9YHiv+jvXbF26v9+4P/HLbpfYriBEDlTD/r4QSx26dAkuI
 M9apZdUgKHhUyHaHEbZZ0aIw==
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/82476a55-bb26-b566-e6d9-907592ce732e%40gmx.net>
Precedence: bulk

On Thu, 11 Jul 2024, Tom Lane wrote:

> Dimitrios Apostolou <jimis@gmx.net> writes:
>> The TABLE test_runs_raw has 1000 partitions on RANGE(workitem_n).
>
> So don't do that.  Adding partitions is not cost-free.
>

I understand that, they also add an administrative cost that I'd rather
avoid. But I ended up adding all these partitions because of performance
issues on a multi-billion rows table. There is probably some message from
me on this list a couple of years ago.

At the moment I have a work-around. I'm thankful that everyone is willing
to provide workarounds to all potential issues/bugs I have presented, but
unfortunately workarounds are not fixes, one will hit the same wall again
at some point.

My current concern is **reporting my findings responsibly**. I want to
provide as much data needed to pinpoint the issue, so that the developers
know exactly what's going on. Having right data is half the fix.

A way to track the issue would be nice. I might revisit it and even try to
submit a patch. I wonder how the postgres development community is
tracking all these issues, I've even started forgetting the ones I have
found, and I'm sure I have previously reported (on this list) a couple of
should-be-easy issues that would be ideal for beginners.


Regards,
Dimitris