From: basti <mailinglist@unix-solution.de>
To: pgsql-general@lists.postgresql.org
Subject: Re: Query take a long time and use no index
Date: Mon, 17 Jul 2023 20:19:08 +0200
Message-ID: <4b927a7f-ff01-0df3-27b6-30b0fee17969@unix-solution.de> (raw)
In-Reply-To: <7599651d-7269-ec5b-b968-46795a76c375@enterprisedb.com>
References: <5f826baf-e345-48a8-5672-db07add3e6b2@unix-solution.de>
<d9708792-19fc-2dc7-4ecf-f93b69a4c2b0@enterprisedb.com>
<7599651d-7269-ec5b-b968-46795a76c375@enterprisedb.com>
Thanks a lot tomas, i will try it.
I have find out that there is a 'aggregation' function in the frontend.
But this is MySQL specific and I have no idea the transform it to postgres.
It looks like:
'REPLACE INTO aggregate (channel_id, type, timestamp, value, count)
SELECT channel_id, ? AS type, MAX(agg.timestamp) AS timest
amp, COALESCE( SUM(agg.val_by_time) / (MAX(agg.timestamp) -
MIN(agg.prev_timestamp)), AVG(agg.value)) AS value, COUNT(agg.value) AS
count FROM ( SELECT channel_id,
timestamp, value, value * (timestamp - @prev_timestamp) AS
val_by_time, COALESCE(@prev_timestamp, 0) AS prev_timestamp,
@prev_timestamp := timestamp FROM data CROSS
JOIN (SELECT @prev_timestamp := NULL) AS vars WHERE channel_id = ?
AND timestamp < UNIX_TIMESTAMP(DATE_FORMAT(NOW(), "%Y-%m-%d")) * 1000 )
AS agg GROUP BY channel_
id, DATE_TRUNC('day', TIMESTAMP 'epoch' + timestamp * INTERVAL '1
millisecond')' with params [3, 5]:
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near
"REPLACE"
LINE 1: REPLACE INTO aggregate (channel_id, type, timestamp, value,
...
Am 17.07.23 um 13:21 schrieb Tomas Vondra:
> On 7/17/23 13:20, Tomas Vondra wrote:
>> ...
>>
>> It's always going to be slow with the COUNT(DISTINCT), I'm afraid.
>>
>> Not sure how much you can modify the query / database, and how accurate
>> results you need. If you're OK with estimates, you can try postgres-hll
>> extension [2] which estimates count(distinct). For exact reaults, I
>> wrote count_distinct extension [2] that uses hashtable. Might be worth a
>> try, I guess.
>>
>
> Damn, I forgot to add the links:
>
> [1] https://github.com/citusdata/postgresql-hll
>
> [2] https://github.com/tvondra/count_distinct
>
> regards
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-general@postgresql.org
Cc: mailinglist@unix-solution.de
Subject: Re: Query take a long time and use no index
In-Reply-To: <4b927a7f-ff01-0df3-27b6-30b0fee17969@unix-solution.de>
References: <5f826baf-e345-48a8-5672-db07add3e6b2@unix-solution.de> <d9708792-19fc-2dc7-4ecf-f93b69a4c2b0@enterprisedb.com> <7599651d-7269-ec5b-b968-46795a76c375@enterprisedb.com> <4b927a7f-ff01-0df3-27b6-30b0fee17969@unix-solution.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox