From: MichaelDBA Vitale <michaeldba@sqlexec.com>
To: Mladen Gogala <gogala.mladen@gmail.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: Improve "select count(*)" query - takes more than 30 mins for some large tables
Date: Tue, 12 Jul 2022 10:31:17 -0400 (EDT)
Message-ID: <1221536526.20277.1657636277921@email.ionos.com> (raw)
In-Reply-To: <1eb233ff-90f0-529b-f0cc-fa982f668a8f@gmail.com>
References: <CACxU--UJY4sD2SjW8V3W0oAPuuTzE3fAvBzTaej-BmUUNJxN_Q@mail.gmail.com>
<1B1927BA-F7DE-40E8-8F67-BCC001CD4C6B@gmail.com>
<1eb233ff-90f0-529b-f0cc-fa982f668a8f@gmail.com>
<!doctype html>
<html>
<head>
<meta charset="UTF-8">
</head>
<body>
<div>
Perhaps do an analyze on the table and then select reltuples from pg_class for that table. Might be faster than the select count(*).
</div>
<div class="default-style">
<br>
</div>
<div class="default-style">
Regards,
</div>
<div class="default-style">
Michael Vitale
</div>
<div class="default-style">
<br>
</div>
<blockquote type="cite">
<div>
On 07/12/2022 8:51 AM Mladen Gogala <gogala.mladen@gmail.com> wrote:
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div class="moz-cite-prefix">
On 7/11/22 03:23, Florents Tselai wrote:
<br>
</div>
<blockquote type="cite">
<pre class="moz-quote-pre">psql “select id from my_table" | sort -u | wc -l
</pre>
</blockquote>
<p>That will be a lot slower than just "select count(*) from my_table". You are delivering data to the user program (psql) and then shipping them to pipe and then processing the output with "wc". Depending on the version, PostgreSQL has very reliable parallelism and can do counting rather quickly. The speed of "select count(*) from my_table" depends on the speed of I/O. Since the table is big, it cannot be cached in the file system cache, so all that you have at your disposal is the raw disk speed. For the smaller machines, NVME is the king. For larger rigs, you should consider something like Pure, XTremIO or NetApp SolidFire. People frequently expect database to do miracles with under par hardware.<br></p>
<pre class="moz-signature">--
Mladen Gogala
Database Consultant
Tel: (347) 321-1217
<a href="https://dbwhisperer.wordpress.com"; class="moz-txt-link-freetext">https://dbwhisperer.wordpress.com</a;
</pre>
</blockquote>
</body>
</html>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: michaeldba@sqlexec.com, gogala.mladen@gmail.com, pgsql-admin@lists.postgresql.org
Subject: Re: Improve "select count(*)" query - takes more than 30 mins for some large tables
In-Reply-To: <1221536526.20277.1657636277921@email.ionos.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox