Received: from malur.postgresql.org ([217.196.149.56])
	by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256)
	(Exim 4.92)
	(envelope-from <pgsql-performance-owner+archive@lists.postgresql.org>)
	id 1j712Z-0006qa-Gs
	for pgsql-performance@arkaria.postgresql.org; Wed, 26 Feb 2020 18:04:11 +0000
Received: from localhost ([127.0.0.1] helo=malur.postgresql.org)
	by malur.postgresql.org with esmtp (Exim 4.89)
	(envelope-from <pgsql-performance-owner+archive@lists.postgresql.org>)
	id 1j712Y-0007RA-8K
	for pgsql-performance@arkaria.postgresql.org; Wed, 26 Feb 2020 18:04:10 +0000
Received: from makus.postgresql.org ([2001:4800:3e1:1::229])
	by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256)
	(Exim 4.89)
	(envelope-from <gc@mnc.ch>)
	id 1j712X-0007R2-RV
	for pgsql-performance@lists.postgresql.org; Wed, 26 Feb 2020 18:04:09 +0000
Received: from mail.mnc.ch ([62.2.77.205])
	by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(Exim 4.92)
	(envelope-from <gc@mnc.ch>)
	id 1j712U-0008Sa-QD
	for pgsql-performance@postgresql.org; Wed, 26 Feb 2020 18:04:07 +0000
X-Virus-Scanned: amavisd-new at mnc.ch
Sender: gc@mnc.ch
Received: by gc-P7xxDM3-G.localdomain (Postfix, from userid 1000)
	id 7818C80960; Wed, 26 Feb 2020 19:04:02 +0100 (CET)
From: Guillaume Cottenceau <gc@mnc.ch>
To: Michael Lewis <mlewis@entrata.com>,
 postgres performance list <pgsql-performance@postgresql.org>
Subject: Re: much slower query in production
References: <878skpnx0u.fsf@mnc.ch> <20200226162809.GZ31889@telsasoft.com>
	<CAHOFxGrEX4FGV=uEtzWdS-MNgh4t5U139TJKPwYFFLfWcBrEfA@mail.gmail.com>
Date: Wed, 26 Feb 2020 19:04:02 +0100
In-Reply-To: 
 <CAHOFxGrEX4FGV=uEtzWdS-MNgh4t5U139TJKPwYFFLfWcBrEfA@mail.gmail.com>
	(Michael Lewis's message of "Wed, 26 Feb 2020 09:52:37 -0700")
Message-ID: <87sgixmdil.fsf@mnc.ch>
MIME-Version: 1.0
Content-Type: text/plain
List-Id: <pgsql-performance.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-performance@lists.postgresql.org>
List-Owner: <mailto:pgsql-performance-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-performance>
Precedence: bulk

Michael Lewis <mlewis 'at' entrata.com> writes:

> By the way, I expect the time is cut in half while heap fetches stays similar because the index is now in OS cache on the
> second run and didn't need to be fetched from disk. Definitely need to check on vacuuming as Justin says. If you have a fairly
> active system, you would need to run this query many times in order to push other stuff out of shared_buffers and get this
> query to perform more like it does on dev.
>
> Do you have the option to re-write the query or is this generated by an ORM? You are forcing the looping as I read this query.
> If you aggregate before you join, then the system should be able to do a single scan of the index, aggregate, then join those
> relatively few rows to the multicards table records.
>
> SELECT transaction_uid, COALESCE( sub.count, 0 ) AS count FROM multicards LEFT JOIN (SELECT multicard_uid, COUNT(*) AS count
> FROM tickets GROUP BY multicard_uid ) AS sub ON sub.multicard_uid = multicards.uid;

Thanks for this hint! I always hit this fact that I never write
good queries using explicit joins :/

Execution time (before vacuuming the table as adviced by Justin)
down 38x to 44509ms using this query :)

Real query was an UPDATE of the multicards table to set the count
value. I rewrote this using your approach but I think I lack what
coalesce did in your query, this would update only the rows where
count >= 1 obviously:

UPDATE multicards
   SET defacements = count
  FROM ( SELECT multicard_uid, COUNT(*) AS count FROM tickets GROUP BY multicard_uid ) AS sub
 WHERE uid = multicard_uid;

Any hinted solution to do that in one pass? I could do a first
pass setting defacements = 0, but that would produce more garbage :/

Thanks!

-- 
Guillaume Cottenceau