Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rl28O-00HBKG-50 for pgsql-general@arkaria.postgresql.org; Fri, 15 Mar 2024 07:37:44 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rl28M-006oSY-PK for pgsql-general@arkaria.postgresql.org; Fri, 15 Mar 2024 07:37:43 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rl28M-006oSO-A4 for pgsql-general@lists.postgresql.org; Fri, 15 Mar 2024 07:37:42 +0000 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1rl28J-004bji-0O for pgsql-general@lists.postgresql.org; Fri, 15 Mar 2024 07:37:41 +0000 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-414023d288cso937995e9.2 for ; Fri, 15 Mar 2024 00:37:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec-at.20230601.gappssmtp.com; s=20230601; t=1710488258; x=1711093058; darn=lists.postgresql.org; h=mime-version:user-agent:content-transfer-encoding:autocrypt :references:in-reply-to:date:cc:to:from:subject:message-id:from:to :cc:subject:date:message-id:reply-to; bh=7dDRvT0p47ocHbng2u+WhNc/WfWlWMO6KZ9mD0cmb9U=; b=GnQY+6R5EX/F7zEhqfDmKJwTTMX/aXLyl6gDYgcJqYAYbMNX8obYBEcOyjXZKJpH76 +Bhn9aLf3sKhIEOo/fyvjmwrXO3AU5G1F8Nf9YtWLp6P7OVd0iBgtYZnnmG0/qCWdUvs tktTCUGk4J7/tm3tfdqaq7K8ndxS2cpcTd8Uj+faNqNtBdgpxDnoMCizGQVMXrIjZBLY RzeGv06TaLKfoloJTLoYTvsvm6OuMcbdVs0U672GfNCeeSpfRBoBwEDO+xOP3Y0PyttX CbSL9c4BjLQyDAlid5jydok+nFXUChXG+tQaNN21pJnR0jwgXK829ccuD1gT+OciLG3z C1ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710488258; x=1711093058; h=mime-version:user-agent:content-transfer-encoding:autocrypt :references:in-reply-to:date:cc:to:from:subject:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7dDRvT0p47ocHbng2u+WhNc/WfWlWMO6KZ9mD0cmb9U=; b=TJ0ohrkLLjb1FzozfsCs9t9rFRHmB3seKOUILvOb1BfCEUxyGRzikej0cq2dMYMbqo dJ7/pFlmcH8pcrs0rzpPIb0hN5WSR2jD45RWsxnlQ4QCVYWVbh9k6t1gdTD7mG/UIaZ9 +U+/sY2uzZvZPJRDJ3eM0Ia3wyzhdN7tfTXTZHr9Poz0aRqnpLvMyh9ajqeaaR6QMgLu unojnY+RtrF98GXD9BxF5Y4oB8bDOO1Fl4EKWjhBmyUAFJxrQmgolvcwDYTWjVIkVr0m zluVk631BgYbBgK2ZIMNTP1Yj+thvbuLSfZtjhVjQPdr2SNQ8HobslBxC/FdNekg1ASW 4nug== X-Forwarded-Encrypted: i=1; AJvYcCXXgheyS6PeYjYQZiFPIbDQSkKX72fxxzOaIHhsg5JsgCZbqvzuUE49CBTPOF+ESBQ24iOWB+skiwdxT74YGdIt3X6mKEH8I3IES7pkUgRchpwF X-Gm-Message-State: AOJu0YxeCOd1v52n/nQTTCUzl0bc4a2t4LPuy7cvIKkepqOtM86U4jFW qOb9eIHxEAYn7s3Rz6HbBzJ6EJRLd9LRzwIja0OM245rOHHE+fObIAz1Fq5r+AU= X-Google-Smtp-Source: AGHT+IHXWCcvy1AsuQ5cDBoUnMtJPLs0k0HnSBlO83IOeg/PxOKfDy/lz+6jOME45whX6mJIv8m+GA== X-Received: by 2002:a05:600c:3508:b0:414:273:67d2 with SMTP id h8-20020a05600c350800b00414027367d2mr306914wmq.13.1710488258477; Fri, 15 Mar 2024 00:37:38 -0700 (PDT) Received: from localhost.localdomain ([41.66.99.175]) by smtp.gmail.com with ESMTPSA id b8-20020a5d45c8000000b0033e7715bafasm2498086wrs.59.2024.03.15.00.37.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Mar 2024 00:37:38 -0700 (PDT) Message-ID: <4c76cdec1ce57463f69e2907d6580153759cdca4.camel@cybertec.at> Subject: Re: Seeing high query planning time on Azure Postgres Single Server version 11. From: Laurenz Albe To: hassan rafi , Ron Johnson Cc: Greg Sabino Mullane , Tom Lane , Robert Treat , pgsql-general@lists.postgresql.org Date: Fri, 15 Mar 2024 08:37:20 +0100 In-Reply-To: References: <1068645.1710007250@sss.pgh.pa.us> Autocrypt: addr=laurenz.albe@cybertec.at; prefer-encrypt=mutual; keydata=mQINBGGDwAQBEADgbWy5cKXQld3N2mF+DFyiNFbi2oBl2T+XgxpPF8wTRw2D/u4bBKXP0SYSE/lA86jIVNWWU0gf1KODIkVvgJm2w4vH2VBV1b7ddVViGl1Iu+9zaRnv9wulhnH42KefepXnoean6UT1EzLM0opF/Ik0j+40TxdRtobkBprkQUyHDXWlHc2ffPs3SipyFEP9AVLf7ejRC46CXWDnsqjOBSMEW8Z4HiK/8RrPZBsKLts8dJxKF4pygOdJb0CWk8k/X1jbcfdxo+zOLjOMvJcSJ2pFdJmQHU+JufB3rePziqQ2S9Ur6sccr9XnTC1GVBWN4Lf5VHq+vf+bFJjVwg+2hrySZnAVfcOrxoqFLErr7ug1zN2nM1kcpgA4VWn4gxlJtYNYYq+9WxX5dtvnNANlG3ZCrRKQzl8lxtzoF6Zo7LUhEqPaHDwn7Rvs+IdbOn41lF5UDTJGqmC4gS/bZydW2Fy3YWm4aSaN9fgFf8D+PVkrlKAZB7gBLz1TyHjbcRf85cYF+GKKrDld5SzMB/V60VX3oP/Eo8ikFpyWaqiz1f9X7MBot3/PjJkY+wDzp3nmb19QEcOBuQiSQ4xds2r0HewbuHTAR68u8jNNMGmpm2j4x+g09Jd/WQDjqlTBZ/jEltH41fYCCPWMfljXTOOXu2eLNGdfi7ETZogtwjM9oTtSPQARAQABtCdMYXVyZW56IEFsYmUgPGxhdXJlbnouYWxiZUBjeWJlcnRlYy5hdD6JAk4EEwEIADgWIQR0CqhbZGGABqoaSbdi8bhXA2EdmAUCYYPABAIbAwULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRBi8bhXA2EdmM/6EADK232JCwmBzhlj8h7U9CjG6kx0JHP3uJGv+XfsHtHAlmY/RCwF1BHMEsRlk bT5UrLvJ2jb99bA9QARzhFaxzyn0F/BUKzuIjRGNs/n6d5dNUFA0kOt8sX+TacmC GEyjEBCrVCm4ranBiUyePn9NhHNWnaex7pJyqvMLLdwW9BEMJx0Fqo+DN8ukbXmYRsmhEtd3ue+x/luYmOmJnaGtzInaY5aOJYbW9XqoRIZkZvOCgbi1FfvNmoqWa+3oVxTOgw9RafjJDyW0lTHzKGjbGI5ofMU98l+/hKJFYJqWUF6VpFJY5YIcN/1lf4ZICMwDl+MPIVo/tpq8L10seJL28nLlvw3K+cI+TVW8IW/qL/LyVoDofI3USeOORuYmhpWRhik8JXX6xf3v6GrRilJIPWNFIJbxm1ZblQiQnOw3IOW7T+8nAmPin1HKqM3VrOrJQ2VtShsefNBibNAsr1oFaqcDBkn3yGG8i6CTW+FyO4PZ+/EwNxMVgktxbYdy5AT1/lpXr5tB+phhLIyVfiBvrWs5EThxYMQ/L8Y85c3GMsAy1l/x4h3jqySIYy3SCU9+jc5UVuNnXljbvkEzJ+NLWJ6C1rACFWrMszgPdh5tCrlRY9PpmYll4JbCgb8BtxEIUmR+xr50/ZElEK5iml7Q00KUekCcDt+36PsyGFTXBzNOrkCDQRhg8AEARAAzOZ2tLHlI4rrhG411h6cdCFjBZxuljaFCxFyHn3m6wbGLqwBUWC5k8UrRqjHMz88KcTSaNO7XGAmCqPdWd2SeflPZRnNTbjsVpw7mLdffsBm4JX7kki2Pvk5h0NtYeidXT1PSpc2ri4DutYXuT9uD8RAm1wUDCE5HQNUihT/WH6opt+hskHW21uHao0+y822tG0QQcGMqdQR5Vxdxj89wiEPdqW+HpU/oOZIhrf2E7prduAppxixjHy/o1rcnoznnJvc8D3+YgI9O0LrBMij89dM55pRGbLovTR1oGR3U74sX774+0xmSzeIKwZfiMUz7Atlvfk5SHOsRUFPN2Ux9kaXiiBibQpHFxt7b lDrT4wxdLJ/XCdbPPAyl+lZtOLsaHEEZvYNyTXwZc35dVf3R4/oz20HoG6s7ct8e1 AQygj43XAERzty9SkWgxs8+grp1PrGx6FHVSYRqBM8dS/ZR6yRVwOwJXPyaSSqfIF21DkE4j1y4n+ItSewPGoRp8K/yWCikt6qlkVkO2ASNIiX04fAbtzwVOaNn8ZMRNqyvLc1fED4sr49onE4cAIcBLjcC3KL+w9DUGRQCdziROj5H2Yl/sXGPdMciUHo/Uz2rggc+2th3bQiMhrHWSsBpUkDQp0yWewemstPpPgBL3h2fHKaX8B9oH5Qu/H1IgrOuX8AEQEAAYkCNgQYAQgAIBYhBHQKqFtkYYAGqhpJt2LxuFcDYR2YBQJhg8AEAhsMAAoJEGLxuFcDYR2YuPwQAMkpGtR80pQ1gVsONhdkqj0H2eU66efP/gO3CoyaoIcvrpKYj7C2HipVSmkt1gpByL0X4AMQ/vKuknUz3wd28Ba+G1dCfbVs/Xiusq+SmpUj5rTwmYqdSjWMuCo1R6oS5hdJMdUUJYGMT0QkVlm1KnW8jkmCTl9GzjDxOAsN9O6/6lPzaGFtk9XF+34Bry/N4HKiJkqpC4+UTd0AprPfzJ2jdT64e1F0+W88X8y1bTTgNrHwK4mDiLnlE4SKRuEm54lNhJz//ar86Or5BErzNpM6TL7lk44QS06hwsMrEdKIy8J/SYJPjfzR8tIUnKscclVpOgjKaBqC+0iFiVaRqAgfOlIEiezX6kMh5Q2FIUfqs46qWhhXjRrdKOEoStYAaikdLu5ZXr7vfb0ZaDh+ZwTQtbSMFolyOkecwI81MCdbMfT/1TqIGTOdAj5as9fAakk0jb2pXgUYQ8X1DVTR8ahSDVEaw9VTmWiSvTxvguVJ1Mb7gG4Gmh6aviDTJhfXtH4rPUNXhDLqrTH8JkJjyKROOMakIF68Hjse5vUfUxreBEOtb5r1Coa2Fe7ncJayaSE7ryrDbFqpZ 36UMAx4ulWMyqJajLNGY0DdG8qIsR5nxRhrnK/mrCidZ8F9/D3bWAl4rjtHlsztN59 +AnW5l0HsQcY9ntFL/zEBOaonjdJf Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.50.4 (3.50.4-1.fc39) MIME-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Fri, 2024-03-15 at 00:31 +0530, hassan rafi wrote: > We have migrated to postgres version 16.1, but still due to very high upd= ate > activity on our DB, we are seeing elevated response times, though now the > planning time is less. >=20 > catalog-v2=3D> explain (analyze, verbose, settings, buffers) SELECT produ= cts_inventory_delta.upc FROM products_inventory_delta WHERE products_invent= ory_delta.modality =3D 'pickup' AND products_inventory_delta.store_id =3D '= 70600372' ORDER BY upc DESC LIMIT 51 OFFSET 30000; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 QUERY PLAN =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 > -------------------------------------------------------------------------= ---------------------------------------------------------------------------= ---------------------------------------------- > =C2=A0Limit=C2=A0 (cost=3D1450.68..1450.73 rows=3D1 width=3D14) (actual t= ime=3D5049.115..5049.116 rows=3D0 loops=3D1) > =C2=A0=C2=A0 Output: upc > =C2=A0=C2=A0 Buffers: shared hit=3D33359 read=3D6590 dirtied=3D9379 > =C2=A0=C2=A0 ->=C2=A0 Index Only Scan Backward using products_inventory_d= elta_pkey on public.products_inventory_delta=C2=A0 (cost=3D0.57..1450.68 ro= ws=3D28606 width=3D14) (actual time=3D1.056..5047.472 rows=3D28299 loops=3D= 1) > =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 Output: upc > =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 Index Cond: ((products_inventory_delta.= store_id =3D '70600372'::text) AND (products_inventory_delta.modality =3D '= pickup'::modality)) > =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 Heap Fetches: 16840 > =C2=A0=C2=A0 =C2=A0 =C2=A0 =C2=A0 Buffers: shared hit=3D33359 read=3D6590= dirtied=3D9379 > =C2=A0Settings: effective_cache_size =3D '192GB', jit =3D 'off', random_p= age_cost =3D '2', work_mem =3D '2097151kB' > =C2=A0Query Identifier: 220372279818787780 > =C2=A0Planning Time: 0.062 ms > =C2=A0Execution Time: 5049.131 ms Your problem are probably the "Heap Fetches: 16840". If you VACUUM the table, the performance should improve. The best solution is to make sure that autovacuum processes that table more= often: ALTER TABLE public.products_inventory_delta SET (autovacuum_vacuum_scale_= factor =3D 0.01); Yours, Laurenz Albe