Subject: Re: Strange nested loop for an INSERT
To: phb07 <phb07@apra.asso.fr>, <pgsql-performance@postgresql.org>
References: 
 <CD1B112115AFCF46A0EB8CFD8EE88B461E2B19D2@FRCRPVV9EX4MSX.ww931.my-it-solutions.net>
 <1f22a3fa-dcc5-eeac-a708-7f10fada4674@apra.asso.fr>
 <25256.1473691295@sss.pgh.pa.us>
 <224290ce-84bb-935f-1743-3f036ae3d508@apra.asso.fr>
 <06b366f8-7e3d-ec7d-d146-1984818d3bcc@BlueTreble.com>
 <ac51c182-ae29-7324-6833-df69ed718dd0@apra.asso.fr>
From: Jim Nasby <Jim.Nasby@BlueTreble.com>
Message-ID: <ed90f8f1-3298-8758-bc4e-9bb8abc90256@BlueTreble.com>
Date: Fri, 23 Sep 2016 13:51:15 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0)
 Gecko/20100101 Thunderbird/45.3.0
MIME-Version: 1.0
In-Reply-To: <ac51c182-ae29-7324-6833-df69ed718dd0@apra.asso.fr>
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: None (protection.outlook.com: bluetreble.com does not designate
 permitted sender hosts)
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2016 18:51:19.4592 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1635
X-Pg-Spam-Score: -1.8 (-)
List-Archive: <http://archives.postgresql.org/pgsql-performance>
List-Help: <mailto:majordomo@postgresql.org?body=help>
List-ID: <pgsql-performance.postgresql.org>
List-Owner: <mailto:pgsql-performance-owner@postgresql.org>
List-Post: <mailto:pgsql-performance@postgresql.org>
List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-performance>
List-Unsubscribe: 
 <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
X-Mailing-List: pgsql-performance
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

On 9/23/16 12:59 PM, phb07 wrote:
>
> Le 21/09/2016 à 23:42, Jim Nasby a écrit :
>> On 9/12/16 1:05 PM, phb07 wrote:
>>> The drawback is the overhead of this added ANALYZE statement. With a
>>> heavy processing like in this test case, it is worth to be done. But for
>>> common cases, it's a little bit expensive.
>>
>> You could always look at the number of rows affected by a command and
>> make a decision on whether to ANALYZE based on that, possibly by
>> looking at pg_stat_all_tables.n_mod_since_analyze.
> I have solved the issue by adding an ANALYZE between both statements. To
> avoid the associated overhead for cases when it is not worth to be done,
> the ANALYZE is only performed when more than 1000 rows have just been
> deleted by the first statement (as the logic is embeded into a plpgsql
> function, the GET DIAGNOSTICS statement provides the information). This
> threshold is approximately the point where the potential loss due to bad
> estimates equals the ANALYZE cost.
> But the idea of using the n_mod_since_analyze data to also take into
> account other recent updates not yet reflected into the statistics is
> very interesting.

Another interesting possibility would be to look at 
pg_catalog.pg_stat_xact_all_tables; if you add n_tup_ins, _upd, and _del 
that will tell you how much n_mod_since_analyze will be increased when 
your transaction commits, so you could guage exactly how much the 
current transaction has changed things.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)   mobile: 512-569-9461


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance