pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Justin Pryzby <pryzby@telsasoft.com>
To: Jeff Davis <pgsql@j-davis.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Melanie Plageman <melanieplageman@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Taylor Vesely <tvesely@pivotal.io>
Cc: Adam Lee <ali@pivotal.io>
Cc: Melanie Plageman <mplageman@pivotal.io>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Memory-Bounded Hash Aggregation
Date: Thu, 19 Mar 2020 01:42:22 -0500
Message-ID: <20200319064222.GR26184@telsasoft.com> (raw)
In-Reply-To: <b2af57f4f8a27b34a0efb80ce823f5cd2bbb9164.camel@j-davis.com>
References: <20200219191636.gvdywx32kwbix6kd@development>
	<20200221202212.ssb2qpmdgrnx52sj@alap3.anarazel.de>
	<9759ab2d0c2c28a175e11efe1788d37dc7e1b768.camel@j-davis.com>
	<20200222180020.ej5gvfcaf2semgji@alap3.anarazel.de>
	<5c371df7cee903e8cd4c685f90c6c72086d3a2dc.camel@j-davis.com>
	<20200224232928.45bcjmdxgnobijgi@alap3.anarazel.de>
	<6e7c269b9a84ff75fefcad8ab2d4758f03581e98.camel@j-davis.com>
	<bd6edb713f0e3c9ecb7f08a6b4bb3f469c6cf152.camel@j-davis.com>
	<20200312210146.GG29065@telsasoft.com>
	<b2af57f4f8a27b34a0efb80ce823f5cd2bbb9164.camel@j-davis.com>

On Sun, Mar 15, 2020 at 04:05:37PM -0700, Jeff Davis wrote:
> > +	if (from_tape)
> > +		partition_mem += HASHAGG_READ_BUFFER_SIZE;
> > +	partition_mem = npartitions * HASHAGG_WRITE_BUFFER_SIZE;
> > 
> > => That looks wrong ; should say += ?
> 
> Good catch! Fixed.

> +++ b/src/backend/executor/nodeAgg.c
> @@ -2518,9 +3499,36 @@ ExecInitAgg(Agg *node, EState *estate, int eflags)
>  	 */
>  	if (use_hashing)
>  	{
> +		Plan   *outerplan = outerPlan(node);
> +		uint64	totalGroups = 0;
> +		for (i = 0; i < aggstate->num_hashes; i++)
> +			totalGroups = aggstate->perhash[i].aggnode->numGroups;
> +
> +		hash_agg_set_limits(aggstate->hashentrysize, totalGroups, 0,

I realize that I missed the train but .. that looks like another += issue?

Also, Andres was educating me about the range of behavior of "long" type, and I
see now while rebasing that you did the same thing.
https://www.postgresql.org/message-id/20200306175859.d56ohskarwldyrrw%40alap3.anarazel.de

-- 
Justin





view thread (81+ messages)  latest in thread

Message-ID: <20200319064222.GR26184@telsasoft.com>
Permalink:  ../20200319064222.GR26184@telsasoft.com/
Also on:    postgresql.org/message-id/20200319064222.GR26184@telsasoft.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: pryzby@telsasoft.com, pgsql@j-davis.com, andres@anarazel.de, tomas.vondra@2ndquadrant.com, melanieplageman@gmail.com, hlinnaka@iki.fi, tvesely@pivotal.io, ali@pivotal.io, mplageman@pivotal.io
  Subject: Re: Memory-Bounded Hash Aggregation
  In-Reply-To: <20200319064222.GR26184@telsasoft.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox