Date: Tue, 10 Apr 2018 11:17:43 -0300
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Mark Rofail <markm.rofail@gmail.com>
Cc: David Steele <david@pgmasters.net>,
	Andreas Karlsson <andreas@proxel.se>,
	Alexander Korotkov <aekorotkov@gmail.com>,
	Erik Rijkers <er@xs4all.nl>,
	=?iso-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <hs@cybertec.at>,
	Robert Haas <robertmhaas@gmail.com>,
	Stephen Frost <sfrost@snowman.net>, Tom Lane <tgl@sss.pgh.pa.us>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] GSoC 2017: Foreign Key Arrays
Message-ID: <20180410141743.aphekyopkbre3avp@alvherre.pgsql>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: 
 <CAJvoCutzj34v_xfVtpQXNMEyYd8n13LSg2WCzHWhTqQhnB4ZzA@mail.gmail.com>
User-Agent: NeoMutt/20170306-137-4415bd-dirty (1.8.0)
Precedence: bulk

Mark Rofail wrote:
> On Tue, Apr 10, 2018 at 3:59 PM, Alvaro Herrera <alvherre@alvh.no-ip.org>
> wrote:
> >
> > documentation to it and a few extensive tests to ensure it works well);
> 
> I think the existing regression tests verify that the patch works as
> expectations, correct?

I meant for the GIN operator. (Remember, these are two patches, and each
of them needs its own tests.)

> We need more *exhaustive* tests to test performance, not functionality.

True.  So my impression from the numbers you posted last time is that
you need to run each measurement case several times, and provide
averages/ stddevs/etc for the resulting numbers, and see about outliers
(maybe throw them away, or maybe they indicate some problem in the test
or in the code); then we can make an informed decision about whether the
variations between the several different scenarios are real improvements
(or pessimizations) or just measurement noise.

In particular: it seemed to me that you decided to throw away the idea
of the new GIN operator without sufficient evidence that it was
unnecessary.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services