From: Joshua D. Drake <jd@commandprompt.com>
To: Andres Freund <andres@anarazel.de>
Cc: Josh berkus <josh@agliodbs.com>
Cc: Robert Haas <robertmhaas@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: Reviewing freeze map code
Date: Fri, 6 May 2016 14:39:57 -0700
Message-ID: <572D0F2D.7050207@commandprompt.com> (raw)
In-Reply-To: <20160506212958.wwwc22hfesvj76af@alap3.anarazel.de>
References: <CA+TgmobF=86sa=6dF30jaiEFOhZnO+asOB5j69PynPCqtaz9xA@mail.gmail.com>
<572D0309.4090402@commandprompt.com>
<20160506205012.ipuhvctss2ozzs6m@alap3.anarazel.de>
<572D0471.3040009@commandprompt.com>
<20160506205824.ysst7mk3k26ztok2@alap3.anarazel.de>
<WM!3c465a785ef366521ed338bb93585bc99a43cc6db976bb0e71111a522ee7f39a31942de6526cb0bccd5f2b7d1678818f!@asav-3.01.com>
<572D062E.5030308@agliodbs.com>
<572D068F.3050608@commandprompt.com>
<20160506210806.f6pnk6ueirjqrlmu@alap3.anarazel.de>
<572D09D9.5080407@commandprompt.com>
<20160506212958.wwwc22hfesvj76af@alap3.anarazel.de>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On 05/06/2016 02:29 PM, Andres Freund wrote:
> Hi,
>
>
> On 2016-05-06 14:17:13 -0700, Joshua D. Drake wrote:
>> How do I test?
>>
>> Is there a script I can run?
>
> Unfortunately there's few interesting things to test with pre-made
> scripts. There's no relevant OS dependency here, so each already
> existing test doesn't really lead to significantly increased coverage
> being run by other people. Generally, when testing for correctness
> issues, it's often of limited benefit to run tests written by the author
> of reviewer - such scripts will usually just test things either has
> thought of. The dangerous areas are the ones neither author or reviewer
> has considered.
I can't argue with that.
>
>
>> Are there specific things I can do to try and break it?
>
> Upgrade clusters using pg_upgrade and make sure things like index only
> scans still work and yield correct data. Set up workloads that involve
> freezing, and check that less WAL (and not more!) is generated with 9.6
> than with 9.5. Make sure queries still work.
>
>
>> What are we looking for exactly?
>
> Data corruption, efficiency problems.
>
I am really not trying to be difficult here but Data Corruption is an
easy one... what is the metric we accept as an efficiency problem?
>
>> A lot of -hackers seem to forget that although we have 100 -hackers, we have
>> 10000 "consultant/practitioners". Could I read the code and with a weekend
>> of WTF and -hackers questions figure out what is going on, yes but a lot of
>> people couldn't and I don't have the time.
>
> I think tests without reading the code are quite sensible and
> important. And it perfectly makes sense to ask for information about
> what to test. But fundamentally testing is a lot of work, as is writing
> and reviewing code; unless you're really really good at destructive
> testing, you won't find much in a 15 minute break.
>
Yes, this is true but with a proper testing framework, I don't need a 15
minute break. I need 1 hour to configure, the rest just "happens" and
reports back.
I have cycles to test, I have team members to help test (as do *lots* of
other people) but sometimes we just get lost in how to help.
>
>> You want me (or people like me) to test more? Give us an easy way to
>> do it.
>
> Useful additional testing and easy just don't go well together. By the
> time I have made it easy I've done the testing that's needed.
I don't know that I can agree with this. A proper harness allows you to
execute: go.sh and boom... 2, 4, even 8 hours later you get a report. I
will not argue that it isn't easy to implement but I know it can be done.
>> Otherwise, we do what we can, which is try and interface on the things that
>> will directly and immediately affect us (like keywords and syntax).
>
> The amount of bikeshedding on -hackers steals energy and time for
> actually working on stuff, including testing. So I have little sympathy
> for the amount of bike shedding done.
Insuring a reasonable and thought out interface for users is not bike
shedding, it is at least as important and possibly more important than
any feature we add.
Sincerely,
JD
>
> Greetings,
>
> Andres Freund
>
--
Command Prompt, Inc. http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: jd@commandprompt.com, andres@anarazel.de, josh@agliodbs.com, robertmhaas@gmail.com, sawada.mshk@gmail.com
Subject: Re: Reviewing freeze map code
In-Reply-To: <572D0F2D.7050207@commandprompt.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox