Message-ID: <65377ba9-dd24-e4c8-83a3-60cba8b22bf6@gmail.com>
Date: Fri, 7 Jul 2023 16:00:01 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Subject: Re: benchmark results comparing versions 15.2 and 16
Content-Language: en-US
From: Alexander Lakhin <exclusion@gmail.com>
To: David Rowley <dgrowleyml@gmail.com>
Cc: MARK CALLAGHAN <mdcallag@gmail.com>, pgsql-hackers@lists.postgresql.org,
 Andres Freund <andres@anarazel.de>
References: <CAFbpF8N4bVFZ+eyWFj_+fZXZFSayk2kJVtZnouUVKGHudiDrSw@mail.gmail.com>
 <b32bed1b-0746-9b20-1472-4bdc9ca66d52@gmail.com>
 <3d89174a-9b11-1df1-6a7d-ea64190ad26d@gmail.com>
 <CAApHDvqeYkHJEkh0unC5OsJDDY9o2ZfwPFdQDFdqYxCYOXgLFw@mail.gmail.com>
 <c152c484-3c6c-3edf-560f-3c2dcfd71f5a@gmail.com>
In-Reply-To: <c152c484-3c6c-3edf-560f-3c2dcfd71f5a@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/65377ba9-dd24-e4c8-83a3-60cba8b22bf6%40gmail.com>
Precedence: bulk

Hello David,

11.05.2023 16:00, Alexander Lakhin wrote:
> Yeah, I see. It's also interesting to me, which tests perform better after
> that commit. It takes several hours to run all tests, so I can't present
> results quickly, but I'll try to collect this information next week.

To my regret, I could not find such tests that week, so I decided to try
later, after reviewing my benchmark running infrastructure.

But for now, as Postgres Pro company graciously shared the benchmarking
infrastructure project [1], it's possible for anyone to confirm or deny my
results. (You can also see which tests were performed and how.)

Having done one more round of comprehensive testing, I still couldn't find
winning tests for commit 3c6fc5820 (but reassured that
test s64da_tpcds.query87 loses a little (and also s64da_tpcds.query38)).
So it seems to me that the tests I performed or their parameters is not
representative enough for that improvement, unfortunately.

[1] https://github.com/postgrespro/pg-mark

Best regards,
Alexander