pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: tests: Consistently use pg_basebackup -cfast --no-sync to accele
3+ messages / 2 participants
[nested] [flat]

* pgsql: tests: Consistently use pg_basebackup -cfast --no-sync to accele
@ 2022-01-17 23:48  Andres Freund <andres@anarazel.de>
  0 siblings, 1 reply; 3+ messages in thread

From: Andres Freund @ 2022-01-17 23:48 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

tests: Consistently use pg_basebackup -cfast --no-sync to accelerate tests.

Most tests invoking pg_basebackup themselves did not yet use -cfast, which
makes pg_basebackup take considerably longer. The only reason this didn't
cause the tests to take many minutes is that spread checkpoints only throttle
when writing out a buffer and there aren't that many dirty buffers in the
tests...

Discussion: https://postgr.es/m/20220117195711.xx4qbxutrrlmo2dg@alap3.anarazel.de

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/839f9636b374d67952fdb2b048f055393bc5a8f4

Modified Files
--------------
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 104 ++++++++++++++-------------
src/bin/pg_verifybackup/t/002_algorithm.pl   |   2 +-
src/bin/pg_verifybackup/t/003_corruption.pl  |   2 +-
src/bin/pg_verifybackup/t/004_options.pl     |   2 +-
src/bin/pg_verifybackup/t/006_encoding.pl    |   2 +-
src/bin/pg_verifybackup/t/007_wal.pl         |   4 +-
6 files changed, 61 insertions(+), 55 deletions(-)



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: pgsql: tests: Consistently use pg_basebackup -cfast --no-sync to accele
@ 2022-01-18 01:18  Michael Paquier <michael@paquier.xyz>
  parent: Andres Freund <andres@anarazel.de>
  0 siblings, 1 reply; 3+ messages in thread

From: Michael Paquier @ 2022-01-18 01:18 UTC (permalink / raw)
  To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers@lists.postgresql.org

Hi Andres,

On Mon, Jan 17, 2022 at 11:48:02PM +0000, Andres Freund wrote:
> tests: Consistently use pg_basebackup -cfast --no-sync to accelerate tests.
> 
> Most tests invoking pg_basebackup themselves did not yet use -cfast, which
> makes pg_basebackup take considerably longer. The only reason this didn't
> cause the tests to take many minutes is that spread checkpoints only throttle
> when writing out a buffer and there aren't that many dirty buffers in the
> tests...

Are you planning to back-patch any of this?  It seems to me that it
would not hurt, and making the tests faster on back-branches saves
time when working on those branches.

Thanks,
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: pgsql: tests: Consistently use pg_basebackup -cfast --no-sync to accele
@ 2022-01-18 01:28  Andres Freund <andres@anarazel.de>
  parent: Michael Paquier <michael@paquier.xyz>
  0 siblings, 0 replies; 3+ messages in thread

From: Andres Freund @ 2022-01-18 01:28 UTC (permalink / raw)
  To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-committers@lists.postgresql.org

Hi,

On 2022-01-18 10:18:36 +0900, Michael Paquier wrote:
> On Mon, Jan 17, 2022 at 11:48:02PM +0000, Andres Freund wrote:
> > tests: Consistently use pg_basebackup -cfast --no-sync to accelerate tests.
> > 
> > Most tests invoking pg_basebackup themselves did not yet use -cfast, which
> > makes pg_basebackup take considerably longer. The only reason this didn't
> > cause the tests to take many minutes is that spread checkpoints only throttle
> > when writing out a buffer and there aren't that many dirty buffers in the
> > tests...
> 
> Are you planning to back-patch any of this?  It seems to me that it
> would not hurt, and making the tests faster on back-branches saves
> time when working on those branches.

I hadn't planned to do so. Back-branch tests don't run even close to as often
and there's enough difference across the branches that it's a bit of
per-branch work to adjust. To me this mostly seems helpful to reduce test
times on buildfarm critters / CI and branches we constantly run the tests for.

Greetings,

Andres Freund





^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2022-01-18 01:28 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-01-17 23:48 pgsql: tests: Consistently use pg_basebackup -cfast --no-sync to accele Andres Freund <andres@anarazel.de>
2022-01-18 01:18 ` Michael Paquier <michael@paquier.xyz>
2022-01-18 01:28   ` Andres Freund <andres@anarazel.de>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox