Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sYHYO-00AG2k-Sg for pgsql-hackers@arkaria.postgresql.org; Mon, 29 Jul 2024 04:00:08 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1sYHYN-00EqB4-DY for pgsql-hackers@arkaria.postgresql.org; Mon, 29 Jul 2024 04:00:07 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sYHYN-00EqAv-4R for pgsql-hackers@lists.postgresql.org; Mon, 29 Jul 2024 04:00:07 +0000 Received: from mail-lf1-x12c.google.com ([2a00:1450:4864:20::12c]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sYHYK-001yKu-Fv for pgsql-hackers@postgresql.org; Mon, 29 Jul 2024 04:00:05 +0000 Received: by mail-lf1-x12c.google.com with SMTP id 2adb3069b0e04-52f00ad303aso4762668e87.2 for ; Sun, 28 Jul 2024 21:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1722225603; x=1722830403; darn=postgresql.org; h=content-transfer-encoding:subject:cc:to:from:content-language :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=2vCq5nKh1mxk8eph15UxylDTFwGnMcWWL4l6pli7Tgs=; b=F6KUduuC/ogDADcNALJQD9cswY6HWdtgduFxGIK0+B6UA08piKEcHimos/UzOLdS5W eKCHLIb1ilWrQyBlyY0R9DL4W1VL9rHKpKiZfTGDD/dfuzZbH5PLyBsI8UiHIEnyGuwq JyBZr6EoiUbZOi+/fOVbvh3qi7f383Yxb6l2DdkP+OnHiehm2KTiRFMXmlbj4NMr1LYI ddgD5LkE017YBufJqn+GiBUTP3mkxl1nL7c4Rfas/fhq6ryDHn/EdpuxuAwgnIx/4ZG6 hi+Z6hxvpYBYxqD4A02PKX06VLDZaxg9Hi9vHcAaAAp477EjngbJZtf0MQ4wuIKfw1az a2MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722225603; x=1722830403; h=content-transfer-encoding:subject:cc:to:from:content-language :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=2vCq5nKh1mxk8eph15UxylDTFwGnMcWWL4l6pli7Tgs=; b=j/+psiX+r+tNINOdlX3G7v4pqHMxL3fqTWiUEhUhOZDKvRPhBCO0e4LD7OkZXJBzE9 g+6ShDYI6Pa035bAsvF5Iylujbr32i10nmv8MliHCufkPKpbZlCPPMOO2hDfF3jjk2Vf jU7aHXDMIPAz/Zd8uQiNs8JlEObC4LACgbRZcjSR3jtZWriXN+BvXgPyfOTA77h75dy8 2G7y3OOKt8k6ZM3eNJEW+nUKCWoRBKXMiMgg76wasKjYN6hp5DyMlHzgwR9GkBDyD53A g2s4N0etNId1dkZDXW+3KyHAKpkSP/Vaq31zi9G+5/djiyEmf5A+x3KR/a4WAeXaPtcn 3TAA== X-Gm-Message-State: AOJu0YxrC0NQnO28iOUy2EilD00+LpwXmvMauk79WnLkXb9wThGodJsV MW0i3ZGg2LgFdvvdzvEWjL9doGnxxY9tzJnWAJCNVzAXgEdCddnh X-Google-Smtp-Source: AGHT+IFmLWJRZ3W++eKwR9M4FX0Ej0nwUw9UlL9EKQtIVcWTpMkI0HPXA1LVt2URiDLM5CdDmapLFA== X-Received: by 2002:a05:6512:2c06:b0:52f:cdf3:2934 with SMTP id 2adb3069b0e04-5309b259af0mr3882896e87.8.1722225602237; Sun, 28 Jul 2024 21:00:02 -0700 (PDT) Received: from [1.0.0.7] ([178.155.31.111]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-52fd5c2d111sm1302490e87.279.2024.07.28.21.00.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 28 Jul 2024 21:00:01 -0700 (PDT) Message-ID: <877b1f23-35d2-31b2-2fcd-d176fd3d05c4@gmail.com> Date: Mon, 29 Jul 2024 07:00:00 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Content-Language: en-US From: Alexander Lakhin To: Tomas Vondra Cc: pgsql-hackers Subject: dikkop failed the pg_combinebackupCheck/006_db_file_copy.pl test Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hello Tomas, Please take a look at a recent dikkop's failure [1]. The regress_log_006_db_file_copy file from that run shows: [02:08:57.929](0.014s) # initializing database system by copying initdb template ... [02:09:22.511](24.583s) ok 1 - full backup ... [02:10:35.758](73.247s) not ok 2 - incremental backup 006_db_file_copy_primary.log contains: 2024-07-28 02:09:29.441 UTC [67785:12] 006_db_file_copy.pl LOG: received replication command: START_REPLICATION SLOT "pg_basebackup_67785" 0/4000000 TIMELINE 1 2024-07-28 02:09:29.441 UTC [67785:13] 006_db_file_copy.pl STATEMENT:  START_REPLICATION SLOT "pg_basebackup_67785" 0/4000000 TIMELINE 1 2024-07-28 02:09:29.441 UTC [67785:14] 006_db_file_copy.pl LOG: acquired physical replication slot "pg_basebackup_67785" 2024-07-28 02:09:29.441 UTC [67785:15] 006_db_file_copy.pl STATEMENT:  START_REPLICATION SLOT "pg_basebackup_67785" 0/4000000 TIMELINE 1 2024-07-28 02:10:29.487 UTC [67785:16] 006_db_file_copy.pl LOG: terminating walsender process due to replication timeout 2024-07-28 02:10:29.487 UTC [67785:17] 006_db_file_copy.pl STATEMENT:  START_REPLICATION SLOT "pg_basebackup_67785" 0/4000000 TIMELINE 1 It looks like this incremental backup operation was performed slower than usual (it took more than 60 seconds and apparently was interrupted due to wal_sender_timeout). But looking at regress_log_006_db_file_copy from the 6 previous (successful) test runs, we can see: [14:22:16.841](43.215s) ok 2 - incremental backup [02:14:42.888](34.595s) ok 2 - incremental backup [17:51:16.152](43.708s) ok 2 - incremental backup [04:07:16.757](31.087s) ok 2 - incremental backup [12:15:01.256](49.432s) ok 2 - incremental backup [01:06:02.482](52.364s) ok 2 - incremental backup Thus reaching 60s (e.g., due to some background activity) on this animal seems pretty possible. So maybe it would make sense to increase wal_sender_timeout for it, say, to 120s? [1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=dikkop&dt=2024-07-27%2023%3A22%3A57 Best regards, Alexander