Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pH6VQ-0005r7-Qm for pgsql-hackers@arkaria.postgresql.org; Sun, 15 Jan 2023 17:09:17 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pH6VO-0004p3-4E for pgsql-hackers@arkaria.postgresql.org; Sun, 15 Jan 2023 17:09:14 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pH6VN-0004ou-DV for pgsql-hackers@lists.postgresql.org; Sun, 15 Jan 2023 17:09:13 +0000 Received: from mail-ej1-x630.google.com ([2a00:1450:4864:20::630]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pH6VF-0003yi-7a for pgsql-hackers@lists.postgresql.org; Sun, 15 Jan 2023 17:09:12 +0000 Received: by mail-ej1-x630.google.com with SMTP id mp20so16486449ejc.7 for ; Sun, 15 Jan 2023 09:09:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb.com; s=google; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=9cXr/U37uIyfEU3pUqcelJYyzwwqQUb9kyvAcfTEVdQ=; b=L74ApupwJzNtPpSCujQhK6wM8DWlWFtEDiRcFnv1vaILNa9xwQm98DaZjc95hLt638 4ca+//Rx3kSTatw0kdDdLvp4T8NXr4liwh7L+nnjsf3iHdPmbbOAYhRq0OpvtIwXnYrM jJ9TSgOW3DVbe1CYxsW69mynSqGZwbAvv2R6srMwrMJIu5BfJXlAqVagBHh6YTbtR4c+ D+NA2CddGtVr7rUxMUI7wElhCfvIfSv6i3dCF2PwCSfnJFh/ExqoHBNNDKITGqIoyb2E 5/UhluwFJ0BjLZsIPtzcu9zjXsdNwpQ0gG+GlSdIsBYgJn024lWvKTjrkhZt1Tg3D7PX IyDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9cXr/U37uIyfEU3pUqcelJYyzwwqQUb9kyvAcfTEVdQ=; b=sg2WG3VmgCxoKooOTIob89HxdhU6XkHm5zGbpGFcCvjvgzDQYHTDUQUhT3HIs5oVU7 MMR2JDPzydmKnlnF4uJMuQkbUYa/jHhIs0wwGA4eUfEOKSJKa5kPBm4PVrJsx76YF8JZ 7iHXN91GgAX0Oznk1qqxJT32dF+ciBp95+MsGIq+GLKQn29eb6e/Jo71kcl0fu+RgaQ4 YI2NHdh8YsND4XT4HbR+HaTOaVtn+YSd7i6YAV6wXAq0Xni2Kgz6tOtND+f1ah+pDRYc o2AU/0gKYNPF+12kf2vJUxTqzjvknw3Y6kpujb1PMoqU1G1TXDogOvevF7O5qnv+5rRG YueQ== X-Gm-Message-State: AFqh2krhaCltJQm0QasBCvB0z4O0tHDpQHC7HP4J0cZyJ2SFIM0TSa66 wBOWWTK1grsw9yfFIO5EjU+NITR26TR8LQXO7XI3y/FpGVyg8F5Ib+R1Br+QwltYP1AutU7bbeX tdtIYuwH+9bFdtLMJ8FcDuCsdzkmNqpoGxnduhHvMM3OXulFU8XI47LwXsuEfwlEwVdOTdMfkoc FiPKj1VJXSGUWLSCIbBLwG+06E5ly5K1EjvSzylcUFQm7ovobBLVaV8RgkjVw0W81vnxx4IXw5l IddPIMNJNDJ9WggNDggq1clUqCxLACgy154MsUC8dEG/CWp5JR0SYp9DhJy X-Google-Smtp-Source: AMrXdXsnlLo1IeFiiDcq7zSpcaLtE7VDqQz+BjDO3SP8fQm8tjt22H8Ei/vwZT0Y0g5xVL76ZR9BrQ== X-Received: by 2002:a17:907:1118:b0:86e:147c:9be1 with SMTP id qu24-20020a170907111800b0086e147c9be1mr4518961ejb.3.1673802543150; Sun, 15 Jan 2023 09:09:03 -0800 (PST) Received: from [10.137.0.17] (ip-86-49-228-162.bb.vodafone.cz. [86.49.228.162]) by smtp.gmail.com with ESMTPSA id c10-20020a17090618aa00b0084d14646fd9sm10715167ejf.165.2023.01.15.09.09.02 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 15 Jan 2023 09:09:02 -0800 (PST) Message-ID: <984ff689-adde-9977-affe-cd6029e850be@enterprisedb.com> Date: Sun, 15 Jan 2023 18:09:00 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Subject: Re: Perform streaming logical transactions by background workers and parallel apply To: Amit Kapila , "houzj.fnst@fujitsu.com" Cc: Masahiko Sawada , "wangw.fnst@fujitsu.com" , Peter Smith , "shiy.fnst@fujitsu.com" , PostgreSQL Hackers , Dilip Kumar References: Content-Language: en-US From: Tomas Vondra In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk Hi, I think there's a bug in how get_transaction_apply_action() interacts with handle_streamed_transaction() to decide whether the transaction is streamed or not. Originally, the code was simply: /* not in streaming mode */ if (!in_streamed_transaction) return false; But now this decision was moved to get_transaction_apply_action(), which does this: if (am_parallel_apply_worker()) { return TRANS_PARALLEL_APPLY; } else if (in_remote_transaction) { return TRANS_LEADER_APPLY; } and handle_streamed_transaction() then uses the result like this: /* not in streaming mode */ if (apply_action == TRANS_LEADER_APPLY) return false; Notice this is not equal to the original behavior, because the two flags (in_remote_transaction and in_streamed_transaction) are not inverse. That is, in_remote_transaction=false does not imply we're processing streamed transaction. It's allowed both flags are false, i.e. a change may be "non-transactional" and not streamed, though the only example of such thing in the protocol are logical messages. Which are however ignored in the apply worker, so I'm not surprised no existing test failed on this. So I think get_transaction_apply_action() should do this: if (am_parallel_apply_worker()) { return TRANS_PARALLEL_APPLY; } else if (!in_streamed_transaction) { return TRANS_LEADER_APPLY; } FWIW I've noticed this after rebasing the sequence decoding patch, which adds another type of protocol message with the transactional vs. non-transactional behavior, similar to "logical messages" except that in this case the worker does not ignore that. Also, I think get_transaction_apply_action() would deserve better comments explaining how/why it makes the decisions. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company