Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ltdVP-0006aT-Q3 for pgsql-hackers@arkaria.postgresql.org; Wed, 16 Jun 2021 21:55:27 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1ltdVO-0002nq-Pj for pgsql-hackers@arkaria.postgresql.org; Wed, 16 Jun 2021 21:55:26 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ltdVO-0002lm-Eg for pgsql-hackers@lists.postgresql.org; Wed, 16 Jun 2021 21:55:26 +0000 Received: from mail-ed1-x535.google.com ([2a00:1450:4864:20::535]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1ltdVL-0004jI-9t for pgsql-hackers@postgresql.org; Wed, 16 Jun 2021 21:55:26 +0000 Received: by mail-ed1-x535.google.com with SMTP id d7so1179015edx.0 for ; Wed, 16 Jun 2021 14:55:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=Nm9adSSfnoT8EorJtuDv2XijycA95h5Sp5iTuQdocho=; b=p3aUBwU/NK/ojz9PIVYo3RbVIiJcYCp5KlTuZ25AESZeRmIDJpR7trzapkrJQe+tlW pSq+9ag3DW1TdtM3lVfBNXFMWu1bvJBkNGo3mevTB7Df7498iMFFhw2dmIBJfIKRb2rv qte71lhI17mVQOkMyBN2E9SG0slecvlhilYGaWmPVQOmvfMePJmmuagubKa9BEM2gS7P MYQu7ReLbSUvFNkwp7fSaQ2oroNiclES32/eK/Ks5vPq/Xz2I8QtjSJCzyHJ/HlsHG+5 o+e3ER19goMHDjdU0+syxHzwkjHU4L0pG12a/YeSu+qpDxKOuRARbB0dOHdndSyBRvzs 4SXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Nm9adSSfnoT8EorJtuDv2XijycA95h5Sp5iTuQdocho=; b=oTZC+Hqijd95vwI6iTvdS/6jzw8SIPYjIGpIvd0OuAJ635icHeLggBzNJirHLJpCDB /BXnH2f0ogQ+Qw4JB/XPp3UacoIfGZKQ7mCUpjVWKCOIdBfgSkMOWwX/xReXUU5uJ5p7 s5AjXUJcUJr2nl1oWm82vc8e8rvL0L7+vT8mKmrA7z9H9oloYjF8p/FZ0q165HInc5Lc n8dh0ZUNhQiW8fje6hWQ3qtFUFpJ48TPTYjHzTK/FT560NbhEUdFWwqrzvUls9te+pA4 n8yKBFY8xlOSZ/8W2N3du87WmJiY9pVI3HU4Q9Rf9y24wKYz559NIKQglX0KO8xwoJx9 gVpA== X-Gm-Message-State: AOAM53208uR64W4sBqvHV3VwPaVOWaMMZEoAIAPAu265ugg160/Suswx mnMh4mv97u+pdNd/Ja3BNgMnC1LkAibeg4OHTAg5dZPIjvLAN7c9teQGyKS8baVnR9b2Ou9Usca 3XHfUkh6YOwpnUapAOW+fivUXV7CUDJJcL4lT0f1ImVFHJdtb8N/at1xzuzdTLkrT6xx1nxwCLn zx0HzGovL3sFasHAMwNipAaYkRfzb+JaZEGzsvu6IseH5OdSGcomk3ZbEoKNnqqdZ3JtFOh7UGO PW/onxBeXce7vSvgafn8937yuCoTs++p7DsMXQ6 X-Google-Smtp-Source: ABdhPJy1BVtZ7dM8qNXBDVUSc5KqQO7jYGJxwOYNV6gCA5tWsdZQZ8dbQS13kp62S2MYoPewvHsk2A== X-Received: by 2002:a05:6402:b76:: with SMTP id cb22mr2272941edb.112.1623880521622; Wed, 16 Jun 2021 14:55:21 -0700 (PDT) Received: from [10.137.0.18] (ip-86-49-254-183.net.upcbroadband.cz. [86.49.254.183]) by smtp.gmail.com with ESMTPSA id c11sm2728143edr.80.2021.06.16.14.55.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Jun 2021 14:55:20 -0700 (PDT) Subject: Re: postgres_fdw batching vs. (re)creating the tuple slots From: Tomas Vondra To: Alexander Pyhalov Cc: Bharath Rupireddy , Tom Lane , Andres Freund , PostgreSQL-development References: <20210530205831.vib4p7mt23cymhpc@alap3.anarazel.de> <2399204.1622409059@sss.pgh.pa.us> <20210530212616.5k5nxnbunwnwet2a@alap3.anarazel.de> <2433405.1622416631@sss.pgh.pa.us> <4e7ebff8-59ac-c90c-3ecd-8d9dfdfc75ad@enterprisedb.com> <6dc1c07b-87e6-63e6-10b3-cca4698361fb@enterprisedb.com> <7a8cf8d56b3d18e5c0bccd6cd42d04ac@postgrespro.ru> Message-ID: <0079c3ee-4645-2cab-50a1-50d0d7152b66@enterprisedb.com> Date: Wed, 16 Jun 2021 23:55:17 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 6/16/21 4:23 PM, Tomas Vondra wrote: > On 6/16/21 2:36 PM, Alexander Pyhalov wrote: >> >> Hi. >> It seems this commit >> >> commit b676ac443b6a83558d4701b2dd9491c0b37e17c4 >> Author: Tomas Vondra >> Date:   Fri Jun 11 20:19:48 2021 +0200 >> >>      Optimize creation of slots for FDW bulk inserts >> >> has broken batch insert for partitions with unique indexes. >> > > Thanks for the report and reproducer! > > Turns out this is a mind-bogglingly silly bug I made in b676ac443b :-( > The data is copied into the slots only in the branch that initializes > them, so the subsequent batches just insert the same data over and over. > > The attached patch fixes that, and adds a regression test (a bit smaller > version of your reproducer). I'll get this committed shortly. > Pushed, after a bit more cleanup and testing. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company