Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1l2P2H-0001KF-GP for pgsql-hackers@arkaria.postgresql.org; Thu, 21 Jan 2021 01:45:21 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1l2P2G-0002Pz-D5 for pgsql-hackers@arkaria.postgresql.org; Thu, 21 Jan 2021 01:45:20 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1l2P2G-0002Pp-6G for pgsql-hackers@lists.postgresql.org; Thu, 21 Jan 2021 01:45:20 +0000 Received: from mail-ej1-x633.google.com ([2a00:1450:4864:20::633]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1l2P2D-0002mx-UD for pgsql-hackers@postgresql.org; Thu, 21 Jan 2021 01:45:19 +0000 Received: by mail-ej1-x633.google.com with SMTP id w1so360659ejf.11 for ; Wed, 20 Jan 2021 17:45:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb-com.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=X5I7XtFX2w8TeDOd3JHdBlpnh/nGPXHch23J+MjiTVY=; b=o1t35Gl/oUhsKcCUTucdMP6UdA+TZQ9FJcNQzNGezxmynwDbuojRCpJbtUPzsMXO0Y VkezB4OjPBgAWwOE5rWm5kukkdHlS3HmVLlTliGnw5UDEQ8kLjnrhC8FomMHc20hchOm uOCWVDy+R+4aHaP+1D34YO/v0yRAyAM/+dxwkyH6zdr/0V7bjcLFPZwVLhv2k7oGhaVR xV3gtphQn/Z5vNlwdmsEOcx7aoFj5iZo/rs7Zs6j/KsB0+kvGXnhQwIDOumEgR0mJric 6F1WpSeJ0wERFxk/K5ihLvhHTf9JR+djjtvQ5W6yfoSZCf+lWrLl7NFHzLePquhoilfY G8xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=X5I7XtFX2w8TeDOd3JHdBlpnh/nGPXHch23J+MjiTVY=; b=av5lABTG1UqDh66GNEYfbq8g9vw/g5f4iEx35XuMa6WQ3BosfMGvxxCPkfmvPT50oS 9K851PcUgsC1K+01wd/mu+9uz+cxhSaKZ0Ge3Dgj+2uk+SNgjZXY3/FyIk6h8eSxhlGO EG98vrj4dPUAxLB0L+8D80v29diGIOacB27cRBOJR6/Xg+Yt1nLh92oT+7V2p8uccrzN kcW++Glj+MxQFZFlD0W6JshD8fDm/muONN847c+HoMIVf8jj8V+ePSEkRz8esj/UL631 9eZhVTKk7KHROUg1dAloW82R3XUImTh/eyAAO2mav7E11vz5MrNr1sw5VVNcsNUQon2n /NLQ== X-Gm-Message-State: AOAM5320uhu/jBoPyY80qLUjm7waN4skQXL7WDcqQPy4kOZI+QqaD+4z BJus2mqI8+dBognzXeNpaox7fTvfAKhAb1ysg35J7ARTPpWz9aNNIaIIlnlXUG/SH//0LYd3anr nbAwij3DuPupwSMcITiPxA3cbuxt3pcTztI/MkJKRZ55OrTWYTHplSAZbJjTUn6FXPcPNsIE1xe Zov8vCza0oMzZOJFeHrs9tmIdJvAuwpKE9cO95c7+25hT1Nc1uWWrAffk5n34aPJekcF/hdDeAL 3U/7/uL/L74e/IlCcEGqKRjQv+wyZEcGBtVXA== X-Google-Smtp-Source: ABdhPJzsPRUB1QZUegV6uk6O+jakrkgQMTadYnH2wqGYlyGfocLRm17hJEpV5HKp118RTymohhXJXw== X-Received: by 2002:a17:906:653:: with SMTP id t19mr7762612ejb.44.1611193515870; Wed, 20 Jan 2021 17:45:15 -0800 (PST) Received: from [10.137.0.20] (ip-86-49-253-127.net.upcbroadband.cz. [86.49.253.127]) by smtp.gmail.com with ESMTPSA id g21sm1998544edh.20.2021.01.20.17.45.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 20 Jan 2021 17:45:14 -0800 (PST) Subject: Re: POC: postgres_fdw insert batching To: Tom Lane Cc: Amit Langote , "tsunakawa.takay@fujitsu.com" , "Andrey V. Lepikhov" , Tomas Vondra , "pgsql-hackers@postgresql.org" References: <20200628151002.7x5laxwpgvkyiu3q@development> <16fcfd4c-2d5e-67f7-fa69-9b5d119bac52@enterprisedb.com> <9d94c863-ce7c-ea7f-6b84-834c98929e2f@enterprisedb.com> <7e70a11e-cf5d-2e5f-a802-274c5e0e236b@enterprisedb.com> <261fbedb-1ced-82f7-e978-60cfe17bce07@enterprisedb.! com> <797826.1611192177@sss.pgh.pa.us> From: Tomas Vondra Message-ID: <442cdcdc-5501-50f7-6efd-5df1668e11f9@enterprisedb.com> Date: Thu, 21 Jan 2021 02:45:12 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <797826.1611192177@sss.pgh.pa.us> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 1/21/21 2:22 AM, Tom Lane wrote: > Tomas Vondra writes: >> I may be wrong, but the most likely explanation seems to be this is due >> to the junk filter initialization, which simply moves past the end of >> the mtstate->resultRelInfo array. > > resultRelInfo is certainly pointing at garbage at that point. > Yup. It's pretty amazing the x86 machines seem to be mostly OK with it. >> It kinda seems the GetForeignModifyBatchSize call should happen before >> that block. The attached patch fixes this for me (i.e. regression tests >> pass with no valgrind reports. > >> Or did I get that wrong? > > Don't we need to initialize ri_BatchSize for *each* resultrelinfo, > not merely the first one? That is, this new code needs to be > somewhere inside a loop over the result rels. > Yeah, I think you're right. That's an embarrassing oversight :-( regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company