Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nSOFJ-0005Vi-Tv for pgsql-hackers@arkaria.postgresql.org; Thu, 10 Mar 2022 19:14:46 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1nSOFI-0003UK-Nn for pgsql-hackers@arkaria.postgresql.org; Thu, 10 Mar 2022 19:14:44 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nSOFI-0003UA-Dw for pgsql-hackers@lists.postgresql.org; Thu, 10 Mar 2022 19:14:44 +0000 Received: from mail-ej1-x62f.google.com ([2a00:1450:4864:20::62f]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1nSOFF-0003LF-J8 for pgsql-hackers@postgresql.org; Thu, 10 Mar 2022 19:14:43 +0000 Received: by mail-ej1-x62f.google.com with SMTP id d10so14227476eje.10 for ; Thu, 10 Mar 2022 11:14:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=enterprisedb.com; s=google; h=message-id:date:mime-version:user-agent:subject:content-language:to :cc:references:from:in-reply-to:content-transfer-encoding; bh=L8vnl7oSwAlGFqkKgnuih/0OtBWVpLPlDQd7bAimB6s=; b=a+ORfiGp0OObBtfAA4cM/i4DJ8f49Wo71ZY45IGbi3EPvVNKjcMDuhTeyWpPDpsl/+ zwTy+oTg5PATMCKCgnfXWigqRaZi6M0as9o0ZUkZBzNPa64vY5F4AQzTjv/NhmmOo4RO 26uRd0maYLTJImwwcVoBiVA6IURZ9rAub/saMd1kHnCFyUEz7l8pdGdKg2wEUI8Kl98X VLnZV4hLvfPaf3cHSq0NiPV7G6pMD4t1tld3+Dx0CsT5sK4CM7ofhb3No22NMX92Ka6a sCBBi5rMWq4W7xxQN1M0Iw5TWE95iqSoEZ4PoPx7ZvxpBpddMklUMcEhDKTq6qepknUq KPnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=L8vnl7oSwAlGFqkKgnuih/0OtBWVpLPlDQd7bAimB6s=; b=JORhDfdvbB658fQsx6V8kvO/U2xNtIiuwfVLhnu+DZKYjBqN0gSjtKPNqDwtROGh8L 4Zjbc4785f9jJrgq3S2HXLyYjPGSn9lZsROvHJNqKChD3WoFf9qWl5mrx3UhDQJ5MZED Q5+d3Kn4xGcOjGUQVnz6GjwQ6kwIhEi7yZ6DcyIagtGFH3rGNSHSyXR3tfBIISYCNpwV 7R/BS2uEdCzYAtLjouRoG+wWoNOIQF78Y7M/1jlXsz56BQv8CjYgwDp5zXEqnZb2p9ZD WRANYwBhcHD2CQAp2DeR8CY4zWm/L36ehQHrKwElKU0GTPF9xMAidfBW/Pmzlo8gTYjn eI0g== X-Gm-Message-State: AOAM533daO55bZ1iCSPxOpgRwJ4FfQO9Vyi6lzNOrFcCEczdjlsaZtBy DtPEyQMhs6celumM0JybRVS7Tjyd70Fvd1i4Jvi9F6JPHqy2a9+tdtoZ70gHiW1aUCMr/wzBE1W q0mvKu4wTxAt1wdzPjwV2qYa4jpobpsOHKqBy8utX2nQWp6/m1Htom4h2fxoDFaS9u9r7I6r9Mi vWtp0NAD1XS0OOIs1Mk7erUOv54iN5odKrGYqR X-Google-Smtp-Source: ABdhPJw6qhkpbNO5YAxkPNUTSN+coQ+ksac0GMMFlibYpjqLDaA2Ki10gHOwXwMmU4/RgCLnqcjzAA== X-Received: by 2002:a17:907:2cc3:b0:6da:e6cb:2efa with SMTP id hg3-20020a1709072cc300b006dae6cb2efamr5469429ejc.169.1646939680183; Thu, 10 Mar 2022 11:14:40 -0800 (PST) Received: from [10.137.0.18] (static-84-42-175-93.net.upcbroadband.cz. [84.42.175.93]) by smtp.gmail.com with ESMTPSA id n6-20020a170906724600b006d4e2f6857dsm2089265ejk.139.2022.03.10.11.14.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 10 Mar 2022 11:14:39 -0800 (PST) Message-ID: <5af7066b-15ac-1989-9afe-4e5dc1d60154@enterprisedb.com> Date: Thu, 10 Mar 2022 20:14:37 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: Column Filtering in Logical Replication Content-Language: en-US To: Amit Kapila Cc: Peter Eisentraut , Alvaro Herrera , Justin Pryzby , "houzj.fnst@fujitsu.com" , Rahila Syed , Peter Smith , pgsql-hackers , shiy.fnst@fujitsu.com References: <202201120041.p24wvsfcsope@alvherre.pgsql> From: Tomas Vondra In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-CLOUD-SEC-AV-Info: enterprisedb,google_mail,monitor X-CLOUD-SEC-AV-Sent: true X-Gm-Spam: 0 X-Gm-Phishy: 0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 3/10/22 04:09, Amit Kapila wrote: > On Wed, Mar 9, 2022 at 3:33 PM Amit Kapila wrote: >> >> On Mon, Mar 7, 2022 at 8:48 PM Tomas Vondra >> wrote: >> >>> OK, I reworked this to do the same thing as the row filtering patch. >>> >> >> Thanks, I'll check this. >> > > Some assorted comments: > ===================== > 1. We don't need to send a column list for the old tuple in case of an > update (similar to delete). It is not required to apply a column > filter for those cases because we ensure that RI must be part of the > column list for updates and deletes. I'm not sure which part of the code does this refer to? > 2. > + /* > + * Check if all columns referenced in the column filter are part of > + * the REPLICA IDENTITY index or not. > > I think this comment is reverse. The rule we follow here is that > attributes that are part of RI must be there in a specified column > list. This is used at two places in the patch. Yeah, you're right. Will fix. > 3. get_rel_sync_entry() > { > /* XXX is there a danger of memory leak here? beware */ > + oldctx = MemoryContextSwitchTo(CacheMemoryContext); > + for (int i = 0; i < nelems; i++) > ... > } > > Similar to the row filter, I think we need to use > entry->cache_expr_cxt to allocate this. There are other usages of > CacheMemoryContext in this part of the code but I think those need to > be also changed and we can do that as a separate patch. If we do the > suggested change then we don't need to separately free columns. I agree a shorter-lived context would be better than CacheMemoryContext, but "expr" seems to indicate it's for the expression only, so maybe we should rename that. But do we really want a memory context for every single entry? > 4. I think we don't need the DDL changes in AtExecDropColumn. Instead, > we can change the dependency of columns to NORMAL during publication > commands. I'll think about that. > 5. There is a reference to check_publication_columns but that function > is removed from the patch. Right, will fix. > 6. > /* > * If we know everything is replicated and the row filter is invalid > * for update and delete, there is no point to check for other > * publications. > */ > if (pubdesc->pubactions.pubinsert && pubdesc->pubactions.pubupdate && > pubdesc->pubactions.pubdelete && pubdesc->pubactions.pubtruncate && > !pubdesc->rf_valid_for_update && !pubdesc->rf_valid_for_delete) > break; > > /* > * If we know everything is replicated and the column filter is invalid > * for update and delete, there is no point to check for other > * publications. > */ > if (pubdesc->pubactions.pubinsert && pubdesc->pubactions.pubupdate && > pubdesc->pubactions.pubdelete && pubdesc->pubactions.pubtruncate && > !pubdesc->cf_valid_for_update && !pubdesc->cf_valid_for_delete) > break; > > Can we combine these two checks? > I was worried it'd get too complex / hard to understand, but I'll think about maybe simplifying the conditions a bit. > I feel this patch needs a more thorough review. > I won't object to more review, of course. regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company