Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oIuTB-0006Lj-5T for pgsql-performance@arkaria.postgresql.org; Tue, 02 Aug 2022 16:10:09 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1oIuT9-0005em-0i for pgsql-performance@arkaria.postgresql.org; Tue, 02 Aug 2022 16:10:07 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oIuT8-0005ed-E2 for pgsql-performance@lists.postgresql.org; Tue, 02 Aug 2022 16:10:06 +0000 Received: from cp1.c0.inoc.net ([64.246.141.1]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oIuT3-0002JK-O9 for pgsql-performance@postgresql.org; Tue, 02 Aug 2022 16:10:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=inoc.net; s=default; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:Cc: References:To:Subject:MIME-Version:Date:Message-ID:Sender:Reply-To:Content-ID :Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To: Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe :List-Post:List-Owner:List-Archive; bh=k1vcH16hqDNGb5u3BHOHtdSqp6JkU9JrTA7Y/jLkh28=; b=o1FYmECfFsMAUyFKBWe7YwwfPC 0ClTowVtDEdbM7f0tk0Dr00FAKqtNnCmeL9Hk78xmdMdY0IUSC6GKMVJI8hUZYgKnXxLvJ+0PuVtu aDZM1MNer5ZEVwfbBRVZTNBCrP64cOn9OLDiUUuevX4Iigj/2C/Mnfe3j+Lo9v/vi5nci/JT2FwUP ARPeh+AKAPAi6+mcAzfnchTCVkcM7U6y7ly9Dq0QYEHT687+gJ7ZmJfMyR+g3Fg+6/VtnP/ynU6g9 oBtjTlUpuRuOLYyTAgfX2iT0TmII2gpCNRFeXC7DJIKEg2C61HTlwH3PshfqnuCfB/tjSNWnPh+Cc cQwRLm8Q==; Received: from [2607:f058:11:a:f0ee:5a5f:f28:8a5d] (port=49692) by cp1.c0.inoc.net with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.95) (envelope-from ) id 1oIuT2-0006Ez-97; Tue, 02 Aug 2022 16:09:59 +0000 Message-ID: <8bd0f6cd-cbfc-2033-431c-d4ac613c1878@inoc.net> Date: Tue, 2 Aug 2022 12:09:58 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: PgSQL 14 - Logical Rep - Single table multiple publications? Content-Language: en-US To: Rory Campbell-Lange References: <3d5996ac-a181-21a5-62a0-1cd34e6b3f1a@inoc.net> Cc: pgsql-performance@postgresql.org From: Robert Blayzor In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - cp1.c0.inoc.net X-AntiAbuse: Original Domain - postgresql.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - inoc.net X-Get-Message-Sender-Via: cp1.c0.inoc.net: authenticated_id: rblayzor@inoc.net X-Authenticated-Sender: cp1.c0.inoc.net: rblayzor@inoc.net X-Source: X-Source-Args: X-Source-Dir: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 8/2/22 10:57, Rory Campbell-Lange wrote: > Special care must be taken with the "replica identity" of published > tables, as set out at > https://www.postgresql.org/docs/current/logical-replication-publication.html. > For example, you may need a unique identifying column for each source > table in addition to the normal row identifier to differentiate*this* > table's id 1 row from the*other* table's id 1 row, otherwise the > subscriber won't be able to identify the row to delete if this table's > id 1 row is deleted (for example). > > Although this seems to work fine with native replication, the pglogical > extension has more knobs. For instance, the > pglogical.wait_for_subscription_sync_complete function is useful to ensure that > sync finishes when part of a migration. We would literally just be merging bulk data rows that are considered immutable, meaning they would never be updated or deleted. We would replicate only inserts, not deletes, updates, etc. Would the table identifier still be required in this case? We have a half a dozen DB's that just collect call records, they are in different locations. They get their local call data and store it into their local table. We would want to aggregate all that data in a central subscription database table for pure analytics/reporting purposes... -- inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP: https://pgp.inoc.net/rblayzor/