Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ozbJZ-0005UZ-Ak for pgsql-general@arkaria.postgresql.org; Mon, 28 Nov 2022 10:24:41 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1ozbJX-0005xx-6N for pgsql-general@arkaria.postgresql.org; Mon, 28 Nov 2022 10:24:39 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ozbJW-0005xn-Mi for pgsql-general@lists.postgresql.org; Mon, 28 Nov 2022 10:24:38 +0000 Received: from mail-ej1-x62d.google.com ([2a00:1450:4864:20::62d]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1ozbJT-0003YC-Jd for pgsql-general@lists.postgresql.org; Mon, 28 Nov 2022 10:24:37 +0000 Received: by mail-ej1-x62d.google.com with SMTP id b2so8261544eja.7 for ; Mon, 28 Nov 2022 02:24:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cybertec-at.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:date:to:from:subject:message-id:from:to:cc:subject:date :message-id:reply-to; bh=iezNuS3FBeKMzEPaafPPzxm4G+ubw+vegfAO2KkiGQ0=; b=X+J0Qcs6NDHLsula4GgvPbHndOOdMcfsa2YrAOKNxRBhpCGoPNxxgyigxrUoZVn3AM joZBSiMc1N4fXzoRSRxWhYlV69IGO81YkpPojE03wAljvGDeOvDMPKLO7DB57L+wQtXq s61E6n1CtLny5pEwXFICnezDVIgpAWLDvjVg+t6A7Q2XyHMXGQEz+fY4IcjEgtzWRhox 5/ERWfGAsqKibqr5rS69A1m0KvfhI3lZ/yEs+bypmVU2rpLYqlq26QDjOupOQP85lKlj XMG5NEmx3Mjcf+3udrZDDHrPYsF6WFvrIB5JKhQIoV4qFita4mJaXcqfqwa3CcM6CkPv wMXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:date:to:from:subject:message-id:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=iezNuS3FBeKMzEPaafPPzxm4G+ubw+vegfAO2KkiGQ0=; b=KdeDfSApPbXI8Asi04qN425p1Ouj4Uxm3m5wRIPW7hX0J1+xJtA8VfM0+L+o2KAyqq GPr6Y3xLqvSj1Yiei1+am77WP0x74p85/vDndEFAgoQ4L02UxrzWw7/+oq7LsLckpUBi rj9lEmHZqkms7RiZUl5pZ19a1fRJRH1zjEHLNPaZkfK9L0dWwsvmQFxIi1JMeFniyQji AFxcF+6k1FtzWtHJGcpc1nkbwixRgBYAY8CzgCxMN/1/p+T4avCYb9NK6NAaq7DMwPTn 6dd10q7+mFwftrZp+shRhANgLNusZMXqwz0EgyQQoFmsxJq8u2QwCkTkFJXX6L2EaGFt rlRg== X-Gm-Message-State: ANoB5pkRnh++A/rMxXIesh8D90+GKw4VrAdwC8NF2HlDeMBaLwTYGb66 o4PK16kkG3SO4C7AtUGDlQOmBw== X-Google-Smtp-Source: AA0mqf6xnm9hIlUOU4QUYKFM1demUBsznfdVgTlQwCvuYfd7SjnlrcURj6x90TuKcbKBvKffVEZXhw== X-Received: by 2002:a17:907:a50c:b0:7bc:98f6:35c4 with SMTP id vr12-20020a170907a50c00b007bc98f635c4mr13538809ejc.123.1669631073545; Mon, 28 Nov 2022 02:24:33 -0800 (PST) Received: from localhost.localdomain ([2001:871:5e:b47b:e706:88d7:ceb1:a099]) by smtp.gmail.com with ESMTPSA id a1-20020a05640213c100b0044ef2ac2650sm4986388edx.90.2022.11.28.02.24.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Nov 2022 02:24:33 -0800 (PST) Message-ID: <7036a495a71b92e96c1a4f4fbd9ba0b520bf9acf.camel@cybertec.at> Subject: Re: Replicating an existing (huge) database From: Laurenz Albe To: Srinivasa T N , PostgreSQL General Date: Mon, 28 Nov 2022 11:24:32 +0100 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.4 (3.42.4-2.fc35) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Mon, 2022-11-28 at 15:47 +0530, Srinivasa T N wrote: > I am using postgresql 12.  As part of streaming replication setup, I run pg_basebackup > on the slave which copies the database from master to slave.  But the database is huge > and it takes around 2 to 3 days for the pg_basebackup to finish.  When pg_basebackup > is running, a huge number of wal files are generated on the master which occupies a lot > of space (even though it is for 2-3 days, disk space is low). > Is there any other way to start replication without using pg_baseback? There is the "low level backup API" (pg_start_backup() and pg_stop_backup()) that allows you to copy the data yourself, for example with storage snapshots, which can be much faster. The essential process is like with pg_basebackup, but you can use more efficient methods to copy the data. Yours, Laurenz Albe -- Cybertec | https://www.cybertec-postgresql.com