Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pWgpk-000384-CH for pgsql-general@arkaria.postgresql.org; Mon, 27 Feb 2023 16:58:40 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pWgpj-0007bp-7S for pgsql-general@arkaria.postgresql.org; Mon, 27 Feb 2023 16:58:39 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pWgpi-0007bf-Sd for pgsql-general@lists.postgresql.org; Mon, 27 Feb 2023 16:58:38 +0000 Received: from out4-smtp.messagingengine.com ([66.111.4.28]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pWgpg-0006ij-FF for pgsql-general@postgresql.org; Mon, 27 Feb 2023 16:58:37 +0000 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id DEE1D5C00FE; Mon, 27 Feb 2023 11:58:35 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Mon, 27 Feb 2023 11:58:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aklaver.com; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm3; t=1677517115; x= 1677603515; bh=yaIHG/+E6YU+Cf+iQOXelXo8I727845MM7DmVb1WvRw=; b=O Fysiiy8Jk8pNeRUlf9SLTl3sB32AExPlBPlv7O6VTXKCBnlaRCwlbVfBJpOWXZq1 IhzQhOgs05w4RQ5sa1wzRjxcWxmMS4J0zktAjb0rJkcv43t+133N8wfmYKGSNE4N +fawabceDhNZw/UOy+9RgraU4y7i0v/jouTujCCYbhi927AHSAHclfwLDgleri/x A/Ry/6XJziDefi1VFMWTfns2polpeUq7ea0SgDM9TyamhFVsJXvp1l49a8TFNSUX aQPcLuPlj++mPqmhDPN3AB9aS9vk8Sh/35egEGHXqELCoVlI7eI82purfG9z2ESm SJr8VIjCR/QunSJHbG4gw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1677517115; x= 1677603515; bh=yaIHG/+E6YU+Cf+iQOXelXo8I727845MM7DmVb1WvRw=; b=Z 05k5ChXTh85hnaVqyPmUALPOTu2EQQqOq1hJONYwUHI051gFpQJfheqPn8dXsmR8 DVn+r1r2KR9CfFw4TJgPceCe8pScLI8FDvyoEFP0eFQURwpUXGEqy5n4+9yCTFIW AXgIax1x3qV3/P+JJBH1QMEvnf76lsYhyk4WFfPrakOg08j9qFUgZXQ42Y+LQBTF v3SY2RpNCVilEIPniEvmVAQNLmFPttSiX1SMsI/bcuz88HUd2JIjkU8+fKq9a1RL MCMWVKD3pIpkq8xQTjgJifSFuL90eaAx9zBZlCGdBSKqQjVMfZzTm7pG9r+jkJZQ PLEloFFMU7YGXTqGZnjEg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrudeltddgleduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefkffggfgfuvfhfvefhjggtgfesth ekredttdefjeenucfhrhhomheptegurhhirghnucfmlhgrvhgvrhcuoegrughrihgrnhdr khhlrghvvghrsegrkhhlrghvvghrrdgtohhmqeenucggtffrrghtthgvrhhnpeehveeihe fhfeetgeektdfftedvfffhudelueeuhfeuleffheegudelffetteehteenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegrughrihgrnhdrkhhlrg hvvghrsegrkhhlrghvvghrrdgtohhm X-ME-Proxy: Feedback-ID: i76984098:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 27 Feb 2023 11:58:34 -0500 (EST) Message-ID: Date: Mon, 27 Feb 2023 08:58:34 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: Re: pg_upgradecluster transfering only a portion of the data Content-Language: en-US To: =?UTF-8?Q?D=c3=a1vid_Suchan?= References: Cc: pgsql-general From: Adrian Klaver In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 2/27/23 08:49, Dávid Suchan wrote: Reply to list Ccing list for real this time. > 1) I used \l+ in psql and then counted rows- millions were missing \l lists databases. Are you saying there are millions of database? Otherwise what rows where you counting? > 3) nothing at all, everything was "success" > 4) I did not, I presume it is there, the question is why only 700 mb was > transferred If you have not connected how could you do the \l and row count? > 5) would it be inside main pg log? Or some special one? > > Dňa po 27. 2. 2023, 17:14 Adrian Klaver > napísal(a): > > On 2/27/23 07:44, Dávid Suchan wrote: > > Hello, I tried upgrading pg db from version 9.6 to 14 by using > > pg_upgradecluster command. I freshly installed pg 14 -> ran > > pg_dropcluster 14 main --stop -> and then upgraded using > > pg_upgradecluster 9.6 main. > > After a successful prompt finished, I checked the database and > the size > > went from originally 20gb (in 9.6) to 700~ mb (in 14) while the disk > > space available shrank by about 2gb meaning that there is still > the 20gb > > of data. I tried the entire process twice (since I had created an > AWS > > 1) How did you measure the size of the database clusters? > > 2) pg_upgrade will not remove the old cluster automatically so it not > surprising the overall disk usage increased. > > 3) Did you see any messages at end of upgrade mentioning issues? > > 4) Have you connected to new cluster to see if everything is there? > > 5) Does the Postgres log provide any relevant information? > > > EC2 snapshot for this) and the result was the same. > > Is my solution to migrating old pg version to the new one wrong? > Before > > this I tried the same process with around 300mb of data and all > of that > > transferred successfully. If I did not understand the > pg_upgradecluster > > command, what would be the best practice when upgrading pg > version with > > huge amounts of data(could be a terabyte)? > > -- > Adrian Klaver > adrian.klaver@aklaver.com > -- Adrian Klaver adrian.klaver@aklaver.com