Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mXRnL-0002D2-EB for pgsql-general@arkaria.postgresql.org; Mon, 04 Oct 2021 17:30:31 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1mXRnK-0008PP-CM for pgsql-general@arkaria.postgresql.org; Mon, 04 Oct 2021 17:30:30 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mXRnK-0008PG-3V for pgsql-general@lists.postgresql.org; Mon, 04 Oct 2021 17:30:30 +0000 Received: from out3-smtp.messagingengine.com ([66.111.4.27]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mXRnG-0001LH-Ej for pgsql-general@lists.postgresql.org; Mon, 04 Oct 2021 17:30:29 +0000 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 0B1AC5C0209; Mon, 4 Oct 2021 13:30:24 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 04 Oct 2021 13:30:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aklaver.com; h= subject:from:to:cc:references:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=L raHffg6PVnbfLK1gfSRH9sLq/QXGdfdXJP9ykFbDTQ=; b=rrvh6Xc8EX4GYRwdw FRsSmUswsz674jeJZkg0vfZJYYxUc7BHQ/aV9HUD8DcinbyON97q4o+rYhSTEKMP ya2RqXyJMHyM5fruEq5uoI/J0wy1oyfhL3hCa3ayqJoDlIoRYTtcK07A17vEVb1A kEJC6plTyEdCYDbxtp304Ay4NZRiGIRKbY81r5o1TJTvX5TNMIPH3eKIlxhusn2w B95Ofh85mgw7g96PynkWQJdwwmb2H+26EZNJkGnHPnJm0aFIG4S8wRQfGw9Dpr3t zic4svyBTa7DsNn6Lkvy72sjrNM6JmHysadShtw9uB8CksvNOQvVEDXJq9WmPISU vGQhw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=LraHffg6PVnbfLK1gfSRH9sLq/QXGdfdXJP9ykFbD TQ=; b=B2pE7W5JKZ1tBZmrIaaCGXz3lT1lYlGaXkAV68I0i5d5Y273nlJenTU0G J0hcLgP39NA+05gNR5X2AqownAjWLzq/Ff1JT9zyBPiV44xUaMfBf32QWeCQfWVz lAzeVAS/TIQlzrEsJIu/rP5D5PNOyEWEVepyDMek2RyVz4Qs6OArYln42MfaWJOE bBrLcixRHzynJG7FZBKlW03na4PutE+Qpxlr7xjY/f0avqPkZYi5CFVkG/2VzP2J Qp8QPpFnseUxDjH//3j4IDC+GpPb9CohtFatpXWoQ5b94tOtUjGqddTQoJJxMNv2 2n4xLT6dWJNeArXYigJ13IQ5Eumvw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudelvddguddutdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuhffvfhfkffgfgggjtgfgsehtjeertddtfeejnecuhfhrohhmpeetughr ihgrnhcumfhlrghvvghruceorggurhhirghnrdhklhgrvhgvrhesrghklhgrvhgvrhdrtg homheqnecuggftrfgrthhtvghrnhepgeehjedvffdtledtieekjedtvedtgefgteetgffg hfegleduheeludekueejkefgnecuffhomhgrihhnpehphigurghtrgdrohhrghenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegrughrihgrnhdr khhlrghvvghrsegrkhhlrghvvghrrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 4 Oct 2021 13:30:23 -0400 (EDT) Subject: Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres From: Adrian Klaver To: Shaozhong SHI Cc: pgsql-general References: <1b86e754-0317-84df-848f-9218e8df73e9@aklaver.com> Message-ID: Date: Mon, 4 Oct 2021 10:30:22 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 10/4/21 10:28 AM, Adrian Klaver wrote: > On 10/4/21 10:10 AM, Shaozhong SHI wrote: >> Hello, Adrian Klaver, >> What is the robust way to upgrade Pandas? > > Carefully. > > The most recent version is 1.3.3, which is approximately 5 versions > ahead of where you are now. The big jump is when Pandas went from 0.25 > to 1.0. See docs here: > > https://pandas.pydata.org/docs/whatsnew/v1.0.0.html?highlight=upgrade > > So the process should be 0.24 -> 0.25, verify, 0.25 -> 1.0, verify. Then > on to wherever you want to end up a step at a time. > > Before each step spend time here: > > https://pandas.pydata.org/docs/whatsnew/ > > to see what the gotcha's are. Should have added: If you are not already working in a virtualenv it would be a good idea to do the above in one or more. > > >> Regards, >> David >> > > > -- Adrian Klaver adrian.klaver@aklaver.com