Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mXRle-000250-KK for pgsql-general@arkaria.postgresql.org; Mon, 04 Oct 2021 17:28:46 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1mXRlc-00048g-NG for pgsql-general@arkaria.postgresql.org; Mon, 04 Oct 2021 17:28:44 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mXRlc-00048T-D1 for pgsql-general@lists.postgresql.org; Mon, 04 Oct 2021 17:28:44 +0000 Received: from out3-smtp.messagingengine.com ([66.111.4.27]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mXRlV-0001et-AC for pgsql-general@lists.postgresql.org; Mon, 04 Oct 2021 17:28:42 +0000 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 2BA3A5C0195; Mon, 4 Oct 2021 13:28:36 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Mon, 04 Oct 2021 13:28:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aklaver.com; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=O 6ij9e6j4cTdjHtK/WX0B9INbeKSO8ijb5bBXAlLHiE=; b=GVoizPlszk1uNpVUJ IhfujPzAIhxiNn5g1BABAh5rgM61wFIz2i99yW0RH3ZZiXzGpNgP2vIZHLojyA/L O80BzZpvz5xa5YjDhTVT8WEMXooqxAhtSLi/I16RVKjJg0jcmckVoSp8mz8UNXZA nievlnYjk2Ioi/5W0EB9knYkmfxrKU+XVZE0NgfXF/hMdEoitG+X+yXZbDKg60uH y5CsPqC7rG2ImtgCXhB0YH9fpuj8AcrEa8QoQWvbS7NKuvl0lBx1UVKiY1q73JB1 uG54ky0ZF2qLtgNsRV0qo62rJVZmF10OWbDBsGp2L3LrN1OFgmrCZ+hMtsDdIcFq Ul8Ig== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=O6ij9e6j4cTdjHtK/WX0B9INbeKSO8ijb5bBXAlLH iE=; b=R/3VS/ficYK2sb1PdVR2hs9EcHWP1MIJB86K5UUgaXOWtNdpr0W0isKnO LyI5cdjI1+UyGf5D/4wYAZ/el44x9bTweF3F44szrxA8Ai+1T/xd8/pbdx5QhsEO RiOeYDZdFk39j14Wlz3oBk/sKgIbHh5xbjqjmMWV/EO0+9pxS0D0iBPr81OTN8Lp vh07qSDguZnuQGdqmAfi8KqR9xxbi6Rmiee13UtTTZKYku3W4whkoTg1QlVZsMPR 6Wu2Xm4FgTEHRW3K4VElgsTSLhl6zdZ0VpyOTdjXTyzrWMq/8QjRfPM7cWoRMPBI b0LaD+TJ0V8185FMTK8ZKN9MNHVCQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrudelvddguddutdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefuvfhfhffkffgfgggjtgfgsehtjeertddtfeejnecuhfhrohhmpeetughr ihgrnhcumfhlrghvvghruceorggurhhirghnrdhklhgrvhgvrhesrghklhgrvhgvrhdrtg homheqnecuggftrfgrthhtvghrnhepueejieefgfektdeghfdvudehgffgheefvdduudeh gfffhfefvefhuddthfehieegnecuffhomhgrihhnpehphigurghtrgdrohhrghenucevlh hushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegrughrihgrnhdr khhlrghvvghrsegrkhhlrghvvghrrdgtohhm X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 4 Oct 2021 13:28:35 -0400 (EDT) Subject: Re: Testing of a fast method to bulk insert a Pandas DataFrame into Postgres To: Shaozhong SHI Cc: pgsql-general References: <1b86e754-0317-84df-848f-9218e8df73e9@aklaver.com> From: Adrian Klaver Message-ID: Date: Mon, 4 Oct 2021 10:28:34 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 10/4/21 10:10 AM, Shaozhong SHI wrote: > Hello, Adrian Klaver, > What is the robust way to upgrade Pandas? Carefully. The most recent version is 1.3.3, which is approximately 5 versions ahead of where you are now. The big jump is when Pandas went from 0.25 to 1.0. See docs here: https://pandas.pydata.org/docs/whatsnew/v1.0.0.html?highlight=upgrade So the process should be 0.24 -> 0.25, verify, 0.25 -> 1.0, verify. Then on to wherever you want to end up a step at a time. Before each step spend time here: https://pandas.pydata.org/docs/whatsnew/ to see what the gotcha's are. > Regards, > David > -- Adrian Klaver adrian.klaver@aklaver.com