Received: from maia.hub.org (maia-2.hub.org [200.46.204.251]) by mail.postgresql.org (Postfix) with ESMTP id 6A4A51337B88 for ; Tue, 30 Nov 2010 21:48:03 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.251]) (amavisd-maia, port 10024) with ESMTP id 57912-02 for ; Wed, 1 Dec 2010 01:47:56 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from spinlock.commandprompt.com (host-215.commandprompt.net [207.173.203.215]) by mail.postgresql.org (Postfix) with ESMTP id 1341D1337B8E for ; Tue, 30 Nov 2010 21:47:56 -0400 (AST) Received: from localhost (localhost [127.0.0.1]) by spinlock.commandprompt.com (Postfix) with ESMTP id D0B16348023; Tue, 30 Nov 2010 17:47:48 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at commandprompt.com Received: from spinlock.commandprompt.com ([127.0.0.1]) by localhost (spinlock.commandprompt.com.commandprompt.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EX0qN5F-QQGo; Tue, 30 Nov 2010 17:47:43 -0800 (PST) Received: from [192.168.1.100] (unknown [71.94.248.62]) by spinlock.commandprompt.com (Postfix) with ESMTPSA id 54C7F348022; Tue, 30 Nov 2010 17:47:42 -0800 (PST) Subject: Re: SELECT INTO large FKyed table is slow From: "Joshua D. Drake" Reply-To: jd@commandprompt.com To: Mario Splivalo Cc: pgsql-performance@postgresql.org In-Reply-To: <4CF24103.8030209@megafon.hr> References: <4CF24103.8030209@megafon.hr> Content-Type: text/plain; charset="UTF-8" Organization: Command Prompt, Inc. Date: Tue, 30 Nov 2010 17:47:32 -0800 Message-ID: <1291168052.20631.1963.camel@jd-desktop> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.9 tagged_above=-10 required=5 tests=BAYES_00=-1.9 X-Spam-Level: X-Archive-Number: 201011/425 X-Sequence-Number: 41449 On Sun, 2010-11-28 at 12:46 +0100, Mario Splivalo wrote: > The database for monitoring certain drone statuses is quite simple: > > This is the slow part: > INSERT INTO drones_history (sample_id, drone_id, drone_log_notice, > drone_temperature, drone_pressure) > SELECT * FROM tmpUpdate; > > For 100 rows this takes around 2 seconds. For 1000 rows this takes > around 40 seconds. For 5000 rows this takes around 5 minutes. > For 50k rows this takes around 30 minutes! Now this is where I start lag > because I get new CSV every 10 minutes or so. Have you considered making the foreign key check deferrable? JD -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support, Custom Development, Engineering http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt