Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from ) id 1jHqcW-0001uN-Bt for pgsql-performance@arkaria.postgresql.org; Fri, 27 Mar 2020 15:10:04 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1jHqcV-0003eE-3J for pgsql-performance@arkaria.postgresql.org; Fri, 27 Mar 2020 15:10:03 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1jHqcU-0003e7-MK for pgsql-performance@lists.postgresql.org; Fri, 27 Mar 2020 15:10:02 +0000 Received: from mail-qt1-x82f.google.com ([2607:f8b0:4864:20::82f]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1jHqcR-0004Qn-Ev for pgsql-performance@postgresql.org; Fri, 27 Mar 2020 15:10:02 +0000 Received: by mail-qt1-x82f.google.com with SMTP id z12so8793265qtq.5 for ; Fri, 27 Mar 2020 08:09:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telsasoft-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=bkFtbVhSGF/RLh4KBSbrl1zE++vuITUC4lotWcrVpmc=; b=waf4Ze0eWvBf80zP5yPMZMPo787H1TblX7Sv+S5Wg5UqmGw1PmuCN3TDWtoReNYRrQ s/qnd6CLhBZ8QafznyTNtn17rCsVivFYqwDDzmrKIriDU9cZ7ki8Futp1svsQ1EcUWXY Zx5wI4L5houWOriYGTk8I8yDnqZ2X1zHSwkNwi0yer/a9ntaumvzxgjjK/qq3se8UYBW 0PkxqGYYWphcZuTCXuqapb1RV02dEAMIXaeGJlYr501EoTujm3pgF24jHj91B6vCAjde 0dXZF9aFqQhz2mBCIbnI+H7qY55DJyyHKZ1DHD60+e4ckZFudBAQUdq/kSmLnNbabwz1 +4rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=bkFtbVhSGF/RLh4KBSbrl1zE++vuITUC4lotWcrVpmc=; b=AwtU12THTcM0LDpxdzJNsoRIhuHxwN7pcm4rUxlIDLHvii9nfP5MT9Sbr9qx9ra1Hc k5YelCSbbthYOzmUjzyR9EsktK/aJItta3KZbN+vu3McPWta6bcVOdUU8ezabWuydRF1 VPYn25CKlNhzcY8mEyIreEeTTeyJmw8kYSTXnQkNyjlDH5ogkTkpZpoQ3saBvazzID+L oW9yI58TYuNGU351Gt+HG4xAdRjpgWhRrLkpH5svVlNDMsajlSgA5K3EmqmdLfvAzpyH eOIA4HLnxJDDzncbMECjvyQ8DqZfk+RCV6iUJgEcJtq4rswkrBgBbne8YjRMM9+iikeQ YoqQ== X-Gm-Message-State: ANhLgQ1cXfTfZVEd000YbxNdlNJ1QCd0QfEjBYIGyCQJsNsI7Ta3K5qN J1mgQuYoS+Q4QPC8Lj9FNqeZvA== X-Google-Smtp-Source: ADFU+vtYvczeym/8VDf2mauK0+Fe8Dj9VL7XwqU0F+4KlzzdnQJorObIBiga8N5r1UKfjMXib5jHuQ== X-Received: by 2002:ac8:352d:: with SMTP id y42mr14232106qtb.385.1585321797404; Fri, 27 Mar 2020 08:09:57 -0700 (PDT) Received: from pryzbyj (charmander.telsasoft.com. [50.244.222.1]) by smtp.gmail.com with ESMTPSA id v1sm4076860qtc.30.2020.03.27.08.09.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Mar 2020 08:09:56 -0700 (PDT) Received: by pryzbyj (Postfix, from userid 1000) id 02710800828; Fri, 27 Mar 2020 10:09:54 -0500 (CDT) Date: Fri, 27 Mar 2020 10:09:54 -0500 From: Justin Pryzby To: Michael Lewis Cc: Ekaterina Amez , pgsql-performance@postgresql.org Subject: Re: Best way to delete big amount of records from big table Message-ID: <20200327150954.GG20103@telsasoft.com> References: <6e456b07-a396-f9e3-6a22-23a96b73420c@zunibal.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Fri, Mar 27, 2020 at 08:41:04AM -0600, Michael Lewis wrote: > 2) If you are deleting/moving most of the table (91 of 150 million), > consider moving only the records you are keeping to a new table, renaming > old table, and renaming new table back to original name. Then you can do > what you want to shift the data in the old table and delete it. You could also make the old table a child of (inherit from) the new table. That allows you to remove rows separately from removing them. Partitioning (with legacy inheritence or the new, integrated way available in postgres 10) allows DROPing oldest tables rather than DELETEing from one gigantic table. You should consider somehow cleaning up the old table after you DELETE from it, maybe using vacuum full (which requires a long exclusive lock) or pg_repack (which briefly acquires an exclusive lock). -- Justin