Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ib76O-0001nx-9Q for pgsql-hackers@arkaria.postgresql.org; Sat, 30 Nov 2019 18:04:16 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1ib76N-0004u6-2U for pgsql-hackers@arkaria.postgresql.org; Sat, 30 Nov 2019 18:04:15 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ib76M-0004tz-Ms for pgsql-hackers@lists.postgresql.org; Sat, 30 Nov 2019 18:04:14 +0000 Received: from mail-yw1-xc2d.google.com ([2607:f8b0:4864:20::c2d]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ib76K-0001yl-3y for pgsql-hackers@postgresql.org; Sat, 30 Nov 2019 18:04:13 +0000 Received: by mail-yw1-xc2d.google.com with SMTP id u139so436896ywf.13 for ; Sat, 30 Nov 2019 10:04:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=Fv/8mMEPY0cWS0Pue4LLf7rpy/14xCUqfYTmXx5+y+0=; b=Wb2dEo0c17kCrCeGNNmZCuYQpzD70nTvGT1VRoyvzOhAsS5KGBbkY0iYU3WQhppJ/D uZZRDdWZH59jdnd8ERPT+g06YqPK7972GgsuM05qpVtOc1M0mlgsOA6m5xwR7/6sIEId T+5iBhbBgs51TWNpoc3wSXBMSjXiiirMNDPkFf9YWon8AfMIa6O3PbJJnFkEAquMe+it A+0bUk5NbCb52RK2prUQuBgAEvObq6snwXU6zCFzuLc7N0DILEKypwxz04xm4tk2J7xI nv95EwtwjJcA8hK5ssYrEEPlcKPOAF6RQ0Yc89nJs3M5iCkTdOwNhBYF21wXwqlTvnhi kAWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Fv/8mMEPY0cWS0Pue4LLf7rpy/14xCUqfYTmXx5+y+0=; b=MVPKbBn0TEfnhGQYLUTiQnsDo8xqks9fBmb3NpcpFoKG3HIAWPD+aNqdAPYsvU/riZ sM2CClHJvEXQd2wByeaVX4V/EE+ZEWBcYEPHsw2EdMukvK5LymdrGuT6rhXFQLyzSLQH q60mH3A0qodFjUE+Svmh3cG0KhH/DQwuvRjAoHRG2SGJeJKJ3I30DgjBKv5XD3YMLsvg DwklvH8NQvIdAMV8KEB/5C2DB1qNTIA8iqXhmKrTHlm8WQIXBDNfHxkAe+zgFSE2QgVp lePAO/I8b7N6MlUJ8xVhSkpef/swTkwJucGrU3J5O7NgPN8gTpkqUD9WIPUOHh+R54lj CHAg== X-Gm-Message-State: APjAAAVjhpT7BjLRe+8LPjBQzlsJlZD30ZOzyquuC3YmOqG4YJrO90lD MRUmeFjCpQfPEDwCp3OuwKxRPQYV X-Google-Smtp-Source: APXvYqy7nArVo5u4IYWvdoqc8oxhO7uUhpXeXYRhanP8G8xSL3hFr1zKQYHIQ2XxwiHHMPyxpwkmyg== X-Received: by 2002:a81:2ecc:: with SMTP id u195mr344735ywu.39.1575137050387; Sat, 30 Nov 2019 10:04:10 -0800 (PST) Received: from [192.168.1.8] (static-50-53-66-203.bvtn.or.frontiernet.net. [50.53.66.203]) by smtp.gmail.com with ESMTPSA id b12sm11289698ywb.35.2019.11.30.10.04.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 30 Nov 2019 10:04:09 -0800 (PST) Subject: Re: Make autovacuum sort tables in descending order of xid_age To: David Fetter , PostgreSQL Development Cc: Christophe Pettus References: <20191129180138.GD8731@fetter.org> <20191129222152.GE8731@fetter.org> From: Mark Dilger Message-ID: <2488acea-328b-1328-d437-8c156bc8f7a4@gmail.com> Date: Sat, 30 Nov 2019 10:04:07 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: <20191129222152.GE8731@fetter.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 11/29/19 2:21 PM, David Fetter wrote: > On Fri, Nov 29, 2019 at 07:01:39PM +0100, David Fetter wrote: >> Folks, >> >> Per a suggestion Christophe made, please find attached a patch to >> $Subject: >> >> Apart from carefully fudging with pg_resetwal, and short of running in >> production for a few weeks, what would be some good ways to test this? > > Per discussion on IRC with Sehrope Sarkuni, please find attached a > patch with one fewer bug, this one in the repalloc() calls. Hello David, Here are my initial thoughts. Although you appear to be tackling the problem of vacuuming tables with older Xids first *per database*, have you considered changing the logic in building and sorting the database list in get_database_list and rebuild_database_list? I'm just curious what your thoughts might be on this subject. As far as sorting the list of tables in an array and then copying that array into a linked list, I think there is no need. The copying of table_ages into table_oids is followed immediately by foreach(cell, table_oids) and then table_oids seems not to serve any further purpose. Perhaps you can just iterate over table_ages directly and avoid the extra copying. I have not tested this change, but I may do so later today or perhaps on Monday. -- Mark Dilger