Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ittxA-0007Bk-Tb for pgsql-bugs@arkaria.postgresql.org; Tue, 21 Jan 2020 13:52:25 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1ittx9-00064u-OZ for pgsql-bugs@arkaria.postgresql.org; Tue, 21 Jan 2020 13:52:23 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ittx9-00064n-DH for pgsql-bugs@lists.postgresql.org; Tue, 21 Jan 2020 13:52:23 +0000 Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1ittx4-0003XT-A7 for pgsql-bugs@lists.postgresql.org; Tue, 21 Jan 2020 13:52:23 +0000 Received: by mail-wr1-x443.google.com with SMTP id b6so3321340wrq.0 for ; Tue, 21 Jan 2020 05:52:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=2ndquadrant-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=7nxGxVegda0qtqdsCvWRLfXIGp3rwGPzrruMzGn1XDE=; b=MzTmhEVWDKVwicYnH2V37O0w7a71fpKjd1aacnXHYifmQGDjCHZKnldY3FH3neY6T1 CvTaGb7uQ+r3Gv0sk3CPVvsoDYw7DuUWn5VaU81c3rnVqzRUXQx1NRlE3vEWRycBA7C4 xCwQS1D0Qf+T5u3VEP5itPqBlUwqzi28tsIy7HY3lXPVITaMspGzDKd1zs8sOxP002u3 qXuLJjQNNzmWARK6YbmiRuxgoZ6AwH2vMc/ijt11zNAQ2xZQgs7IDGNgICcGcaEMr4By BSpHMVGI/IIG/ETxeGrza9tlNx/AcAwvTzTWx2gq90+XLeUTgq6J2tSLQ72whDRFXW4M lSjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=7nxGxVegda0qtqdsCvWRLfXIGp3rwGPzrruMzGn1XDE=; b=j+EntBGSSBgHdH7lFwfDuxOlQfeWVKCU6oq3laHIuTD2CxNPKuKjDqPlEEE4f3uP5o mH+WoCRUP2GvXeP4gswQmc21fuqGFq2ToeksJKvSWORlpf06ciU2nFLbm00APO9t7gig rLcR7rttqIiygrG5xPgICMx88VgWtIc3/0YAVfF0bBlOoPenTQbgdiwGToHiIreR7Qrs kXq+gKmwvZvgSHkDkTcyczqaQ83bqTT/FD4yG3+vBgAMsnOZ+M4HGf9vSC3GV1+r3W3k cFuVk3X1RC7yS9HP2GZITD4E+/04Dy4EWLISzUVhuowA9yx+SumdTFY2m3QVxeMUQxSB pFLQ== X-Gm-Message-State: APjAAAV+6+VZR0eOwkHt2YmMs7nW686eWnKAaWr3hqC3i1LbZKdTLqyg cswDvZb8JsP0XlF7grMm6G0HWv44DF4= X-Google-Smtp-Source: APXvYqwuVdIcgWaLz+EalUR8e40UxWjV2cgXmlvBg1R2+Vkfaj3xWAOVGqLdD3jzal27HoiIgbA/0Q== X-Received: by 2002:adf:f581:: with SMTP id f1mr5428517wro.264.1579614736412; Tue, 21 Jan 2020 05:52:16 -0800 (PST) Received: from localhost (static-84-42-175-93.net.upcbroadband.cz. [84.42.175.93]) by smtp.gmail.com with ESMTPSA id s1sm4370432wmc.23.2020.01.21.05.52.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jan 2020 05:52:15 -0800 (PST) Date: Tue, 21 Jan 2020 14:52:13 +0100 From: Tomas Vondra To: Radek Slupik Cc: pgsql-bugs@lists.postgresql.org Subject: Re: High table creation rate results in =?utf-8?B?4oCcRmlsZSBl?= =?utf-8?B?eGlzdHPigJ0=?= error Message-ID: <20200121135213.3ud6frut457d7yyp@development> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On Tue, Jan 21, 2020 at 02:04:24PM +0100, Radek Slupik wrote: >Hi, > >We occasionally run into an internal error when creating tables. The error >message is «could not create file "base/17006/2851173576": File exists» >with SQLSTATE 58P02. The attachment is included with this email contains >the PostgreSQL logs. Note that some identifiers are renamed in the logs >because they contain sensitive information, but identifier names are >unlikely the source of the error. > >An old mailing list thread [1] suggests a problem related to OID >wraparound. Our use case involves creating and dropping thousands of tables >every day. The CREATE TABLE statements for these tables are similar to the >one in the attached logs, but with different table names and columns. >Nonetheless, the error is difficult to reproduce as it happens only >sporadically. Retrying the CREATE TABLE statement fixes the problem >temporarily. > >Some more information about our PostgreSQL instance and usage: > > - PostgreSQL version: PostgreSQL 10.6 (Ubuntu 10.6-1.pgdg16.04+1) on > x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 > 20160609, 64-bit. > - The DBMS did not crash or abruptly shut down recently. > - The file base/17006/2851173576 was actually in use by an index, as > indicated by pg_class.relfilenode. > - Autovacuum is disabled for these tables. Logging is disabled for these > tables upon creation, but enabled after filling them with data. > - The query “SELECT max(oid) FROM pg_class” reports 4,294,603,797, which > is close to 2³² - 1. > So what was the actual rate of table creation? How long did it take to get to this state? What kernel/filesystem was used? What did lsof say? regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services