Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1erGNo-0000lE-Np for pgsql-admin@arkaria.postgresql.org; Thu, 01 Mar 2018 05:03:56 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1erGNn-0008F4-Jq for pgsql-admin@arkaria.postgresql.org; Thu, 01 Mar 2018 05:03:55 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1erGNn-0008Ep-9X for pgsql-admin@lists.postgresql.org; Thu, 01 Mar 2018 05:03:55 +0000 Received: from cat-porwal-prod-mail1.catalyst.net.nz ([2404:130:4080::4]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1erGNj-0005Kb-24 for pgsql-admin@lists.postgresql.org; Thu, 01 Mar 2018 05:03:53 +0000 Authentication-Results: auth=pass smtp.auth=mark.kirkwood@catalyst.net.nz smtp.mailfrom=mark.kirkwood@catalyst.net.nz Received: from [IPv6:2406:e001:b9c:1:c097:a723:68a3:d806] (unknown [IPv6:2406:e001:b9c:1:c097:a723:68a3:d806]) (Authenticated sender: mark.kirkwood@catalyst.net.nz) by cat-porwal-prod-mail1.catalyst.net.nz (Postfix) with ESMTPSA id 0652680E47; Thu, 1 Mar 2018 18:03:44 +1300 (NZDT) Subject: Re: Reliable WAL file shipping over unreliable network To: Rui DeSousa Cc: Dianne Skoll , pgsql-admin@lists.postgresql.org References: <400AC9CD-1E43-422C-A689-70BDBAFF231E@elevated-dev.com> <37C5A42B-4CDD-4F01-8F1E-1B185C1819B9@icloud.com> <20180228132620.2c882f84@hydrogen.roaringpenguin.com> <33220469-5BB2-4A40-8F56-8F4F392C95B3@icloud.com> <21AF0CB4-0873-4960-BA14-E72FA08B352E@icloud.com> <5a49c387-e4f4-f70c-537c-9c338c80b687@catalyst.net.nz> <9DDA6F85-077E-4F8D-8708-632CC2907D3C@icloud.com> <06dd8b81-8b1a-ae5c-8ddd-1556b5946176@catalyst.net.nz> <6056907B-551F-4ED0-804B-0FCACD39EB23@icloud.com> From: Mark Kirkwood Message-ID: Date: Thu, 1 Mar 2018 18:03:42 +1300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <6056907B-551F-4ED0-804B-0FCACD39EB23@icloud.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 01/03/18 17:57, Rui DeSousa wrote: > It looks like rsync is not handling the disk quota when actually syncing the file — Disc quota exceeded (69) vs no space left (28). It only fails if it can’t rename the truncated temp file — and in some cases there is just enough space left to allow the rename to succeed thus leaving a truncated WAL file. > > I’m using ZFS; and have 1GB quota on the given filesystem. > > [postgres@hades ~/dbc1/pg_wal]$ rsync -a 000000010000005B00000035 ~/arch/dbc1/wal/000000010000005B00000035 > rsync: rename "/usr/home/postgres/arch/dbc1/wal/.000000010000005B00000035.6bjO94" -> "000000010000005B00000035": Disc quota exceeded (69) > rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1200) [sender=3.1.2] > > Interesting - so could be rsync not handling the quota error sensibly - or perhaps more likely ZFS and/or quota bugs. If the latter the solution would be not to use these until the devs have fixed them... :-) regards Mark