MIME-Version: 1.0
In-Reply-To: <49CCE1C9-1DD1-4E71-A96F-3924A2571BDB@postgresql.org>
References: 
 <4fe11dc1b660433594624b280b8ad832@SC58MEXGP036.CORP.CHARTERCOM.com>
 <3d7cd4673a80438798ad20b83411e3ca@SC58MEXGP001.CORP.CHARTERCOM.com>
 <1646547df04f4555a548ad7815208d5f@SC58MEXGP036.CORP.CHARTERCOM.com>
 <20180105175438.GO2416@tamriel.snowman.net>
 <68e5e2554e2240c6a22bd384fa7c9659@SC58MEXGP036.CORP.CHARTERCOM.com>
 <20180105180019.GP2416@tamriel.snowman.net>
 <7F963E79-CF4F-4E93-AA46-36E1817332D0@blighty.com>
 <630D932F-79D5-41C8-9007-06E919BB910A@postgresql.org>
 <CABUevEy+m5V9baEqhLeKTmc=EEh1p6H3Vs7O_GM3O8JmWX6S0g@mail.gmail.com>
 <49CCE1C9-1DD1-4E71-A96F-3924A2571BDB@postgresql.org>
From: Simon Riggs <simon@2ndquadrant.com>
Date: Sat, 6 Jan 2018 18:15:15 +0000
Message-ID: 
 <CANP8+jJ+qfuoiZ8RZq9Wr+6FHKM0aqktoUK2EdHCNwAz54FbpQ@mail.gmail.com>
Subject: Re: Is this still accurate?
To: "Jonathan S. Katz" <jkatz@postgresql.org>
Cc: Magnus Hagander <magnus@hagander.net>, Steve Atkins <steve@blighty.com>,
	"pgsql-docs@postgresql.org" <pgsql-docs@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On 6 January 2018 at 16:35, Jonathan S. Katz <jkatz@postgresql.org> wrote:
> Hi,
>
> On Jan 6, 2018, at 9:45 AM, Magnus Hagander <magnus@hagander.net> wrote:
>
>
>
> On Fri, Jan 5, 2018 at 8:09 PM, Jonathan S. Katz <jkatz@postgresql.org>
> wrote:
>>
>> Hi,
>>
>> On Jan 5, 2018, at 1:33 PM, Steve Atkins <steve@blighty.com> wrote:
>>
>>
>> On Jan 5, 2018, at 10:00 AM, Stephen Frost <sfrost@snowman.net> wrote:
>>
>> Greetings,
>>
>> * Moser, Glen G (Glen.Moser@charter.com) wrote:
>>
>> That's really the gist of the concern from a team member of mine.  Not
>> that the 4TB number is wrong but that it could be misleading to assume t=
hat
>> 4TB is some sort of upper bound.
>>
>> That's how this concern was relayed to me and I am just following up.
>>
>>
>> Well, saying 'in excess of' is pretty clear, but I don't think the
>> sentence is really adding much either, so perhaps we should just remove
>> it.
>>
>>
>> It's been useful a few times to reassure people that we can handle "larg=
e"
>> databases operationally, rather than just having large theoretical limit=
s.
>>
>> Updating it would be great, or wrapping a little more verbiage around th=
e
>> 4TB number, but a mild -1 on removing it altogether.
>>
>>
>> Here is a proposed patch that updates the wording:
>>
>> "There are active PostgreSQL instances in production environments that
>> manage many terabytes of data, as well as clusters managing petabytes.=
=E2=80=9D
>>
>> The idea is that it gives a sense of scope for how big instances/cluster=
s
>> can run without fixing people on a number.  People can draw their own
>> conclusions from the hard limits further down the page.
>>
> +1.

I don't think that's as useful, so -1 for removing the stated limit.

People always ask "how big can it go?" and having a specific number
there is important. We have publicly documented cases above 50TB, so I
think we should say that.

Clusters in Petabyte range? We need to be able to substantiate that
with publicly documented cases. They also need to be pure PostgreSQL,
not "with added tech", no?


Also, I can't see that the 1.6 TB per row is accurate, because that
would mean 1600 toast pointers at 20 bytes each =3D 32000 bytes, which
is above what we can normally support with 8kB blocksize as we
normally shipped.

Lastly, the "per table limit" should really say "32 TB per table, 128
PB for a partitioned table (4000 partitions)"

--=20
Simon Riggs                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services