References: <53F7ACF4.1010604@encs.concordia.ca>
	<CAMkU=1xWQdCs0nbZSXrkdK4Y7moLKWV8X60MSdDSw-oR27L7yw@mail.gmail.com>
	<CAOnt2=NUbNPNvy3x-TcwCm-vLcNT7jZvX=Z5Ja6jAoX22j3RuA@mail.gmail.com>
	<1409092428.41025.YahooMailNeo@web122306.mail.ne1.yahoo.com>
 <CAOnt2=MG4L3Prb=AQ3Etqs3kOZbPDw7Y-ap0MHn75GR0LfV2Fg@mail.gmail.com>
Message-ID: <1409144369.45962.YahooMailNeo@web122306.mail.ne1.yahoo.com>
Date: Wed, 27 Aug 2014 05:59:29 -0700
From: Kevin Grittner <kgrittn@ymail.com>
Reply-To: Kevin Grittner <kgrittn@ymail.com>
Subject: Re: autocommit (true/false) for more than 1 million records
To: Alex Goncharov <alex.goncharov.usa@gmail.com>
Cc: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>
In-Reply-To: 
 <CAOnt2=MG4L3Prb=AQ3Etqs3kOZbPDw7Y-ap0MHn75GR0LfV2Fg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

Alex Goncharov <alex.goncharov.usa@gmail.com> wrote:
> Kevin Grittner <kgrittn@ymail.com> wrote:

>> The rows will all be in the table, but not visible to any other
>> transaction.
>
> How much data can I fit there while doing COPY?=A0 Not 1 TB?

As has already been said, why not?=A0 This is not some special
section of the table -- the data is written to the table.=A0 Period.
Commit or rollback just tells new transactions whether data flagged
with that transaction number is visible.

Nobody can tell you how much space that will take -- it depends on
many factors, including how many columns of what kind of data, how
compressible it is, and how it is indexed.=A0 But the point is, we
are not talking about any separate space from what is needed to
store the data in the database.

FWIW, I think the largest single COPY statement I ever ran was
generated by pg_dump and piped directly to psql for a major release
upgrade (before pg_upgrade was available), and it was somewhere in
the 2TB to 3TB range.=A0 It took a long time, but it "just worked".
That should be true for 10TB or 100TB, as long as you have sized
the machine correctly and are loading clean data.=A0 Whether you have
that covered, and how you want to "hedge your bets" based on your
degree of confidence in those things is a judgment call.=A0 When I'm=20
in the position of needing to make such a call, I like to do some=20
tests.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


--=20
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance