pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedGauging progress of COPY?
6+ messages / 6 participants
[nested] [flat]
* Gauging progress of COPY?
@ 2022-01-20 18:00 Wells Oliver <wells.oliver@gmail.com>
0 siblings, 3 replies; 6+ messages in thread
From: Wells Oliver @ 2022-01-20 18:00 UTC (permalink / raw)
To: pgsql-admin
Is there some clever way to gauge the progress of a COPY command? i.e. you
know it has 10m rows, but you can't SELECT COUNT(*).
--
Wells Oliver
wells.oliver@gmail.com <wellsoliver@gmail.com>
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Gauging progress of COPY?
@ 2022-01-20 18:13 Tom Lane <tgl@sss.pgh.pa.us>
parent: Wells Oliver <wells.oliver@gmail.com>
2 siblings, 0 replies; 6+ messages in thread
From: Tom Lane @ 2022-01-20 18:13 UTC (permalink / raw)
To: Wells Oliver <wells.oliver@gmail.com>; +Cc: pgsql-admin
Wells Oliver <wells.oliver@gmail.com> writes:
> Is there some clever way to gauge the progress of a COPY command? i.e. you
> know it has 10m rows, but you can't SELECT COUNT(*).
If it's a COPY IN (to a table) you could monitor growth of the
physical size of the table, cf pg_relation_size(). This isn't
terribly satisfactory if you're not clear on how much space each
row will take, but it's something.
regards, tom lane
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Gauging progress of COPY?
@ 2022-01-20 18:16 Ron <ronljohnsonjr@gmail.com>
parent: Wells Oliver <wells.oliver@gmail.com>
2 siblings, 1 reply; 6+ messages in thread
From: Ron @ 2022-01-20 18:16 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
On 1/20/22 12:00 PM, Wells Oliver wrote:
> Is there some clever way to gauge the progress of a COPY command? i.e. you
> know it has 10m rows, but you can't SELECT COUNT(*).
Use a specific tool like pg_bulkload.
--
Angular momentum makes the world go 'round.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Gauging progress of COPY?
@ 2022-01-21 19:50 Stephen Frost <sfrost@snowman.net>
parent: Ron <ronljohnsonjr@gmail.com>
0 siblings, 0 replies; 6+ messages in thread
From: Stephen Frost @ 2022-01-21 19:50 UTC (permalink / raw)
To: Ron <ronljohnsonjr@gmail.com>; +Cc: pgsql-admin@lists.postgresql.org
Greetings,
* Ron (ronljohnsonjr@gmail.com) wrote:
> On 1/20/22 12:00 PM, Wells Oliver wrote:
> >Is there some clever way to gauge the progress of a COPY command? i.e. you
> >know it has 10m rows, but you can't SELECT COUNT(*).
>
> Use a specific tool like pg_bulkload.
Or maybe just use pv if you're doing it through psql or such.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Gauging progress of COPY?
@ 2022-01-22 10:09 Magnus Hagander <magnus@hagander.net>
parent: Wells Oliver <wells.oliver@gmail.com>
2 siblings, 1 reply; 6+ messages in thread
From: Magnus Hagander @ 2022-01-22 10:09 UTC (permalink / raw)
To: Wells Oliver <wells.oliver@gmail.com>; +Cc: pgsql-admin
On Thu, Jan 20, 2022, 19:01 Wells Oliver <wells.oliver@gmail.com> wrote:
> Is there some clever way to gauge the progress of a COPY command? i.e. you
> know it has 10m rows, but you can't SELECT COUNT(*).
>
If you can open a second connection, you can just query
pg_stat_progress_copy.
/Magnus
>
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Gauging progress of COPY?
@ 2022-01-22 16:05 Greg Spiegelberg <gspiegelberg@gmail.com>
parent: Magnus Hagander <magnus@hagander.net>
0 siblings, 0 replies; 6+ messages in thread
From: Greg Spiegelberg @ 2022-01-22 16:05 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Wells Oliver <wells.oliver@gmail.com>; pgsql-admin
On Sat, Jan 22, 2022, 3:09 AM Magnus Hagander <magnus@hagander.net> wrote:
>
>
> On Thu, Jan 20, 2022, 19:01 Wells Oliver <wells.oliver@gmail.com> wrote:
>
>> Is there some clever way to gauge the progress of a COPY command? i.e.
>> you know it has 10m rows, but you can't SELECT COUNT(*).
>>
>
>
> If you can open a second connection, you can just query
> pg_stat_progress_copy.
>
If COPY FROM STDIN the client can provide feedback since it is sending line
by line.
If size known...1%...5%...
If number of rows known... 10000... 20000...
I would think even a semi capable bash script could do it.
-Greg
>
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2022-01-22 16:05 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-01-20 18:00 Gauging progress of COPY? Wells Oliver <wells.oliver@gmail.com>
2022-01-20 18:13 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-01-20 18:16 ` Ron <ronljohnsonjr@gmail.com>
2022-01-21 19:50 ` Stephen Frost <sfrost@snowman.net>
2022-01-22 10:09 ` Magnus Hagander <magnus@hagander.net>
2022-01-22 16:05 ` Greg Spiegelberg <gspiegelberg@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox