From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Josef Šimánek <josef.simanek@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: [PATCH] Simple progress reporting for COPY command
Date: Thu, 7 Jan 2021 16:54:51 +0100
Message-ID: <4c283c4b-bafe-2327-b41d-5caaa296dedf@enterprisedb.com> (raw)
In-Reply-To: <CAFp7QwqFkrRWv6BY=prTVWWvYY+O0=Bt+kE978twY7vAdf_GeA@mail.gmail.com>
References: <CAFp7Qwr6_FmRM6pCO0x_a0mymOfX_Gg+FEKet4XaTGSW=LitKQ@mail.gmail.com>
<CALj2ACWAAbVuTvnLAL3B82ZLwyQaF6nteFsABABwxPJqL7gP1A@mail.gmail.com>
<CAFp7QwoWx2voi2j5oaFS4Zk1tVs+OAKb8YqGZTtUVxGqwL8xgA@mail.gmail.com>
<6e7b056e-faff-40f6-730d-3ad6d2d1b5e9@enterprisedb.com>
<CAFp7QwqKp6_iM0R7R+f8pjwWfD+xxgm2NbDLLzj3+2RhMhcJog@mail.gmail.com>
<CAFp7Qwop_j94NaZ4OoOSNpCiLY3m+eq6WO3RmCWCw7Y0tOL_Fw@mail.gmail.com>
<1562ca9b-ffb5-e5e9-0399-33db90dbe153@enterprisedb.com>
<CAFp7QwqFkrRWv6BY=prTVWWvYY+O0=Bt+kE978twY7vAdf_GeA@mail.gmail.com>
On 1/7/21 12:06 PM, Josef Šimánek wrote:
> st 6. 1. 2021 v 22:44 odesílatel Tomas Vondra
> <tomas.vondra@enterprisedb.com> napsal:
>>
>> On 1/5/21 11:02 AM, Josef Šimánek wrote:
>>> I'm attaching the whole patch since commitfest failed to ingest the
>>> last incremental on CI.
>>>
>>
>> Yeah, the whole patch needs to be attached for the commitfest tester to
>> work correctly - it can't apply pieces from multiple messages, etc.
>>
>> Anyway, I pushed this last version of patch, after a couple more tweaks,
>> mainly to the docs - one place used pg_stat_copy_progress, the section
>> was not indexed properly, and so on.
>>
>> I see Matthias proposed to change "lines" to "tuples" - I only saw the
>> message after pushing, but I probably wouldn't make that change anyway.
>> The CSV docs seem to talk about lines, newlines etc. so it seems fine.
>> If not, we can change that.
>>
>> One more question, though - I now realize the lines_processed ignores
>> rows skipped because of BEFORE INSERT triggers. I wonder if that's the
>> right thing to do? Imagine you know the number of lines in a file. You
>> can't really use (lines_processed / total_lines) to measure progress,
>> because that may ignore many "skipped" rows. So maybe this should be
>> changed to count all rows. OTOH we still have bytes_processed.
>
> I think that should be fixed. It is called "lines_processed" not
> "lines_inserted". I'll take a look.
>
So we may either rename the column to "lines_inserted", or tweak the
code to count all processed lines. Or track both and have two columns.
regarss
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: tomas.vondra@enterprisedb.com, josef.simanek@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: [PATCH] Simple progress reporting for COPY command
In-Reply-To: <4c283c4b-bafe-2327-b41d-5caaa296dedf@enterprisedb.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox