MIME-Version: 1.0
References: 
 <CAEphvJPSh1Cd+NA14Q3e-1B9JH6Ebm1oLgLx93HkkTZa=EBA_g@mail.gmail.com>
In-Reply-To: 
 <CAEphvJPSh1Cd+NA14Q3e-1B9JH6Ebm1oLgLx93HkkTZa=EBA_g@mail.gmail.com>
From: Ron Johnson <ronljohnsonjr@gmail.com>
Date: Sun, 15 Dec 2024 21:14:53 -0500
Message-ID: 
 <CANzqJaDAo9eLZpwbh0MJhj3RKvjhRJF4PheRUOtHRSjL9gbwBw@mail.gmail.com>
Subject: Re: Performance
To: Pgsql-admin <pgsql-admin@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="000000000000aa0053062959bf05"
Archived-At: 
 <https://www.postgresql.org/message-id/CANzqJaDAo9eLZpwbh0MJhj3RKvjhRJF4PheRUOtHRSjL9gbwBw%40mail.gmail.com>
Precedence: bulk

--000000000000aa0053062959bf05
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, Dec 15, 2024 at 8:22=E2=80=AFPM Anex Hul <anexsql2014@gmail.com> wr=
ote:

> Hello everyone,
>
> Testing 100 million records data import from Azure blob storage to Azure
> postgresql. I did run the test 5 times and the time it took keep increasi=
ng
> for each run.
> Is there know justification for this linear increment of the time it took
> for same size of data?
>

1. What version of PG is it?  ("SELECT VERSION();" should tell you.)
2. Are you truncating the table after each test run, or deleting all
records, or appending?
3. Is the blob data stored in BYTEA column data, or are you using the
(discouraged) "Large Objects"?
4. How are you loading the blob data?

--=20
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

--000000000000aa0053062959bf05
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Sun, Dec 15, 2024 at 8:22=E2=80=AFPM A=
nex Hul &lt;<a href=3D"mailto:anexsql2014@gmail.com">anexsql2014@gmail.com<=
/a>&gt; wrote:</div><div class=3D"gmail_quote gmail_quote_container"><block=
quote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1=
px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto">Hello everyon=
e,<div dir=3D"auto"><br></div><div dir=3D"auto">Testing 100 million records=
 data import from Azure blob storage to Azure postgresql. I did run the tes=
t 5 times and the time it took keep increasing for each run.=C2=A0</div><di=
v dir=3D"auto">Is there know justification for this linear increment of the=
 time it took for same size of data?</div></div>
</blockquote></div><div><br clear=3D"all"></div><div>1. What version of PG =
is it?=C2=A0 (&quot;SELECT VERSION();&quot; should tell you.)</div><div>2. =
Are you truncating the table after each test run, or deleting all records, =
or appending?</div><div>3. Is the blob data stored in BYTEA column data, or=
 are you using the (discouraged) &quot;Large Objects&quot;?</div><div>4. Ho=
w are you loading the blob data?</div><div><br></div><span class=3D"gmail_s=
ignature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><=
div dir=3D"ltr">Death to &lt;Redacted&gt;, and butter sauce.<div>Don&#39;t =
boil me, I&#39;m still alive.<br><div><div>&lt;Redacted&gt; lobster!</div><=
/div></div></div></div></div>

--000000000000aa0053062959bf05--