MIME-Version: 1.0
References: <CAEzWdqd-22B-bpVdT3yzegiOig9zvJnfJvi=GOMFfHT-Jg8CgQ@mail.gmail.com>
In-Reply-To: <CAEzWdqd-22B-bpVdT3yzegiOig9zvJnfJvi=GOMFfHT-Jg8CgQ@mail.gmail.com>
From: Juan Rodrigo Alejandro Burgos Mella <rodrigoburgosmella@gmail.com>
Date: Sat, 14 Sep 2024 23:51:49 -0300
Message-ID: <CAHbZ42wTJSHZL+0NVHyDNmi=v7xEGLKfoVei=Y=_URfW9ySHBg@mail.gmail.com>
Subject: Re: update faster way
To: yudhi s <learnerdatabase99@gmail.com>
Cc: pgsql-general <pgsql-general@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="0000000000005776ff06221f8a88"
Archived-At: <https://www.postgresql.org/message-id/CAHbZ42wTJSHZL%2B0NVHyDNmi%3Dv7xEGLKfoVei%3DY%3D_URfW9ySHBg%40mail.gmail.com>
Precedence: bulk

--0000000000005776ff06221f8a88
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

The only way that I see as plausible to use a subquery, both in the query
and in the setting of the variable, is that the relationship is one to one,
and that there is an index that responds to the predicate

UPDATE table1 t1
SET column_value =3D (SELECT <value> FROM table2 t2 WHERE t2.column_relatio=
n
=3D t1.column_relation)
WHERE (colum_relation) IN (SELECT column_relation FROM table2)

PD: the index of being in table2

Atte
JRBM

El s=C3=A1b, 14 sept 2024 a las 0:22, yudhi s (<learnerdatabase99@gmail.com=
>)
escribi=C3=B3:

> Hello,
> We have to update a column value(from numbers like '123' to codes like
> 'abc' by looking into a reference table data) in a partitioned table with
> billions of rows in it, with each partition having 100's millions rows. A=
s
> we tested for ~30million rows it's taking ~20minutes to update. So if we =
go
> by this calculation, it's going to take days for updating all the values.
> So my question is
>
> 1) If there is any inbuilt way of running the update query in parallel
> (e.g. using parallel hints etc) to make it run faster?
> 2) should we run each individual partition in a separate session (e.g.
> five partitions will have the updates done at same time from 5 different
> sessions)? And will it have any locking effect or we can just start the
> sessions and let them run without impacting our live transactions?
>
> UPDATE tab_part1
> SET column1 =3D reftab.code
> FROM reference_tab reftab
> WHERE tab_part1.column1 =3D subquery.column1;
>
> Regards
> Yudhi
>

--0000000000005776ff06221f8a88
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br>The only way that I see as plausible to use a subquery=
, both in the query and in the setting of the variable, is that the relatio=
nship is one to one, and that there is an index that responds to the predic=
ate<br><br>UPDATE table1 t1<div>SET column_value =3D (SELECT &lt;value&gt; =
FROM table2 t2 WHERE t2.column_relation =3D t1.column_relation)<br>WHERE (c=
olum_relation) IN (SELECT column_relation FROM table2)<br></div><div><br></=
div><div>PD:=C2=A0the index of being in table2</div><div><br></div><div>Att=
e</div><div>JRBM</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr"=
 class=3D"gmail_attr">El s=C3=A1b, 14 sept 2024 a las 0:22, yudhi s (&lt;<a=
 href=3D"mailto:learnerdatabase99@gmail.com">learnerdatabase99@gmail.com</a=
>&gt;) escribi=C3=B3:<br></div><blockquote class=3D"gmail_quote" style=3D"m=
argin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left=
:1ex"><div dir=3D"ltr">Hello,<br>We have to update a column value(from numb=
ers like &#39;123&#39; to codes like &#39;abc&#39; by looking into a refere=
nce table data) in a partitioned table with billions of rows in it, with ea=
ch partition having 100&#39;s millions rows. As we tested for ~30million ro=
ws it&#39;s taking ~20minutes to update. So if we go by this calculation, i=
t&#39;s going to take days for updating all the values. So my question is<b=
r><br>1) If there is any inbuilt way of running the update query in paralle=
l (e.g. using parallel hints etc) to make it run faster?<br>2) should we ru=
n each individual partition in a separate session (e.g. five partitions wil=
l have the updates done at same time from 5 different sessions)? And will i=
t have any locking effect or we can just=C2=A0start the sessions and let th=
em run without impacting our live transactions?<br><br>UPDATE tab_part1<br>=
SET column1 =3D reftab.code<br>FROM reference_tab reftab<br>WHERE tab_part1=
.column1 =3D subquery.column1;<br><div><br></div><div>Regards</div><div>Yud=
hi</div></div>
</blockquote></div>

--0000000000005776ff06221f8a88--