MIME-Version: 1.0
References: <CAPqV3pRO=u1Mno06WLCFxoyxbzkYTetiqTs7WaW=_fkjRPgYhw@mail.gmail.com>
 <CAPqV3pQquw63Fv_fCJzAAnUWk86qHfg1cH=+mJ2N=4LGdDyrNw@mail.gmail.com>
 <CAKFQuwb2MtE02QgMQ3vvLOj7bvGMar6qwnigCBBaYLBKsdk3xg@mail.gmail.com> <2805642.1714793392@sss.pgh.pa.us>
In-Reply-To: <2805642.1714793392@sss.pgh.pa.us>
From: Siddharth Jain <siddhsql@gmail.com>
Date: Tue, 7 May 2024 09:48:38 -0700
Message-ID: <CAPqV3pTe+b3Q44jdvhAtZNnroteRLp7kHFLvWFypu0JDwtaYjA@mail.gmail.com>
Subject: Re: Question regarding how databases support atomicity
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: "David G. Johnston" <david.g.johnston@gmail.com>, 
	"pgsql-general@lists.postgresql.org" <pgsql-general@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="000000000000c93aa00617dff538"
Archived-At: <https://www.postgresql.org/message-id/CAPqV3pTe%2Bb3Q44jdvhAtZNnroteRLp7kHFLvWFypu0JDwtaYjA%40mail.gmail.com>
Precedence: bulk

--000000000000c93aa00617dff538
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thanks All for the kind responses. I understand how MVCC takes care of
atomicity for updates to rows. I was developing a project where lets say
data for each table is stored in its own folder together with metadata (we
are not talking postgres now). So if I have two tables A and B I have a
folder structure like:
A
\_ metadata.json
B
\_ metadata.json
Now if I want to rename a table, I need to move the folder and also update
metadata accordingly. These are two separate operations but need to be done
atomically - all or none. in this case it is possible that we succeed in
renaming the folder but fail to update metadata for whatever reason. then
if we try to undo the folder rename we get another failure for whatever
reason. how to deal with such scenarios? are there no such scenarios in
postgres?


On Fri, May 3, 2024 at 8:29=E2=80=AFPM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> "David G. Johnston" <david.g.johnston@gmail.com> writes:
> > On Friday, May 3, 2024, Siddharth Jain <siddhsql@gmail.com> wrote:
> >> The way I understand this is that if there is a failure in-between, we
> >>> start undoing and reverting the previous operations one by one.
>
> > Not in PostgreSQL.  All work performed is considered provisional until =
a
> > commit succeeds.  At which point all provisional work, which had been
> > tagged with the same transaction identifier, becomes reality to the res=
t
> of
> > the system, by virtue of marking the transaction live.
>
> Right.  We don't use UNDO; instead, we use multiple versions of
> database rows (MVCC).  A transaction doesn't modify the contents
> of existing rows, but just marks them as provisionally outdated, and
> then inserts new row versions that are marked provisionally inserted.
> Other transactions ignore the outdate markings and the uncommitted new
> rows, until the transaction commits, at which time the new versions
> become live and the old ones become dead.  If the transaction never
> does commit -- either through ROLLBACK or a crash -- then the old row
> versions remain live and the new ones are dead.  In either case, we
> don't have a consistency or correctness problem, but we do have dead
> rows that must eventually get vacuumed away to prevent indefinite
> storage bloat.  That can be done by background housekeeping processes
> though (a/k/a autovacuum).
>
> I believe Oracle, for one, actually does use UNDO.  I don't know
> what they do about failure-to-UNDO.
>
>                         regards, tom lane
>

--000000000000c93aa00617dff538
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Thanks All for the kind responses. I understand how MVCC t=
akes care of atomicity for updates to rows. I was developing a project wher=
e lets say data for each table is stored in its own folder together with me=
tadata (we are not talking postgres now). So if I have two tables A and B I=
 have a folder structure like:<div>A</div><div>\_ metadata.json</div><div>B=
</div><div>\_ metadata.json</div><div>Now if I want to rename a table, I ne=
ed to move the folder and also update metadata accordingly. These are two s=
eparate operations but need to be done atomically - all or none. in this ca=
se it is possible that we succeed in renaming the folder but fail to update=
 metadata for whatever reason. then if we try to undo the folder rename we =
get another failure for whatever reason. how to deal with such scenarios? a=
re there no such scenarios in postgres?</div><div><br></div></div><br><div =
class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, May 3, =
2024 at 8:29=E2=80=AFPM Tom Lane &lt;<a href=3D"mailto:tgl@sss.pgh.pa.us">t=
gl@sss.pgh.pa.us</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pa=
dding-left:1ex">&quot;David G. Johnston&quot; &lt;<a href=3D"mailto:david.g=
.johnston@gmail.com" target=3D"_blank">david.g.johnston@gmail.com</a>&gt; w=
rites:<br>
&gt; On Friday, May 3, 2024, Siddharth Jain &lt;<a href=3D"mailto:siddhsql@=
gmail.com" target=3D"_blank">siddhsql@gmail.com</a>&gt; wrote:<br>
&gt;&gt; The way I understand this is that if there is a failure in-between=
, we<br>
&gt;&gt;&gt; start undoing and reverting the previous operations one by one=
.<br>
<br>
&gt; Not in PostgreSQL.=C2=A0 All work performed is considered provisional =
until a<br>
&gt; commit succeeds.=C2=A0 At which point all provisional work, which had =
been<br>
&gt; tagged with the same transaction identifier, becomes reality to the re=
st of<br>
&gt; the system, by virtue of marking the transaction live.<br>
<br>
Right.=C2=A0 We don&#39;t use UNDO; instead, we use multiple versions of<br=
>
database rows (MVCC).=C2=A0 A transaction doesn&#39;t modify the contents<b=
r>
of existing rows, but just marks them as provisionally outdated, and<br>
then inserts new row versions that are marked provisionally inserted.<br>
Other transactions ignore the outdate markings and the uncommitted new<br>
rows, until the transaction commits, at which time the new versions<br>
become live and the old ones become dead.=C2=A0 If the transaction never<br=
>
does commit -- either through ROLLBACK or a crash -- then the old row<br>
versions remain live and the new ones are dead.=C2=A0 In either case, we<br=
>
don&#39;t have a consistency or correctness problem, but we do have dead<br=
>
rows that must eventually get vacuumed away to prevent indefinite<br>
storage bloat.=C2=A0 That can be done by background housekeeping processes<=
br>
though (a/k/a autovacuum).<br>
<br>
I believe Oracle, for one, actually does use UNDO.=C2=A0 I don&#39;t know<b=
r>
what they do about failure-to-UNDO.<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 regards, tom lane<br>
</blockquote></div>

--000000000000c93aa00617dff538--