pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Philip Warner <pjw@rhyme.com.au>
To: Ross J. Reedstrom <reedstrm@rice.edu>
To: Cristi Petrescu-Prahova <cristipp@lasting.ro>
Cc: pgsql-sql@postgresql.org
Subject: Re: how to continue a transaction after an error?
Date: Tue, 14 Nov 2000 16:30:35 +1100
Message-ID: <3.0.5.32.20001114163035.02bac100@mail.rhyme.com.au> (raw)
In-Reply-To: <20001113140808.A12500@rice.edu>
References: <002e01c04da9$a8f95c20$25efe6c1@lasting.ro>
	<002e01c04da9$a8f95c20$25efe6c1@lasting.ro>

At 14:08 13/11/00 -0600, Ross J. Reedstrom wrote:
>On Mon, Nov 13, 2000 at 09:41:04PM +0200, Cristi Petrescu-Prahova wrote:
>> Hello,
>> 
>> I would like to insert a bunch of rows in a table in a transaction. Some of
>> the insertions will fail due to constraints violation. When this happens,
>> Postgres automatically ends the transaction and rolls back all the previous
>> inserts. I would like to continue the transaction and issue the
>> commit/rollback command myself.
>> 
>> How to do it?
>> Is there any setting I am missing?
>> Is it possible at all?
>

Not possible; the error handling in PGSQL is a bit of a mess (not
necessarily a fault of PG), and it's not possible (currently) to rollback
single statements inside a larger transaction. This feature has to come,
but *not* AFAICT in the next release.


>But seriously, this comes up from time to time. PostgreSQL is a little
>stricter than most DBMSs with regards to transactional semantics, but
>there are good reasons for this, involving tradeoffs of locking, MVCC,
>"autocommit" mode, etc.

Not to mention the fact that we did not support sub-transactions. But I
think WAL does this for us, so we can hope for the feature RSN.


>When you start a transaction,
>you're telling the backend "treat all of these statements as one, big,
>all or nothing event." 

This is actually contrary to the standard. Statements are atomic, and a
failed statement should not abort the TX:

    The execution of all SQL-statements other than SQL-control 
    statements is atomic with respect to recovery. Such an 
    SQL-statement is called an atomic SQL-statement.

    ...

    An SQL-transaction cannot be explicitly terminated within an 
    atomic execution context. If the execution of an atomic 
    SQL-statement is unsuccessful, then the changes to SQL-data or schemas
    made by the SQL-statement are canceled.


>If you want (need, if you're using large objects) transactions, you
>really need to think about your transaction boundries. Don't just wrap
>your whole frontend in one big, long lived transaction

Totally agree; transactions will keep locks. Release them as soon as the
business rules and application design says that you can. Note that
commit-time constraints may make the commit fail; in this case PG will
force a rollback, but it *should* allow corrective action and another
attempt at a commit.


>close and reopen
>your transaction for those inserts that are allowed to fail.

This is very good advice for PGSQL, but bad advice for general DB
programming. At the end of the day, the database is the final arbiter of
valid data (through triggers, constraints etc that implement business
rules). Since we don't want to duplicate all of the rules from the database
within the application, we need to rely on the database telling us that the
last operation failed so that we can *choose* to rollback or choose to
change the processing. 

A classic example would be processing batches of data - open a file, read a
line, insert it, if the data is bad, insert it into a list of exceptions,
keep reading file; this needs to be done in one TX since after commit we
delete the file, and we don't want partially loaded batches if the machine
crashes. I admit this example demonstrates my age, but this kind of
processing is still necessary.


----------------------------------------------------------------
Philip Warner                    |     __---_____
Albatross Consulting Pty. Ltd.   |----/       -  \
(A.B.N. 75 008 659 498)          |          /(@)   ______---_
Tel: (+61) 0500 83 82 81         |                 _________  \
Fax: (+61) 0500 83 82 82         |                 ___________ |
Http://www.rhyme.com.au          |                /           \|
                                 |    --________--
PGP key available upon request,  |  /
and from pgp5.ai.mit.edu:11371   |/



view thread (10+ messages)  latest in thread

Message-ID: <3.0.5.32.20001114163035.02bac100@mail.rhyme.com.au>
Permalink:  ../3.0.5.32.20001114163035.02bac100@mail.rhyme.com.au/
Also on:    postgresql.org/message-id/3.0.5.32.20001114163035.02bac100@mail.rhyme.com.au

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-sql@postgresql.org
  Cc: pjw@rhyme.com.au, reedstrm@rice.edu, cristipp@lasting.ro
  Subject: Re: how to continue a transaction after an error?
  In-Reply-To: <3.0.5.32.20001114163035.02bac100@mail.rhyme.com.au>
  References: <002e01c04da9$a8f95c20$25efe6c1@lasting.ro> <3.0.5.32.20001114163035.02bac100@mail.rhyme.com.au>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox