From: Richard Huxton <dev@archonet.com>
To: Glenn Maynard <glenn@zewt.org>
Cc: pgsql-sql@postgresql.org
Subject: Re: Bit by "commands ignored until end of transaction block" again
Date: Thu, 23 Jul 2009 06:31:34 +0100
Message-ID: <4A67F5B6.4080409@archonet.com> (raw)
In-Reply-To: <bd36f99e0907222158k6b4b3a0al65766c0d9c4906cf@mail.gmail.com>
References: <bd36f99e0907222158k6b4b3a0al65766c0d9c4906cf@mail.gmail.com>
Glenn Maynard wrote:
> Postgres kills a transaction when an error happens. This is a pain;
> it assumes that all SQL errors are unexpected and fatal to the
> transaction.
>
> There's a very simple case where it's not: UNIQUE checks.
Ah, it's usually "syntax errors" wrt interactive sessions.
> I'm
> generating a cache, with a simple flow:
> - Search for the cache key; if it exists, return its value.
> - If it didn't exist, create the data based on the key, insert it
> into the table, and return it.
>
> This has an obvious race: another thread looks up the same key and
> creates it between the search and the insert. Both threads will
> create the cached data, thread A will insert it into the table, and
> thread B will get an integrity error when it tries to insert it, since
> it duplicates the unique key.
Yep.
> Here, by far the simplest fix is simply to ignore the integrity error.
> Both threads generated the same data; the failed insert is expected
> and harmless. Postgres is turning this into a fatal error.
Well, all errors are considered fatal. But same difference.
> There's so much that could make this trivially easy:
>
> - SQLite has the handy ON CONFLICT IGNORE, but Postgres has nothing
> like that. (ON CONFLICT REPLACE is great, too.)
True. Been proposed. Fiddly to implement for all use-cases if I remember
correctly.
> - Let me use SAVEPOINT outside of a transaction,
You are never outside a transaction. All queries are executed within a
transaction.
> with the effect of
> starting a transaction with the savepoint and ending it when it's
> committed. Then, I could use savepoints without needing to know
> whether I'm already in a transaction or not; one would simply be
> started and committed for me if necessary. (That's by far my biggest
> issue with savepoints: they force me to either specify "a transaction
> must be open when this function is called", or need to be able to
> query whether one is running to decide whether to start a transaction
> or a savepoint. My function's use of transactions should be invisible
> to the caller.)
I think this is the root of your problem - all queries are within a
transaction so either:
1. You have a transaction that wraps a single statement. If you get an
error then only that statement was affected.
2. You have an explicit BEGIN...COMMIT transaction which could use a
savepoint.
Nothing to stop you setting savepoints in #1 (although they're not much
use).
Typically, if you're in a plpgsql function you would just catch "unique"
exception codes from your insert. Or, update, see if any rows were
affected, if not try an insert and if that gives a duplicate go back and
try the update. You might want the second approach if 99% of the time
the cache is already populated.
> - Let me disable this error. I don't want it. (We're grownups; we
> can decide for ourselves which errors are fatal.)
You could always try submitting a patch. However, that's really what
savepoints do - let you decide whether an error can be worked around.
> The first two are cleaner, since ignoring the error means I might
> ignore some other integrity error from the same statement, but I can
> live with that.
>
> Lacking anything better, I'll probably end up dropping out of the ORM
> and using some uglier SQL to work around this, but this is so trivial
> that it's silly to have to do that. I can't do it within the ORM; it
> doesn't have the vocabulary.
The ORM can't control transactions, can't call functions or can't set
savepoints?
--
Richard Huxton
Archonet Ltd
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: dev@archonet.com, glenn@zewt.org
Subject: Re: Bit by "commands ignored until end of transaction block" again
In-Reply-To: <4A67F5B6.4080409@archonet.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox