From: Thomas SIMON <tsimon@neteven.com>
To: Magnus Hagander <magnus@hagander.net>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: Weird behavior with unique constraint not respected, and random results on same queries
Date: Thu, 15 Oct 2020 08:53:49 +0200
Message-ID: <40baef4b-b604-95f4-b467-3c347e095a2b@neteven.com> (raw)
In-Reply-To: <50676167-ccd7-6187-93ce-7dd082de3dd7@neteven.com>
References: <01d4165c-201a-971f-b1e4-acea04872238@neteven.com>
<CABUevEwX+YrJ42LQJJM6R40Fh+e4zOsGrkEceoCyHiuJf9oz=Q@mail.gmail.com>
<50676167-ccd7-6187-93ce-7dd082de3dd7@neteven.com>
Hi Magnus,
Just for the record, you got it right with locale data change, after
index rebuild, we no longer have duplicates.
thanks again, have a good day
thomas
Le 12/10/2020 à 19:47, Thomas SIMON a écrit :
>
> Hi Magnus,
>
> thank you for you quick and relevant answer, you saved my day !
>
> Le 12/10/2020 à 17:53, Magnus Hagander a écrit :
>>
>>
>> On Mon, Oct 12, 2020 at 5:43 PM Thomas SIMON <tsimon@neteven.com
>> <mailto:tsimon@neteven.com>> wrote:
>>
>> Hi all,
>>
>> I encounter strange behavior since a few days, and the promote of a
>> recently installed server as master in my infrastructure (debian
>> 10, was
>> added as slave a few days ago)
>> I use postgresql 12.4 on master an slave.
>>
>>
>> Is the previous master also Debian 10?
> Yes old master was debian 9
>>
>> Specifically, when upgrading to Debian 10 you need to reindex all
>> indexes on text fields (see
>> https://wiki.postgresql.org/wiki/Locale_data_changes
>> <https://wiki.postgresql.org/wiki/Locale_data_changes;). This can in
>> particular cause problems in replication, whereby your primary and
>> standby nodes have to run the same version of Debian since you cannot
>> do a separate reindex on the standby.
>>
>> E.g. if your master is Debian 9, then you can expect some lookups to
>> be incorrect on the standby, and after you do a failover you may see
>> entries with invalid values in a unique index go in because of the
>> lack of reindex.
> I haven't found this note before, ans it seems indeed to describe the
> exact problem that we have...
> I just upgraded old master to avoid these kind of problems.
>>
>>
>> Since the promote, I find on database some duplicated entries ,
>> despite
>> the fact that the constraint should not allow this.
>> It seems to only happens with entries having "special" characters
>> (understand " ", "-", "+", ...)
>>
>>
>> As you can see from the locale page above, the changes are
>> specifically around special characters, so this is an indication that
>> it might be this problem.
> Exactly yes
>>
>> If this is the problem, the fix is a REINDEX and to make sure that
>> all nodes in the replication cluster use the same version of the
>> collections in the future. If the reindex fails because of existing
>> duplicate entries, you will have to find a way to clean those up
>> before reindexing. You may have to drop the indexes, then clean up,
>> and then recreate them.
>
> Yes, we are working on a script which will delete all duplicate
> values, and then we will use reindex with concurrently option on all
> listed lindexs
>
> We'll do these actions tomorrow morning, and I'll keep you updated
> about the result.
>
>>
>> --
>> Magnus Hagander
>> Me: https://www.hagander.net/ <http://www.hagander.net/;
>> Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/;
--
Thomas SIMON
Responsable Infrastructures
Neteven
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: tsimon@neteven.com, magnus@hagander.net, pgsql-admin@lists.postgresql.org
Subject: Re: Weird behavior with unique constraint not respected, and random results on same queries
In-Reply-To: <40baef4b-b604-95f4-b467-3c347e095a2b@neteven.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox