Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pPqtA-0002JT-Mz for pgsql-hackers@arkaria.postgresql.org; Wed, 08 Feb 2023 20:17:56 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1pPqsA-0005F1-QW for pgsql-hackers@arkaria.postgresql.org; Wed, 08 Feb 2023 20:16:54 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pPqsA-0005Es-CN for pgsql-hackers@lists.postgresql.org; Wed, 08 Feb 2023 20:16:54 +0000 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by magus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1pPqs7-0002eC-Ce for pgsql-hackers@postgresql.org; Wed, 08 Feb 2023 20:16:53 +0000 Received: by mail-pl1-x633.google.com with SMTP id f6so84656pln.12 for ; Wed, 08 Feb 2023 12:16:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=j-davis-com.20210112.gappssmtp.com; s=20210112; h=mime-version:user-agent:references:in-reply-to:date:cc:to:from :subject:message-id:from:to:cc:subject:date:message-id:reply-to; bh=Tf3+zuSM5JW34sSXOaUP8rjPs47MJWvFo8GAD6hRTy4=; b=RYd2zZ97NzH1PL51dpLCabEBW9S0krrZbuezbY8LyIu49sMXoli5VVyFAEjIk5r6zq SfBgQo7S0ojKJZ5kBOJsFh+2uqNOLGaZTcGjMFfA1ZHbp2ljcetY3EaFCxoZ4aJo7JKK Yk/L/aUjickJn0lBcxkWn4bkCiu/+dajCpsCHVXOwHixFqG7mQATwpDoo+Z7uW1I9eeo 0VUvfvmhGGnqmi71skVTemtDcBBufSx7dQ4VXX/iUq12MDiXDm4Yrb0GieKKZXFUASnX YL7mwLSS6mNSLgtic+Ms4EGM82Zz+MZsTmyvn4xKy6HXSdrT+zSN25klHMnDPiy0KjaT ZAYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:references:in-reply-to:date:cc:to:from :subject:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Tf3+zuSM5JW34sSXOaUP8rjPs47MJWvFo8GAD6hRTy4=; b=yGGRp3A8uITvImk2rdz9XvQgH+BzUfc0E7v//2XXTF6my4nzmdjCI2bOZmxHAQEOxl xyDXab+wKOsW7p59AXKlUCFwLYG6cUbYOBMKRV8I5ESMD99aj+71t9LPC08e7YV1bXTt z5jHFO1ortiYkjvZ/Y8pR1c2xvFzZUcntmdOzFAanopFR5MHVB7hgNgSg6G/aIKWTI7H XSTjA8WvXULjnYgOSMvt6C1zxH8K5CIrWcLHZM2bBpUNPPhhSltOVEy+6GHeedN48kdR twf6TlX6dd/IgQXma1ZuVyfWAlGPnhfVawYudewMIEBRVXnLv7I2hzB9Hrn/Qni0v+vY ZwOA== X-Gm-Message-State: AO0yUKUe6YSlCyqhkhbkJbjErBw/rMutVQlCt/HUSphOVKOJf6kwvybg NdKI0FKWNJTtYkl7hWGAoXxcow== X-Google-Smtp-Source: AK7set9f7TLsCMTmyUb7U3DZIg4ZwtXnOg+m71udLfZ/r963Q0omjXuipzzEaBNNEOGHK4YiX61qQA== X-Received: by 2002:a17:90a:7f98:b0:230:f874:3600 with SMTP id m24-20020a17090a7f9800b00230f8743600mr7454762pjl.28.1675887408410; Wed, 08 Feb 2023 12:16:48 -0800 (PST) Received: from jeff-laptop.lan (c-73-231-146-4.hsd1.ca.comcast.net. [73.231.146.4]) by smtp.gmail.com with ESMTPSA id d23-20020a17090ad99700b00213202d77d9sm1950480pjv.43.2023.02.08.12.16.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Feb 2023 12:16:47 -0800 (PST) Message-ID: <337d581bae5d6253919f70acdfc184631f0ad41b.camel@j-davis.com> Subject: Re: Move defaults toward ICU in 16? From: Jeff Davis To: Tom Lane , Thomas Munro Cc: Robert Haas , pgsql-hackers@postgresql.org Date: Wed, 08 Feb 2023 12:16:46 -0800 In-Reply-To: <1590310.1675379449@sss.pgh.pa.us> References: <510d284759f6e943ce15096167760b2edcb2e700.camel@j-davis.com> <1590310.1675379449@sss.pgh.pa.us> Content-Type: multipart/mixed; boundary="=-7KdSVwPIaw2LGF8iQ5rO" User-Agent: Evolution 3.44.4-0ubuntu1 MIME-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --=-7KdSVwPIaw2LGF8iQ5rO Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, 2023-02-02 at 18:10 -0500, Tom Lane wrote: > Yeah.=C2=A0 I would be resistant to making ICU a required dependency, > but it doesn't seem unreasonable to start moving towards it being > our default collation support. Patch attached. To get the default locale, the patch initializes a UCollator with NULL for the locale name, and then queries it for the locale name. Then it's converted to a language tag, which is consistent with the initial collation import. I'm not sure that's the best way, but it seems reasonable. If it's a user-provided locale (--icu-locale=3D), then the patch leaves it as-is, and does not convert it to a language tag (consistent with CREATE COLLATION and CREATE DATABASE). I opened another discussion about whether we want to try harder to validate or canonicalize the locale name: https://www.postgresql.org/message-id/11b1eeb7e7667fdd4178497aeb796c48d26e6= 9b9.camel@j-davis.com --=20 Jeff Davis PostgreSQL Contributor Team - AWS --=-7KdSVwPIaw2LGF8iQ5rO Content-Disposition: attachment; filename="v1-0001-Use-ICU-by-default-at-initdb-time.patch" Content-Type: text/x-patch; name="v1-0001-Use-ICU-by-default-at-initdb-time.patch"; charset="UTF-8" Content-Transfer-Encoding: base64 RnJvbSAxYjdkOTQwYzBmMTIwNjIxODViOGI0MmJmOGQzYzBhNmYwNWE3NGQ0IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBKZWZmIERhdmlzIDxqZWZmQGotZGF2aXMuY29tPgpEYXRlOiBX ZWQsIDggRmViIDIwMjMgMTI6MDY6MjYgLTA4MDAKU3ViamVjdDogW1BBVENIIHYxXSBVc2UgSUNV IGJ5IGRlZmF1bHQgYXQgaW5pdGRiIHRpbWUuCgpJZiB0aGUgSUNVIGxvY2FsZSBpcyBub3Qgc3Bl Y2lmaWVkLCBpbml0aWFsaXplIHRoZSBkZWZhdWx0IGNvbGxhdG9yCmFuZCByZXRyaWV2ZSB0aGUg bG9jYWxlIG5hbWUgZnJvbSB0aGF0LgoKRGlzY3Vzc2lvbjogaHR0cHM6Ly9wb3N0Z3IuZXMvbS81 MTBkMjg0NzU5ZjZlOTQzY2UxNTA5NjE2Nzc2MGIyZWRjYjJlNzAwLmNhbWVsQGotZGF2aXMuY29t Ci0tLQogc3JjL2Jpbi9pbml0ZGIvaW5pdGRiLmMgfCA3NCArKysrKysrKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKystLQogMSBmaWxlIGNoYW5nZWQsIDcyIGluc2VydGlvbnMoKyksIDIg ZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvc3JjL2Jpbi9pbml0ZGIvaW5pdGRiLmMgYi9zcmMv YmluL2luaXRkYi9pbml0ZGIuYwppbmRleCA3YTU4YzMzYWNlLi43MzIxNjUyZGIzIDEwMDY0NAot LS0gYS9zcmMvYmluL2luaXRkYi9pbml0ZGIuYworKysgYi9zcmMvYmluL2luaXRkYi9pbml0ZGIu YwpAQCAtNTMsNiArNTMsNyBAQAogI2luY2x1ZGUgPG5ldGRiLmg+CiAjaW5jbHVkZSA8c3lzL3Nv Y2tldC5oPgogI2luY2x1ZGUgPHN5cy9zdGF0Lmg+CisjaW5jbHVkZSA8dW5pY29kZS91Y29sLmg+ CiAjaW5jbHVkZSA8dW5pc3RkLmg+CiAjaW5jbHVkZSA8c2lnbmFsLmg+CiAjaW5jbHVkZSA8dGlt ZS5oPgpAQCAtMTMzLDcgKzEzNCwxMSBAQCBzdGF0aWMgY2hhciAqbGNfbW9uZXRhcnkgPSBOVUxM Owogc3RhdGljIGNoYXIgKmxjX251bWVyaWMgPSBOVUxMOwogc3RhdGljIGNoYXIgKmxjX3RpbWUg PSBOVUxMOwogc3RhdGljIGNoYXIgKmxjX21lc3NhZ2VzID0gTlVMTDsKKyNpZmRlZiBVU0VfSUNV CitzdGF0aWMgY2hhciBsb2NhbGVfcHJvdmlkZXIgPSBDT0xMUFJPVklERVJfSUNVOworI2Vsc2UK IHN0YXRpYyBjaGFyIGxvY2FsZV9wcm92aWRlciA9IENPTExQUk9WSURFUl9MSUJDOworI2VuZGlm CiBzdGF0aWMgY2hhciAqaWN1X2xvY2FsZSA9IE5VTEw7CiBzdGF0aWMgY29uc3QgY2hhciAqZGVm YXVsdF90ZXh0X3NlYXJjaF9jb25maWcgPSBOVUxMOwogc3RhdGljIGNoYXIgKnVzZXJuYW1lID0g TlVMTDsKQEAgLTIwMjQsNiArMjAyOSw3MiBAQCBjaGVja19pY3VfbG9jYWxlX2VuY29kaW5nKGlu dCB1c2VyX2VuYykKIAlyZXR1cm4gdHJ1ZTsKIH0KIAorLyoKKyAqIENoZWNrIHRoYXQgSUNVIGFj Y2VwdHMgdGhlIGxvY2FsZSBuYW1lOyBvciBpZiBub3Qgc3BlY2lmaWVkLCByZXRyaWV2ZSB0aGUK KyAqIGRlZmF1bHQgSUNVIGxvY2FsZS4KKyAqLworc3RhdGljIHZvaWQKK2NoZWNrX2ljdV9sb2Nh bGUoKQoreworI2lmZGVmIFVTRV9JQ1UKKwlVQ29sbGF0b3IJKmNvbGxhdG9yOworCVVFcnJvckNv ZGUgICBzdGF0dXM7CisKKwlzdGF0dXMgPSBVX1pFUk9fRVJST1I7CisJY29sbGF0b3IgPSB1Y29s X29wZW4oaWN1X2xvY2FsZSwgJnN0YXR1cyk7CisJaWYgKFVfRkFJTFVSRShzdGF0dXMpKQorCXsK KwkJaWYgKGljdV9sb2NhbGUpCisJCQlwZ19mYXRhbCgiSUNVIGxvY2FsZSBcIiVzXCIgY291bGQg bm90IGJlIG9wZW5lZDogJXMiLAorCQkJCQkgaWN1X2xvY2FsZSwgdV9lcnJvck5hbWUoc3RhdHVz KSk7CisJCWVsc2UKKwkJCXBnX2ZhdGFsKCJkZWZhdWx0IElDVSBsb2NhbGUgY291bGQgbm90IGJl IG9wZW5lZDogJXMiLAorCQkJCQkgdV9lcnJvck5hbWUoc3RhdHVzKSk7CisJfQorCisJLyogaWYg bm90IHNwZWNpZmllZCwgZ2V0IGxvY2FsZSBmcm9tIGRlZmF1bHQgY29sbGF0b3IgKi8KKwlpZiAo aWN1X2xvY2FsZSA9PSBOVUxMKQorCXsKKwkJY29uc3QgY2hhcgkqZGVmYXVsdF9sb2NhbGU7CisK KwkJc3RhdHVzID0gVV9aRVJPX0VSUk9SOworCQlkZWZhdWx0X2xvY2FsZSA9IHVjb2xfZ2V0TG9j YWxlQnlUeXBlKGNvbGxhdG9yLCBVTE9DX1ZBTElEX0xPQ0FMRSwKKwkJCQkJCQkJCQkJICAmc3Rh dHVzKTsKKwkJaWYgKFVfRkFJTFVSRShzdGF0dXMpKQorCQl7CisJCQl1Y29sX2Nsb3NlKGNvbGxh dG9yKTsKKwkJCXBnX2ZhdGFsKCJjb3VsZCBub3QgZGV0ZXJtaW5lIGRlZmF1bHQgSUNVIGxvY2Fs ZSIpOworCQl9CisKKwkJaWYgKFVfSUNVX1ZFUlNJT05fTUFKT1JfTlVNID49IDU0KQorCQl7CisJ CQljb25zdCBib29sCSBzdHJpY3QgPSB0cnVlOworCQkJY2hhcgkJKmxhbmd0YWc7CisJCQlpbnQJ CQkgbGVuOworCisJCQlsZW4gPSB1bG9jX3RvTGFuZ3VhZ2VUYWcoZGVmYXVsdF9sb2NhbGUsIE5V TEwsIDAsIHN0cmljdCwgJnN0YXR1cyk7CisJCQlsYW5ndGFnID0gcGdfbWFsbG9jKGxlbiArIDEp OworCQkJc3RhdHVzID0gVV9aRVJPX0VSUk9SOworCQkJdWxvY190b0xhbmd1YWdlVGFnKGRlZmF1 bHRfbG9jYWxlLCBsYW5ndGFnLCBsZW4gKyAxLCBzdHJpY3QsCisJCQkJCQkJICAgJnN0YXR1cyk7 CisKKwkJCWlmIChVX0ZBSUxVUkUoc3RhdHVzKSkKKwkJCXsKKwkJCQl1Y29sX2Nsb3NlKGNvbGxh dG9yKTsKKwkJCQlwZ19mYXRhbCgiY291bGQgbm90IGRldGVybWluZSBsYW5ndWFnZSB0YWcgZm9y IGRlZmF1bHQgbG9jYWxlIFwiJXNcIjogJXMiLAorCQkJCQkJIGRlZmF1bHRfbG9jYWxlLCB1X2Vy cm9yTmFtZShzdGF0dXMpKTsKKwkJCX0KKworCQkJaWN1X2xvY2FsZSA9IGxhbmd0YWc7CisJCX0K KwkJZWxzZQorCQkJaWN1X2xvY2FsZSA9IHBnX3N0cmR1cChkZWZhdWx0X2xvY2FsZSk7CisJfQor CisJdWNvbF9jbG9zZShjb2xsYXRvcik7CisjZW5kaWYKK30KKwogLyoKICAqIHNldCB1cCB0aGUg bG9jYWxlIHZhcmlhYmxlcwogICoKQEAgLTIwNzcsOCArMjE0OCw3IEBAIHNldGxvY2FsZXModm9p ZCkKIAogCWlmIChsb2NhbGVfcHJvdmlkZXIgPT0gQ09MTFBST1ZJREVSX0lDVSkKIAl7Ci0JCWlm ICghaWN1X2xvY2FsZSkKLQkJCXBnX2ZhdGFsKCJJQ1UgbG9jYWxlIG11c3QgYmUgc3BlY2lmaWVk Iik7CisJCWNoZWNrX2ljdV9sb2NhbGUoKTsKIAogCQkvKgogCQkgKiBJbiBzdXBwb3J0ZWQgYnVp bGRzLCB0aGUgSUNVIGxvY2FsZSBJRCB3aWxsIGJlIGNoZWNrZWQgYnkgdGhlCi0tIAoyLjM0LjEK Cg== --=-7KdSVwPIaw2LGF8iQ5rO--