Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t0QO1-00BMQb-5M for pgsql-hackers@arkaria.postgresql.org; Mon, 14 Oct 2024 19:05:45 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1t0QNz-00GDVa-BW for pgsql-hackers@arkaria.postgresql.org; Mon, 14 Oct 2024 19:05:43 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t0QNy-00GDVP-VE for pgsql-hackers@lists.postgresql.org; Mon, 14 Oct 2024 19:05:43 +0000 Received: from momjian.us ([72.94.173.45]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t0QNv-0013NA-Ez; Mon, 14 Oct 2024 19:05:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=momjian.us; s=2024011501; h=In-Reply-To:Content-Transfer-Encoding:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-ID:Content-Description; bh=E9EKZ12ciToYXXJ9beNn2DbZQcNyASUetBZGlQoKt20=; b=JrSjn5uIgk+/8jAPsSkaR1sMNN WZJkiG4V6QyPGWeclleoYlcrI0pZXdL/PQHDJnBmVbKCpD0Es4yga78Cn78pvkwrgHWIL+SBmjcz1 +BCQtOTk9JCF63pdYNyG41hdDoq0J8nwhBFBFylLNQSP4XVGleF+5M+ZWhLMIqWZZ74OxPOdBmIJf ZcenHn/AOKnJeErEcStUutrHyy1nMDuypQEN3CnfMzkEBDjCYveSoVLSqw94h9dBzfK+05VFg2w31 xSBMjmVxTj9oGM/KeH6C8tEAkiNTPp8unua5YUaKrcgJ0pK0It4G4wwaMqcx96KY4uYEa+jTrH1bd p+MBBe0A==; Received: from bruce by momjian.us with local (Exim 4.96) (envelope-from ) id 1t0QNr-001CPT-0g; Mon, 14 Oct 2024 15:05:35 -0400 Date: Mon, 14 Oct 2024 15:05:35 -0400 From: Bruce Momjian To: Yugo NAGATA Cc: Tatsuo Ishii , tgl@sss.pgh.pa.us, daniel@yesql.se, pgsql-hackers@postgresql.org Subject: Re: Doc: typo in config.sgml Message-ID: References: <2565921.1728531809@sss.pgh.pa.us> <20241010.160041.949661508167788393.ishii@postgresql.org> <20241011114122.c90f8a871462da36f2e2afeb@sraoss.co.jp> <20241011.121650.2077607600187658573.ishii@postgresql.org> <20241011123653.09a67d3ad7b60a59e4558403@sraoss.co.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="V4gWu7IWEDnjv0PA" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20241011123653.09a67d3ad7b60a59e4558403@sraoss.co.jp> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --V4gWu7IWEDnjv0PA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Oct 11, 2024 at 12:36:53PM +0900, Yugo NAGATA wrote: > On Fri, 11 Oct 2024 12:16:50 +0900 (JST) > Tatsuo Ishii wrote: > > > > We can check non-ASCII letters SGML/XML files by preparing "allowlist" > > > that contains lines which are allowed to have non-ascii characters, > > > although this list will need to be maintained when lines in it are modified. > > > I've attached a patch to add a simple Perl script to do this. > > > > I doubt it really works. For example, nbsp can be used formatting > > (that's the purpose of the character in the first place). Whenever a > > developer decides to or not to use nbsp, "allowlist" needs to be > > maintained. It's too annoying. > > I suppose non-ascii characters including nbsp are basically disallowed, > so the allowlist will not increase unless there is some special reason. > > However, it is true that there might be a cost for maintaining the list > more or less, so if people don't think it is worth adding this check, > I will withdraw this proposal.l. I did some more research and we able to clarify our behavior in release.sgml: We can only use Latin1 characters, not all UTF8 characters, because rendering engines must support the referenced characters, and they currently only support Latin1. In the SGML files we encode non-ASCII Latin1 characters as HTML entities, e.g., Álvaro Herrera. Oddly, it is possible to add Latin1 characters as UTF8, but we we currently prevent this via the Makefile "check-non-ascii" check. We used to use UTF8 characters in SGML files, but only UTF8 characters that had Latin1 equivalents, and I think the toolchain would convert UTF8 to Latin1 for us. What I ended up doing was to change the UTF8 encoded characters to HTML entities, and then modify the Makefile to check for any non-ASCII characters. This will catch   and any other UTF8 characters. I also added a dummy 'pdf' target that is the same as the postgres.pdf dummy target; we already had an "html" target, so I thought a "pdf" one made sense. Patch attached. I plan to apply this in a few days to master. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com When a patient asks the doctor, "Am I going to die?", he means "Am I going to die soon?" --V4gWu7IWEDnjv0PA Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="latin1.diff" Content-Transfer-Encoding: 8bit diff --git a/doc/src/sgml/Makefile b/doc/src/sgml/Makefile index 65ed32cd0ab..87d21783e52 100644 --- a/doc/src/sgml/Makefile +++ b/doc/src/sgml/Makefile @@ -143,7 +143,7 @@ postgres.txt: postgres.html ## Print ## -postgres.pdf: +postgres.pdf pdf: $(error Invalid target; use postgres-A4.pdf or postgres-US.pdf as targets) XSLTPROC_FO_FLAGS += --stringparam img.src.path '$(srcdir)/' @@ -194,7 +194,7 @@ MAKEINFO = makeinfo ## # Quick syntax check without style processing -check: postgres.sgml $(ALLSGML) check-tabs check-nbsp +check: postgres.sgml $(ALLSGML) check-tabs check-non-ascii $(XMLLINT) $(XMLINCLUDE) --noout --valid $< @@ -262,10 +262,9 @@ check-tabs: # Non-breaking spaces are harmless, but it is best to avoid them in SGML files. # Use perl command because non-GNU grep or sed could not have hex escape sequence. -check-nbsp: - @ ( $(PERL) -ne '/\xC2\xA0/ and print("$$ARGV:$$_"),$$n++; END {exit($$n>0)}' \ - $(wildcard $(srcdir)/*.sgml $(srcdir)/ref/*.sgml $(srcdir)/*.xsl) ) || \ - (echo "Non-breaking spaces appear in SGML/XML files" 1>&2; exit 1) +check-non-ascii: + @( ! grep -P '[^\x00-\x7f]' $(wildcard $(srcdir)/*.sgml $(srcdir)/ref/*.sgml $(srcdir)/*.xsl) ) || \ + (echo "Non-ASCII characters appear in SGML/XML files; use HTML entities for Latin1 characters" 1>&2; exit 1) ## ## Clean diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml index 1ef5322b912..f5e115e8d6e 100644 --- a/doc/src/sgml/charset.sgml +++ b/doc/src/sgml/charset.sgml @@ -1225,7 +1225,7 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr -- ignore differences in accents and case CREATE COLLATION ignore_accent_case (provider = icu, deterministic = false, locale = 'und-u-ks-level1'); -SELECT 'Å' = 'A' COLLATE ignore_accent_case; -- true +SELECT 'Å' = 'A' COLLATE ignore_accent_case; -- true SELECT 'z' = 'Z' COLLATE ignore_accent_case; -- true -- upper case letters sort before lower case. @@ -1282,7 +1282,7 @@ SELECT 'w;x*y-z' = 'wxyz' COLLATE num_ignore_punct; -- true 'ab' = U&'a\2063b' 'x-y' = 'x_y' 'g' = 'G' - 'n' = 'ñ' + 'n' = 'ñ' 'y' = 'z' @@ -1346,7 +1346,7 @@ SELECT 'w;x*y-z' = 'wxyz' COLLATE num_ignore_punct; -- true At every level, even with full normalization off, basic normalization is - performed. For example, 'á' may be composed of the + performed. For example, 'á' may be composed of the code points U&'\0061\0301' or the single code point U&'\00E1', and those sequences will be considered equal even at the identic level. To treat @@ -1430,8 +1430,8 @@ SELECT 'x-y' = 'x_y' COLLATE level4; -- false false Backwards comparison for the level 2 differences. For example, - locale und-u-kb sorts 'àe' - before 'aé'. + locale und-u-kb sorts 'àe' + before 'aé'. diff --git a/doc/src/sgml/images/genetic-algorithm.svg b/doc/src/sgml/images/genetic-algorithm.svg index fb9fdd1ba78..2ce5f1b2712 100644 --- a/doc/src/sgml/images/genetic-algorithm.svg +++ b/doc/src/sgml/images/genetic-algorithm.svg @@ -72,7 +72,7 @@ a4->end -true   +true @@ -85,7 +85,7 @@ a4->a5 -false    +false diff --git a/doc/src/sgml/release.sgml b/doc/src/sgml/release.sgml index 8433690dead..65c86f54c0e 100644 --- a/doc/src/sgml/release.sgml +++ b/doc/src/sgml/release.sgml @@ -26,13 +26,15 @@ non-ASCII characters find using grep -P '[\x80-\xFF]' or http://www.zipcon.net/~swhite/docs/computers/browsers/entities_page.html https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references - We cannot use UTF8 because rendering engines have to - support the referenced characters. - - Do not use numeric _UTF_ numeric character escapes (&#nnn;), - we can only use Latin1. - - Example: Alvaro Herrera is Álvaro Herrera + We can only use Latin1 characters, not all UTF8 characters, + because rendering engines must support the referenced characters, + and they currently only support Latin1. In the SGML files we + encode non-ASCII Latin1 characters as HTML entities, e.g., + Álvaro Herrera. Oddly, it is possible to add Latin1 + characters as UTF8, but we we currently prevent this via the + Makefile "check-non-ascii" check. + + Do not use numeric _UTF_ numeric character escapes (&#nnn;). wrap long lines diff --git a/doc/src/sgml/stylesheet-man.xsl b/doc/src/sgml/stylesheet-man.xsl index fcb485c2931..2e2564da683 100644 --- a/doc/src/sgml/stylesheet-man.xsl +++ b/doc/src/sgml/stylesheet-man.xsl @@ -213,12 +213,12 @@ - - - - - - + + + + + + --V4gWu7IWEDnjv0PA--