Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t0kof-00D1f6-Ay for pgsql-hackers@arkaria.postgresql.org; Tue, 15 Oct 2024 16:54:37 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1t0kod-008pcL-GM for pgsql-hackers@arkaria.postgresql.org; Tue, 15 Oct 2024 16:54:35 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t0koc-008pbq-UQ for pgsql-hackers@lists.postgresql.org; Tue, 15 Oct 2024 16:54:35 +0000 Received: from momjian.us ([72.94.173.45]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t0koZ-0012BD-Sl; Tue, 15 Oct 2024 16:54:34 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=momjian.us; s=2024011501; h=In-Reply-To:Content-Transfer-Encoding:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-ID:Content-Description; bh=uR0nOXGQ6qqrCfKH4NNbaZS8BVYCsN0CIcnVGWLtkmY=; b=HhZQnkfch1dtaMODGEEahXwSAU G6G/5vxK7r0Y1FIaYTez800nGL5OR0rHpROhyzwVZj+ARPZq2QMVSdOp02iEeRz8KlEYVRyO9+L24 mks/oIe4R8dGslnTnV1dFvDVYFRQZXPQBxDoox+4lINjVPAMLgLSqos4u79eva/5xQXrIvPQw4/h2 vkxbOR1Mf3/l3ZNHdEB1v4cn+GpCwDgdDz4BeazSIzvs/PPvoP+YhDcphaGj+lAsBkcSjKoaSYwNs cpl+IqmxFBidNYvzVjAX95RtmT0lFMUbHgHiyFVb+eiIyKNngTDOZDcqH4NiHn26DhT8zmRI15hTf wSxVnclA==; Received: from bruce by momjian.us with local (Exim 4.96) (envelope-from ) id 1t0koV-004UlQ-2J; Tue, 15 Oct 2024 12:54:27 -0400 Date: Tue, 15 Oct 2024 12:54:27 -0400 From: Bruce Momjian To: Yugo NAGATA Cc: Tatsuo Ishii , tgl@sss.pgh.pa.us, daniel@yesql.se, pgsql-hackers@postgresql.org Subject: Re: Doc: typo in config.sgml Message-ID: References: <2565921.1728531809@sss.pgh.pa.us> <20241010.160041.949661508167788393.ishii@postgresql.org> <20241011114122.c90f8a871462da36f2e2afeb@sraoss.co.jp> <20241011.121650.2077607600187658573.ishii@postgresql.org> <20241011123653.09a67d3ad7b60a59e4558403@sraoss.co.jp> <20241015101036.d8c29e4aada1c3e92ddcef9d@sraoss.co.jp> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="8Cukh7IEshHLsd2y" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20241015101036.d8c29e4aada1c3e92ddcef9d@sraoss.co.jp> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --8Cukh7IEshHLsd2y Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Oct 15, 2024 at 10:10:36AM +0900, Yugo NAGATA wrote: > Hi Bruce, > > On Mon, 14 Oct 2024 16:31:11 -0400 > Bruce Momjian wrote: > > > On Mon, Oct 14, 2024 at 03:05:35PM -0400, Bruce Momjian wrote: > > > I did some more research and we able to clarify our behavior in > > > release.sgml: > > > > I have specified some more details in my patched version: > > > > We can only use Latin1 characters, not all UTF8 characters, > > because some rendering engines do not support non-Latin1 UTF8 > > characters. Specifically, the HTML rendering engine can display > > all UTF8 characters, but the PDF rendering engine can only display > > Latin1 characters. In PDF files, non-Latin1 UTF8 characters are > > displayed as "###". > > > > In the SGML files we encode non-ASCII Latin1 characters as HTML > > entities, e.g., Álvaro. Oddly, it is possible to safely > > represent Latin1 characters in SGML files as UTF8 for HTML and > > PDF output, but we we currently disallow this via the Makefile > > "check-non-ascii" rule. > > > > I agree with encoding non-Latin1 characters and disallowing non-ASCII > characters totally. > > I found your patch includes fixes in *.svg files, so how about checking > also them by check-non-ascii? Also, I think it is better to use perl instead > of grep because non-GNU grep doesn't support hex escape sequences. I've attached > a updated patch for Makefile. The changes in release.sgml above is not applied > yet, though. Yes, good idea on using Perl and checking svg files --- I have used your Makefile rule. Attached is an updated patch. I realized that the new rules apply to all SGML files, not just the release notes, so I have created README.non-ASCII and moved the description there. -- Bruce Momjian https://momjian.us EDB https://enterprisedb.com When a patient asks the doctor, "Am I going to die?", he means "Am I going to die soon?" --8Cukh7IEshHLsd2y Content-Type: text/x-diff; charset=utf-8 Content-Disposition: attachment; filename="latin1.diff" Content-Transfer-Encoding: 8bit diff --git a/doc/src/sgml/Makefile b/doc/src/sgml/Makefile index 65ed32cd0ab..a3ff1168729 100644 --- a/doc/src/sgml/Makefile +++ b/doc/src/sgml/Makefile @@ -143,11 +143,12 @@ postgres.txt: postgres.html ## Print ## -postgres.pdf: +postgres.pdf pdf: $(error Invalid target; use postgres-A4.pdf or postgres-US.pdf as targets) XSLTPROC_FO_FLAGS += --stringparam img.src.path '$(srcdir)/' +# XSL Formatting Objects (FO), https://en.wikipedia.org/wiki/XSL_Formatting_Objects %-A4.fo: stylesheet-fo.xsl %-full.xml $(XSLTPROC) $(XMLINCLUDE) $(XSLTPROCFLAGS) $(XSLTPROC_FO_FLAGS) --stringparam paper.type A4 -o $@ $^ @@ -194,7 +195,7 @@ MAKEINFO = makeinfo ## # Quick syntax check without style processing -check: postgres.sgml $(ALLSGML) check-tabs check-nbsp +check: postgres.sgml $(ALLSGML) check-tabs check-non-ascii $(XMLLINT) $(XMLINCLUDE) --noout --valid $< @@ -262,10 +263,10 @@ check-tabs: # Non-breaking spaces are harmless, but it is best to avoid them in SGML files. # Use perl command because non-GNU grep or sed could not have hex escape sequence. -check-nbsp: - @ ( $(PERL) -ne '/\xC2\xA0/ and print("$$ARGV:$$_"),$$n++; END {exit($$n>0)}' \ - $(wildcard $(srcdir)/*.sgml $(srcdir)/ref/*.sgml $(srcdir)/*.xsl) ) || \ - (echo "Non-breaking spaces appear in SGML/XML files" 1>&2; exit 1) +check-non-ascii: + @ ( $(PERL) -ne '/[^\x00-\x7f]/ and print("$$ARGV: $$_"), $$n++; END { exit($$n > 0) }' \ + $(wildcard $(srcdir)/*.sgml $(srcdir)/ref/*.sgml $(srcdir)/*.xsl $(srcdir)/images/*.svg) ) || \ + (echo "Non-ASCII characters appear in SGML/XML files; use HTML entities for Latin1 characters" 1>&2; exit 1) ## ## Clean diff --git a/doc/src/sgml/README.non-ASCII b/doc/src/sgml/README.non-ASCII new file mode 100644 index 00000000000..a7300bcb2d3 --- /dev/null +++ b/doc/src/sgml/README.non-ASCII @@ -0,0 +1,38 @@ + + +Representation of non-ASCII characters +-------------------------------------- + +Find non-ASCII characters using: + + grep --color='auto' -P "[\x80-\xFF]" + +Convert to HTML4 named entity (&) escapes +----------------------------------------- + +We support several output formats: + +* html (supports all Unicode characters) +* man (supports all Unicode characters) +* pdf (supports only Latin-1 characters) +* info + +While some output formatting tools support all Unicode characters, +others only support Latin-1 characters. Specifically, the PDF rendering +engine can only display Latin-1 characters; non-Latin-1 Unicode +characters are displayed as "###". + +Therefore, in the SGML files, we only use Latin-1 characters. We encode +these characters as HTML entities, e.g., Álvaro. Oddly, in SGML +files it is possible to safely represent Latin-1 characters in UTF8 +encoding for all output formats, but we we currently disallow this via +the Makefile rule "check-non-ascii". + +Do not use UTF numeric character escapes (&#nnn;). + +HTML entities + official: http://www.w3.org/TR/html4/sgml/entities.html + one page: http://www.zipcon.net/~swhite/docs/computers/browsers/entities_page.html + other lists: http://www.zipcon.net/~swhite/docs/computers/browsers/entities.html + http://www.zipcon.net/~swhite/docs/computers/browsers/entities_page.html + https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml index 1ef5322b912..f5e115e8d6e 100644 --- a/doc/src/sgml/charset.sgml +++ b/doc/src/sgml/charset.sgml @@ -1225,7 +1225,7 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr -- ignore differences in accents and case CREATE COLLATION ignore_accent_case (provider = icu, deterministic = false, locale = 'und-u-ks-level1'); -SELECT 'Å' = 'A' COLLATE ignore_accent_case; -- true +SELECT 'Å' = 'A' COLLATE ignore_accent_case; -- true SELECT 'z' = 'Z' COLLATE ignore_accent_case; -- true -- upper case letters sort before lower case. @@ -1282,7 +1282,7 @@ SELECT 'w;x*y-z' = 'wxyz' COLLATE num_ignore_punct; -- true 'ab' = U&'a\2063b' 'x-y' = 'x_y' 'g' = 'G' - 'n' = 'ñ' + 'n' = 'ñ' 'y' = 'z' @@ -1346,7 +1346,7 @@ SELECT 'w;x*y-z' = 'wxyz' COLLATE num_ignore_punct; -- true At every level, even with full normalization off, basic normalization is - performed. For example, 'á' may be composed of the + performed. For example, 'á' may be composed of the code points U&'\0061\0301' or the single code point U&'\00E1', and those sequences will be considered equal even at the identic level. To treat @@ -1430,8 +1430,8 @@ SELECT 'x-y' = 'x_y' COLLATE level4; -- false false Backwards comparison for the level 2 differences. For example, - locale und-u-kb sorts 'àe' - before 'aé'. + locale und-u-kb sorts 'àe' + before 'aé'. diff --git a/doc/src/sgml/images/genetic-algorithm.svg b/doc/src/sgml/images/genetic-algorithm.svg index fb9fdd1ba78..2ce5f1b2712 100644 --- a/doc/src/sgml/images/genetic-algorithm.svg +++ b/doc/src/sgml/images/genetic-algorithm.svg @@ -72,7 +72,7 @@ a4->end -true   +true @@ -85,7 +85,7 @@ a4->a5 -false    +false diff --git a/doc/src/sgml/release.sgml b/doc/src/sgml/release.sgml index 8433690dead..cee577ff8d3 100644 --- a/doc/src/sgml/release.sgml +++ b/doc/src/sgml/release.sgml @@ -16,24 +16,6 @@ pg_[A-Za-z0-9_]+ , \<[a-z]+_[a-z_]+\> , -non-ASCII characters find using grep -P '[\x80-\xFF]' or - (remove 'X') grep -X-color='auto' -P -n "[\x80-\xFF]" - convert to HTML4 named entity (&) escapes - - official: http://www.w3.org/TR/html4/sgml/entities.html - one page: http://www.zipcon.net/~swhite/docs/computers/browsers/entities_page.html - other lists: http://www.zipcon.net/~swhite/docs/computers/browsers/entities.html - http://www.zipcon.net/~swhite/docs/computers/browsers/entities_page.html - https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references - - We cannot use UTF8 because rendering engines have to - support the referenced characters. - - Do not use numeric _UTF_ numeric character escapes (&#nnn;), - we can only use Latin1. - - Example: Alvaro Herrera is Álvaro Herrera - wrap long lines For new features, add links to the documentation sections. diff --git a/doc/src/sgml/stylesheet-man.xsl b/doc/src/sgml/stylesheet-man.xsl index fcb485c2931..2e2564da683 100644 --- a/doc/src/sgml/stylesheet-man.xsl +++ b/doc/src/sgml/stylesheet-man.xsl @@ -213,12 +213,12 @@ - - - - - - + + + + + + --8Cukh7IEshHLsd2y--