pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix limitations on what SQL commands can be issued to a walsende 6+ messages / 1 participants [nested] [flat]
* pgsql: Fix limitations on what SQL commands can be issued to a walsende @ 2022-01-24 20:34 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-01-24 20:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix limitations on what SQL commands can be issued to a walsender. In logical replication mode, a WalSender is supposed to be able to execute any regular SQL command, as well as the special replication commands. Poor design of the replication-command parser caused it to fail in various cases, notably: * semicolons embedded in a command, or multiple SQL commands sent in a single message; * dollar-quoted literals containing odd numbers of single or double quote marks; * commands starting with a comment. The basic problem here is that we're trying to run repl_scanner.l across the entire input string even when it's not a replication command. Since repl_scanner.l does not understand all of the token types known to the core lexer, this is doomed to have failure modes. We certainly don't want to make repl_scanner.l as big as scan.l, so instead rejigger stuff so that we only lex the first token of a non-replication command. That will usually look like an IDENT to repl_scanner.l, though a comment would end up getting reported as a '-' or '/' single-character token. If the token is a replication command keyword, we push it back and proceed normally with repl_gram.y parsing. Otherwise, we can drop out of exec_replication_command() without examining the rest of the string. (It's still theoretically possible for repl_scanner.l to fail on the first token; but that could only happen if it's an unterminated single- or double-quoted string, in which case you'd have gotten largely the same error from the core lexer too.) In this way, repl_gram.y isn't involved at all in handling general SQL commands, so we can get rid of the SQLCmd node type. (In the back branches, we can't remove it because renumbering enum NodeTag would be an ABI break; so just leave it sit there unused.) I failed to resist the temptation to clean up some other sloppy coding in repl_scanner.l while at it. The only externally-visible behavior change from that is it now accepts \r and \f as whitespace, same as the core lexer. Per bug #17379 from Greg Rychlewski. Back-patch to all supported branches. Discussion: https://postgr.es/m/17379-6a5c6cfb3f1f5e77@postgresql.org Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/6aa5186146f15f8b78749b52caee1eabb3a1aa92 Modified Files -------------- src/backend/replication/repl_gram.y | 26 +-------- src/backend/replication/repl_scanner.l | 87 +++++++++++++++++++++++------ src/backend/replication/walsender.c | 43 ++++++++------ src/include/nodes/nodes.h | 1 - src/include/nodes/replnodes.h | 9 --- src/include/replication/walsender_private.h | 1 + 6 files changed, 98 insertions(+), 69 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix limitations on what SQL commands can be issued to a walsende @ 2022-01-24 20:34 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-01-24 20:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix limitations on what SQL commands can be issued to a walsender. In logical replication mode, a WalSender is supposed to be able to execute any regular SQL command, as well as the special replication commands. Poor design of the replication-command parser caused it to fail in various cases, notably: * semicolons embedded in a command, or multiple SQL commands sent in a single message; * dollar-quoted literals containing odd numbers of single or double quote marks; * commands starting with a comment. The basic problem here is that we're trying to run repl_scanner.l across the entire input string even when it's not a replication command. Since repl_scanner.l does not understand all of the token types known to the core lexer, this is doomed to have failure modes. We certainly don't want to make repl_scanner.l as big as scan.l, so instead rejigger stuff so that we only lex the first token of a non-replication command. That will usually look like an IDENT to repl_scanner.l, though a comment would end up getting reported as a '-' or '/' single-character token. If the token is a replication command keyword, we push it back and proceed normally with repl_gram.y parsing. Otherwise, we can drop out of exec_replication_command() without examining the rest of the string. (It's still theoretically possible for repl_scanner.l to fail on the first token; but that could only happen if it's an unterminated single- or double-quoted string, in which case you'd have gotten largely the same error from the core lexer too.) In this way, repl_gram.y isn't involved at all in handling general SQL commands, so we can get rid of the SQLCmd node type. (In the back branches, we can't remove it because renumbering enum NodeTag would be an ABI break; so just leave it sit there unused.) I failed to resist the temptation to clean up some other sloppy coding in repl_scanner.l while at it. The only externally-visible behavior change from that is it now accepts \r and \f as whitespace, same as the core lexer. Per bug #17379 from Greg Rychlewski. Back-patch to all supported branches. Discussion: https://postgr.es/m/17379-6a5c6cfb3f1f5e77@postgresql.org Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/1efcc5946d595a168a1abf737030f2e0ee1f4d43 Modified Files -------------- src/backend/replication/repl_gram.y | 25 +-------- src/backend/replication/repl_scanner.l | 86 +++++++++++++++++++++++------ src/backend/replication/walsender.c | 43 +++++++++------ src/include/replication/walsender_private.h | 1 + 4 files changed, 97 insertions(+), 58 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix limitations on what SQL commands can be issued to a walsende @ 2022-01-24 20:34 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-01-24 20:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix limitations on what SQL commands can be issued to a walsender. In logical replication mode, a WalSender is supposed to be able to execute any regular SQL command, as well as the special replication commands. Poor design of the replication-command parser caused it to fail in various cases, notably: * semicolons embedded in a command, or multiple SQL commands sent in a single message; * dollar-quoted literals containing odd numbers of single or double quote marks; * commands starting with a comment. The basic problem here is that we're trying to run repl_scanner.l across the entire input string even when it's not a replication command. Since repl_scanner.l does not understand all of the token types known to the core lexer, this is doomed to have failure modes. We certainly don't want to make repl_scanner.l as big as scan.l, so instead rejigger stuff so that we only lex the first token of a non-replication command. That will usually look like an IDENT to repl_scanner.l, though a comment would end up getting reported as a '-' or '/' single-character token. If the token is a replication command keyword, we push it back and proceed normally with repl_gram.y parsing. Otherwise, we can drop out of exec_replication_command() without examining the rest of the string. (It's still theoretically possible for repl_scanner.l to fail on the first token; but that could only happen if it's an unterminated single- or double-quoted string, in which case you'd have gotten largely the same error from the core lexer too.) In this way, repl_gram.y isn't involved at all in handling general SQL commands, so we can get rid of the SQLCmd node type. (In the back branches, we can't remove it because renumbering enum NodeTag would be an ABI break; so just leave it sit there unused.) I failed to resist the temptation to clean up some other sloppy coding in repl_scanner.l while at it. The only externally-visible behavior change from that is it now accepts \r and \f as whitespace, same as the core lexer. Per bug #17379 from Greg Rychlewski. Back-patch to all supported branches. Discussion: https://postgr.es/m/17379-6a5c6cfb3f1f5e77@postgresql.org Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/d67354d870bbddbde551e5b9804a42fb78af527e Modified Files -------------- src/backend/replication/repl_gram.y | 25 +-------- src/backend/replication/repl_scanner.l | 86 +++++++++++++++++++++++------ src/backend/replication/walsender.c | 43 +++++++++------ src/include/replication/walsender_private.h | 1 + 4 files changed, 97 insertions(+), 58 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix limitations on what SQL commands can be issued to a walsende @ 2022-01-24 20:34 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-01-24 20:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix limitations on what SQL commands can be issued to a walsender. In logical replication mode, a WalSender is supposed to be able to execute any regular SQL command, as well as the special replication commands. Poor design of the replication-command parser caused it to fail in various cases, notably: * semicolons embedded in a command, or multiple SQL commands sent in a single message; * dollar-quoted literals containing odd numbers of single or double quote marks; * commands starting with a comment. The basic problem here is that we're trying to run repl_scanner.l across the entire input string even when it's not a replication command. Since repl_scanner.l does not understand all of the token types known to the core lexer, this is doomed to have failure modes. We certainly don't want to make repl_scanner.l as big as scan.l, so instead rejigger stuff so that we only lex the first token of a non-replication command. That will usually look like an IDENT to repl_scanner.l, though a comment would end up getting reported as a '-' or '/' single-character token. If the token is a replication command keyword, we push it back and proceed normally with repl_gram.y parsing. Otherwise, we can drop out of exec_replication_command() without examining the rest of the string. (It's still theoretically possible for repl_scanner.l to fail on the first token; but that could only happen if it's an unterminated single- or double-quoted string, in which case you'd have gotten largely the same error from the core lexer too.) In this way, repl_gram.y isn't involved at all in handling general SQL commands, so we can get rid of the SQLCmd node type. (In the back branches, we can't remove it because renumbering enum NodeTag would be an ABI break; so just leave it sit there unused.) I failed to resist the temptation to clean up some other sloppy coding in repl_scanner.l while at it. The only externally-visible behavior change from that is it now accepts \r and \f as whitespace, same as the core lexer. Per bug #17379 from Greg Rychlewski. Back-patch to all supported branches. Discussion: https://postgr.es/m/17379-6a5c6cfb3f1f5e77@postgresql.org Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/689f75d6eb9a1bc09f5f93b7aaaa40357e8d9195 Modified Files -------------- src/backend/replication/repl_gram.y | 25 +-------- src/backend/replication/repl_scanner.l | 86 +++++++++++++++++++++++------ src/backend/replication/walsender.c | 43 +++++++++------ src/include/replication/walsender_private.h | 1 + 4 files changed, 97 insertions(+), 58 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix limitations on what SQL commands can be issued to a walsende @ 2022-01-24 20:34 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-01-24 20:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix limitations on what SQL commands can be issued to a walsender. In logical replication mode, a WalSender is supposed to be able to execute any regular SQL command, as well as the special replication commands. Poor design of the replication-command parser caused it to fail in various cases, notably: * semicolons embedded in a command, or multiple SQL commands sent in a single message; * dollar-quoted literals containing odd numbers of single or double quote marks; * commands starting with a comment. The basic problem here is that we're trying to run repl_scanner.l across the entire input string even when it's not a replication command. Since repl_scanner.l does not understand all of the token types known to the core lexer, this is doomed to have failure modes. We certainly don't want to make repl_scanner.l as big as scan.l, so instead rejigger stuff so that we only lex the first token of a non-replication command. That will usually look like an IDENT to repl_scanner.l, though a comment would end up getting reported as a '-' or '/' single-character token. If the token is a replication command keyword, we push it back and proceed normally with repl_gram.y parsing. Otherwise, we can drop out of exec_replication_command() without examining the rest of the string. (It's still theoretically possible for repl_scanner.l to fail on the first token; but that could only happen if it's an unterminated single- or double-quoted string, in which case you'd have gotten largely the same error from the core lexer too.) In this way, repl_gram.y isn't involved at all in handling general SQL commands, so we can get rid of the SQLCmd node type. (In the back branches, we can't remove it because renumbering enum NodeTag would be an ABI break; so just leave it sit there unused.) I failed to resist the temptation to clean up some other sloppy coding in repl_scanner.l while at it. The only externally-visible behavior change from that is it now accepts \r and \f as whitespace, same as the core lexer. Per bug #17379 from Greg Rychlewski. Back-patch to all supported branches. Discussion: https://postgr.es/m/17379-6a5c6cfb3f1f5e77@postgresql.org Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/4ec54498c5eae338bbc8ce391df4297c314624ff Modified Files -------------- src/backend/replication/repl_gram.y | 25 +-------- src/backend/replication/repl_scanner.l | 86 +++++++++++++++++++++++------ src/backend/replication/walsender.c | 43 +++++++++------ src/include/replication/walsender_private.h | 1 + 4 files changed, 97 insertions(+), 58 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Fix limitations on what SQL commands can be issued to a walsende @ 2022-01-24 20:34 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2022-01-24 20:34 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix limitations on what SQL commands can be issued to a walsender. In logical replication mode, a WalSender is supposed to be able to execute any regular SQL command, as well as the special replication commands. Poor design of the replication-command parser caused it to fail in various cases, notably: * semicolons embedded in a command, or multiple SQL commands sent in a single message; * dollar-quoted literals containing odd numbers of single or double quote marks; * commands starting with a comment. The basic problem here is that we're trying to run repl_scanner.l across the entire input string even when it's not a replication command. Since repl_scanner.l does not understand all of the token types known to the core lexer, this is doomed to have failure modes. We certainly don't want to make repl_scanner.l as big as scan.l, so instead rejigger stuff so that we only lex the first token of a non-replication command. That will usually look like an IDENT to repl_scanner.l, though a comment would end up getting reported as a '-' or '/' single-character token. If the token is a replication command keyword, we push it back and proceed normally with repl_gram.y parsing. Otherwise, we can drop out of exec_replication_command() without examining the rest of the string. (It's still theoretically possible for repl_scanner.l to fail on the first token; but that could only happen if it's an unterminated single- or double-quoted string, in which case you'd have gotten largely the same error from the core lexer too.) In this way, repl_gram.y isn't involved at all in handling general SQL commands, so we can get rid of the SQLCmd node type. (In the back branches, we can't remove it because renumbering enum NodeTag would be an ABI break; so just leave it sit there unused.) I failed to resist the temptation to clean up some other sloppy coding in repl_scanner.l while at it. The only externally-visible behavior change from that is it now accepts \r and \f as whitespace, same as the core lexer. Per bug #17379 from Greg Rychlewski. Back-patch to all supported branches. Discussion: https://postgr.es/m/17379-6a5c6cfb3f1f5e77@postgresql.org Branch ------ REL_10_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/9af6d4b5a588fc115bc4915b8920b82f724387d1 Modified Files -------------- src/backend/replication/repl_gram.y | 25 +-------- src/backend/replication/repl_scanner.l | 86 +++++++++++++++++++++++------ src/backend/replication/walsender.c | 43 +++++++++------ src/include/replication/walsender_private.h | 1 + 4 files changed, 97 insertions(+), 58 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2022-01-24 20:34 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2022-01-24 20:34 pgsql: Fix limitations on what SQL commands can be issued to a walsende Tom Lane <tgl@sss.pgh.pa.us> 2022-01-24 20:34 pgsql: Fix limitations on what SQL commands can be issued to a walsende Tom Lane <tgl@sss.pgh.pa.us> 2022-01-24 20:34 pgsql: Fix limitations on what SQL commands can be issued to a walsende Tom Lane <tgl@sss.pgh.pa.us> 2022-01-24 20:34 pgsql: Fix limitations on what SQL commands can be issued to a walsende Tom Lane <tgl@sss.pgh.pa.us> 2022-01-24 20:34 pgsql: Fix limitations on what SQL commands can be issued to a walsende Tom Lane <tgl@sss.pgh.pa.us> 2022-01-24 20:34 pgsql: Fix limitations on what SQL commands can be issued to a walsende Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox